Splunk Observability (SignalFx)

Learn how to set up Splunk Observability (SignalFX) as a health source for CV

IMPORTANT

When using Splunk Observability [SignalFx] health source, it is not possible to derive a metric by using a combination of other metrics.

Before you begin

This page assumes you have followed the rest of the steps to set up CV. To learn more, go to Configure CV

Ensure that Splunk Observability [SignalFx] is added as a verification provider in Harness. To learn how to add Splunk Observability [SignalFx] as a connector in Harness, go to Add Splunk Observability [SignalFX].

Add Splunk Observability (SignalFx)

To add a health source:

1. In the Health Sources section, select + Add New Health Source.

   The Add New Health Source dialog appears.

2. On the Define Health Source tab, do the following:

   1. In the Define Health Source section, select Splunk Observability [SignalFx] as the health source type.

   2. In the Health Source Name field, enter a name for the health source.

   3. In the Connect Health Source section, select Select Connector. The Create or Select an Existing Connector dialog appears.

   4. Select a connector for the Splunk Observability [SignalFx] health source, and then select Apply Selected. The selected connector appears in the Select Connector dropdown.

   5. Select Next.

      The Configuration tab appears.

NOTE

Currently, Harness supports only Splunk Observability [SignalFx] metrics. The Splunk Observability [SignalFx] Metrics option is selected by default in the Select Feature field.

Define log configuration settings

1. On the Configuration tab, select + Add Metric. The Add Metric dialog appears.

2. Enter the following information, and then select Submit:

   • Metric name: Enter a name for the metric. For example, Memory Metric.

   • Group name: If the group to which you want to add the metric already exists, select it. If you want to create a new group, select + Add New. In the Add Group Name dialog enter a group name, and then select Submit.

3. In the Add Metric dialog, select Submit. New group and metric are created. The query specifications and mapping settings are displayed. These settings help you get the desired metric data from the Splunk Observability [SignalFx] platform and map it to Harness service. To learn about Splunk Observability [SignalFx] metrics and queries, go to https://dev.splunk.com/observability/docs/signalflow.

Define a query

In the Query box, enter your metric query, and then select Run Query. Sample data is displayed in the Records box. The Chart box displays the graph corresponding to the sample data. This helps you verify if the query that you have built is correct.

Sample query to check the number of Kubernetes pods in ready state

Query: data("k8s.container.ready")

Query - disk usage

Assign services

In the Assign section, select the services to which you want to apply the Sumo Logic metric. Following options are available:

• Continuous Verification (Applied to the pipelines in the Continuous Deployment): Select this option to use the metric data in the Continuous Deployment pipeline to ensure that the deployed service is running safely and to perform automatic rollbacks. In addition, the metric will be used to apply Machine Learning in detecting and highlighting future deployment issues.

• Service Health: Select this option to use the metric data to track the changes in the health trend of your monitored service.

• Service Level Indicator (SLI): Select this option to use the metric data to measure the SLI and obtain the performance of the service.

Configure risk profile

If you select Continuous Verification (Applied to the pipelines in the Continuous Deployment) or Service Health, expand the section below and follow the instructions for configuring the risk profile.

Risk Profile settings

Risk Profile

The Risk Profile section is only visible if you have selected Continuous Verification (Applied to the pipelines in the Continuous Deployment) or Service Health in the Assign section.

1. Under Risk Category, select one of the following options:

• Errors

• Infrastructure

• Performance/Throughput

• Performance/Other

• Performance/Response Time

1. Under Deviation Compared To Baseline, select the following settings to measure your service's behavior and calculate deviations from the health source:

• Higher counts = higher risk

• Lower counts = higher risk

  You can select multiple options.

Map service instance identifier

The Map service instance identifier section is visible only if you have selected Continuous Verification (Applied to the pipelines in the Continuous Deployment) in the Assign section.

In Service Instance Identifier (only needed for CV), specify the service instance identifier, which represents a dynamically created service that you deploy using Harness. The default value is _sourceHost.

Advanced (Optional)

The Advanced (Optional) section is only visible if you have selected Continuous Verification (Applied to the pipelines in the Continuous Deployment) in the Assign section.

Ignore Thresholds

You can select the types of events for which you want to set thresholds in CV. Metrics that match the selected rules will not be flagged as anomalous, regardless of the analysis.

To set the Ignore Thresholds for CV:

1. Go to the Ignore Thresholds tab and select the + Add Threshold button.

2. From the Metric dropdown, select the desired metric for which you want to set the rule.

3. In the Criteria field, choose the type of criteria you want to apply for the threshold:

• Absolute Value: Select this option and enter the Greater than and Lesser than values.

• Percentage Deviation: Select this option and enter the Lesser than value.

Fail-Fast Thresholds

You can select the type of events for which you want to set thresholds in CV. Any metric that matches the selected rules will be marked as anomalous and cause the Workflow state to fail.

To set fail-fast thresholds for CV, follow these steps:

1. Go to the Fail-Fast Thresholds tab and select the + Add Threshold button.

2. From the Metric dropdown, select the desired metric for which you want to set the rule.

3. In the Action field, select what the CV should do when applying the rule:

• Fail Immediately

• Fail after multiple occurrences

• Fail after consecutive occurrences

1. In the Count field, set the number of occurrences. This setting is only visible if you have selected Fail after multiple occurrences or Fail after consecutive occurrences in the Action field. The minimum value must be two.

2. In the Criteria field, choose the type of criteria you want to apply for the threshold:

• Absolute Value: Select this option and enter the Greater than and Lesser than values.

• Percentage Deviation: Select this option and enter the Lesser than value.