> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/resilience-testing/chaos-engineering/use-chaos-engineering/probes/apm-probes/splunk-probe.md).

# Splunk observability probe

Features and specification of the Splunk observability probe

The Splunk Observability probe allows you to query Splunk Observability(Signalfx) metrics and compare the results against specified criteria.

#### Prerequisites <a href="#prerequisites" id="prerequisites"></a>

To use the Splunk Observability probe, you need:

* An active Splunk Observability(signalfx) account
* Access to the Splunk Observability API from the kubernetes execution plane
* An API token for authentication

#### Interactive Setup Guide <a href="#interactive-setup-guide" id="interactive-setup-guide"></a>

Follow along with this interactive guide to learn how to configure Splunk Observability probe:

{% embed url="<https://app.tango.us/app/embed/e1f8d801-e55a-4e1d-a831-75b1d1a48142?skipCover=false&defaultListView=false&skipBranding=false&makeViewOnly=false&hideAuthorAndDetails=true>" %}
Create Splunk Observability APM Probe
{% endembed %}

#### Steps to configure <a href="#steps-to-configure" id="steps-to-configure"></a>

1. In Resilience probe section under chaos module, click on **New Probe** button

   ![Create Splunk Probe](https://2284173296-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FlSkpbpeYJ3rfGUkIrcyQ%2Fuploads%2Fgit-blob-466d102092ac9ed522d10ccaf4721e57421af017%2Fcreate-splunk-probe.png?alt=media)
2. Select the **APM Probe**
3. Provide the name of the probe and select **Splunk Observability** under APM Type

   ![Select Splunk Observability Probe](https://2284173296-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FlSkpbpeYJ3rfGUkIrcyQ%2Fuploads%2Fgit-blob-6197293d72af9215be1a0ff3c65dfcfdfe4ef39b%2Fselect-splunk-probe.png?alt=media)
4. Under Splunk Observability connector select connector
5. In Connector settings, you can either choose an existing connector or click **New Connector**

   ![Create Splunk Connector](https://2284173296-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FlSkpbpeYJ3rfGUkIrcyQ%2Fuploads%2Fgit-blob-78bdb5e6b88dcf9e0e4c3ae6e1327a85f6396bc0%2Fcreate-splunk-connector.png?alt=media)
6. Provide the credentials of the Splunk Observability

   ![Splunk Credentials](https://2284173296-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FlSkpbpeYJ3rfGUkIrcyQ%2Fuploads%2Fgit-blob-944e6ffdfd5431661fa944c80c55fe65edaa9057%2Fsplunk-credentials.png?alt=media)
7. Select the delegate and verify the connection and click on Finish

   ![Delegate](https://2284173296-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FlSkpbpeYJ3rfGUkIrcyQ%2Fuploads%2Fgit-blob-f67f4608e3b8ecdf3fd6d78f93f5c6bf99369628%2Fdelegate.png?alt=media)
8. Now connector is created and selected, click on Configure Details

   ![Configure Details](https://2284173296-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FlSkpbpeYJ3rfGUkIrcyQ%2Fuploads%2Fgit-blob-83b6ac0ca4bd93e2a89813781750c082dccf716d%2Fconfigure-details.png?alt=media)
9. Under Probe Properties, Pass the value of Splunk Observability Query and Lookback Window

   * **Splunk Observability Query**:
     * The Splunk Observability Query input is a string that specifies the search criteria for the metric time series (MTS) you want to retrieve. It follows a specific syntax that allows you to search for metrics, dimensions, properties, and tags
     * **Example query**: `sf_metric:cpu.utilization AND host.name:gke-default-pool-667be17c-t588.c.test.internal`
     * For more details refer to query section in [Splunk Observability doc](https://dev.splunk.com/observability/reference/api/retrieve_timeserieswindow/latest)
   * **LookBack Window (In Minutes)**:
     * The lookback window refers to the time range from a specified number of minutes ago up to the current moment, over which data is aggregated

   ![Lookback Window](https://2284173296-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FlSkpbpeYJ3rfGUkIrcyQ%2Fuploads%2Fgit-blob-0312289977f07dc4ca6a92aa28594b6d7a33287a%2Flookback-window.png?alt=media)
10. Provide the comparison criteria under Splunk Observability Data Comparison

    ![Splunk Data Comparison](https://2284173296-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FlSkpbpeYJ3rfGUkIrcyQ%2Fuploads%2Fgit-blob-dfb51da6c6b09a3a5109d3aaea40fff7fc9e1787%2Fsplunk-data-comparison.png?alt=media)
11. Provide the Run Properties

    ![Run Properties](https://2284173296-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FlSkpbpeYJ3rfGUkIrcyQ%2Fuploads%2Fgit-blob-c606a8e1e6959c248708794284e6e310750a2989%2Fcreate-probe.png?alt=media)
12. Then click on **Create Probe**

{% @harness-feedback/feedback %}