> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/continuous-delivery/use-continuous-delivery/verify-deployments/configure-cv/configure-verify-step-with-ai/health-sources/create-a-splunk-logs-health-source.md). # Create a Splunk Logs health source for AI Verify Connect Splunk to AI Verify (v2) to collect and analyze logs during deployments using manual SPL queries. AI Verify (v2) supports a Splunk Logs health source type that lets you connect a Splunk connector to your deployment pipeline and analyze logs during deployment verification. Splunk currently supports the **Logs** feature only; metrics-based analysis for Splunk is not yet available. You configure a Splunk Logs health source using **Manual Setup**, where you write the Splunk SPL query yourself and map the fields the AI analysis engine needs. ## What you will learn from this topic * **Create a Splunk Logs health source:** How to [configure a Splunk Logs health source](#create-a-splunk-logs-health-source), including the SPL query rules and the field mapping the AI analysis engine needs. * **Add multiple queries:** How to [add more than one query](#add-multiple-queries) to a single health source. * **Use the health source in a pipeline:** How to [reference the health source in an AI Verify (v2) step](#use-the-health-source-in-an-ai-verify-pipeline). * **View analysis results:** How to [read the verification verdict](#view-analysis-results) after a pipeline run. *** ## Before you begin * **Splunk connector:** You need a Splunk connector configured with access to the Splunk instance you want to query. Go to [Connect to monitoring and logging systems](https://developer.harness.io/harness-platform/use-harness-platform/connectors/monitoring-and-logging-systems/connect-to-monitoring-and-logging-systems) to create one. * **AI Verify step:** The health source must be referenced in an AI Verify (v2) step inside a Container Step Group. Go to [Configure the AI Verify (v2) step](/continuous-delivery/use-continuous-delivery/verify-deployments/configure-cv/configure-verify-step-with-ai/ai-verify.md) to set it up. *** ## Create a Splunk Logs health source Navigate to **Account/Org/Project Settings**, then select **Health Sources** from the left navigation. Click **+ New Health Source** to open the configuration wizard. {% stepper %} {% step %} **Configure the overview** On the **Overview** tab, provide the basic health source details.

Click to view full size image

1. Enter a **Health Source Name**, for example, `splunk-prod-logs`. 2. In **Select health source type**, select the **Splunk** tile. 3. In **Select Connector**, choose the Splunk connector that points to your Splunk instance. 4. In **Select Feature**, the dropdown shows **Logs** as the only option. Metrics analysis is not yet supported for Splunk. 5. Under **Setup Approach**, select **Manual Setup**. AI-assisted setup is not currently available for Splunk Logs health sources. 6. Click **Continue**. {% endstep %} {% step %} **Define the query specification** The **Query Specifications** tab is where you configure the SPL query that fetches logs from Splunk. Each health source can have one or more named queries. The left panel lists your queries. Click **+ Add Query** to add additional queries. Select a query in the list to edit its settings in the right panel. **Query Name and identifier** Enter a human-readable **Query Name**, for example, `Checkout Service Errors`. Harness derives the query identifier automatically. The identifier must be unique within the health source. **Select Query (Optional)** If you have existing saved searches in Splunk, use the **Select Query (Optional)** dropdown to choose one. Selecting a saved search populates the SPL query field with that search's query and resets the **Timestamp Identifier** and **Message Identifier** fields to their defaults. This step is optional; you can skip it and write an SPL query directly. **SPL query** Enter a Splunk SPL query that returns raw log events. The query must follow these rules: * Do not start the query with the `search` command. * Do not use transforming commands such as `stats`, `timechart`, `chart`, `top`, or `rare`. * Do not include `earliest=`, `latest=`, or `| head` in the query. The response must be an array of individual event objects, not aggregated results, so Harness can map identifiers from each event. Click **Fetch records** to run the query against your Splunk connector. Harness displays the returned events as a list of JSON objects in the **Records** section. {% endstep %} {% step %} **Map log fields** After the query returns records, map the three identifiers the AI analysis engine needs. Each field takes an absolute path from the root of a record, written in bracket notation: object keys in single quotes and array positions as numbers, with segments joined by dots. For example, `['_raw']`. | Field | Required | Default | Description | | ------------------------------- | -------- | ----------- | ------------------------------------------------------------------------- | | **Timestamp Identifier** | Yes | `['_time']` | Path to the event timestamp | | **Service Instance Identifier** | Yes | None | Path to the field that identifies which service instance produced the log | | **Message Identifier** | Yes | `['_raw']` | Path to the raw log message text | To select a path interactively, click the **+** icon next to a field. A path picker opens, showing the parsed record returned by **Fetch records**. Click any value in the tree to fill in its path. You can also type the path directly, but it must use the bracket notation shown above. {% endstep %} {% step %} **Save the health source** Click **Save** to create the health source. The health source appears on the **Health Sources** listing page with a **Log** signal-type badge and **Splunk** as the type. The health source is now ready to reference in an AI Verify (v2) step. {% endstep %} {% endstepper %} *** ## Add multiple queries Splunk Logs health sources support multiple queries within a single health source. You might add multiple queries to cover different log streams from the same Splunk connector, for example, one query for error logs and another for application logs. In the **Query Specifications** tab, click **+ Add Query** to add a new query entry in the left panel. Each query has its own name, SPL query, and field mapping. All queries in the health source run as part of the same AI Verify step execution. *** ## Use the health source in an AI Verify pipeline Reference the Splunk Logs health source by its identifier in the `AIVerifyNG` step. During execution, the step first fetches baseline logs using the saved SPL query, then fetches post-deployment (canary) logs during the deployment window, and submits both sets to the AI analysis engine for comparison.
Kubernetes rolling deployment with Splunk Logs health source: full YAML ```yaml pipeline: name: Splunk Logs E2E identifier: Splunk_Logs_E2E projectIdentifier: your_project orgIdentifier: default stages: - stage: name: test-stage identifier: teststage type: Deployment spec: deploymentType: Kubernetes service: serviceRef: your_service environment: environmentRef: your_environment deployToAll: false infrastructureDefinitions: - identifier: your_k8s_infrastructure execution: steps: - step: name: Rollout Deployment identifier: rolloutDeployment type: K8sRollingDeploy timeout: 10m spec: skipDryRun: false pruningEnabled: false - stepGroup: name: AI Verify identifier: aiVerifyGroup steps: - step: type: AIVerifyNG name: AIVerifyV2_1 identifier: AIVerifyV2_1 spec: dataCollectionWindow: 2m healthSources: - healthSourceRef: splunkprodlogsHS resources: limits: cpu: 250m memory: 256Mi timeout: 10m stepGroupInfra: type: KubernetesDirect spec: connectorRef: your_k8s_connector namespace: default rollbackSteps: - step: name: Rollback Rollout Deployment identifier: rollbackRolloutDeployment type: K8sRollingRollback timeout: 10m spec: pruningEnabled: false failureStrategies: - onFailure: errors: - AllErrors action: type: StageRollback ```
*** ## View analysis results After the pipeline runs, click the AI Verify step in the execution view. The **Analysis** tab shows a result for the Splunk Logs health source. Click the health source entry to see its **Query Summaries** section, which lists each query with a **healthy** or **unhealthy** badge. When a query is marked unhealthy, the analysis displays a summary of the detected anomalous log patterns and a root cause hypothesis. This output is generated by the AI analysis engine, which compares the baseline log window against the post-deployment log window. The **Logs** tab in the execution view shows the raw plugin output, including the baseline log fetch, the canary (post-deployment) log fetch, and the call to the AI Verify agent that produces the health summary. *** ## Next steps * [Configure the AI Verify (v2) step](/continuous-delivery/use-continuous-delivery/verify-deployments/configure-cv/configure-verify-step-with-ai/ai-verify.md): Add it to your pipeline and reference this health source. * [AI Verify (v2) overview](/continuous-delivery/use-continuous-delivery/verify-deployments/configure-cv/configure-verify-step-with-ai/overview.md): Understand how the AI analysis engine compares baseline and post-deployment data to produce a verdict. * [Interpret log results](/continuous-delivery/use-continuous-delivery/verify-deployments/cv-results/interpret-log-results.md): Read and act on the log analysis results from verification runs. {% @harness-feedback/feedback %}