> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/continuous-delivery/use-continuous-delivery/verify-deployments/configure-cv/configure-verify-step-with-ai/overview.md).

# AI Verify (v2) overview

AI-powered deployment verification without baseline configuration

{% hint style="info" %}
This feature is in Beta[^1] and requires enablement. [View all early access features.](/release-notes/features.md)

{% @harness-feature-flag-access/request-feature-flag-access module="continuous-delivery" pagePath="continuous-delivery/use-continuous-delivery/verify-deployments/configure-cv/configure-verify-step-with-ai/overview" featureFlags="%5B%7B%22name%22%3A%22Enable%20Health%20Sources%22%2C%22key%22%3A%22CV\_HEALTH\_SOURCES\_ENABLED%22%2C%22status%22%3A%22BETA%22%2C%22description%22%3A%22Enables%20Health%20Sources%20for%20AI%20Verify%20V2.%22%7D%2C%7B%22name%22%3A%22Enable%20AI%20Verify%22%2C%22key%22%3A%22CV\_AI\_VERIFY\_NG%22%2C%22status%22%3A%22BETA%22%2C%22description%22%3A%22Enables%20AI%20Verify%20V2%20for%20agentic%20verification.%22%7D%5D" %}
{% endhint %}

Traditional continuous verification requires extensive manual setup, creating monitored services, defining baseline periods, configuring threshold rules, and maintaining health checks. This overhead often delays adoption and requires ongoing maintenance as services evolve. AI-powered verification eliminates this complexity by using intelligent agents that automatically analyze deployment health without any baseline configuration.

## AI-powered verification components <a href="#what-is-ai-powered-verification" id="what-is-ai-powered-verification"></a>

AI-powered verification combines two components that work together to provide automated deployment health analysis:

**AI Verify (v2) step**: A pipeline step that deploys data collection plugins into your Kubernetes cluster to collect and analyze logs and metrics during deployments. The plugins aggregate and cluster data while stripping personally identifiable information before it leaves your cluster. Instead of comparing against historical baselines or predefined thresholds, the step uses statistical and algorithmic methods for anomaly detection, with LLMs augmenting the results by contextualizing anomalies against deployment verification criteria, filtering false positives based on business criticality, and synthesizing actionable root cause insights.

**AI-assisted health source configuration**: An LLM-powered Configuration Agent that automatically discovers and configures metrics from your observability platforms. Describe your service and monitoring goals in natural language, and the agent generates appropriate metric categories, queries, and aggregation settings.

## How it differs from AI Verify (v1) <a href="#how-it-differs-from-ai-verify-v1" id="how-it-differs-from-ai-verify-v1"></a>

AI Verify (v1) (the legacy Verify step) and AI Verify (v2) take fundamentally different approaches to deployment verification:

| Aspect                   | AI Verify (v1)                                                          | AI Verify (v2)                                                               |
| ------------------------ | ----------------------------------------------------------------------- | ---------------------------------------------------------------------------- |
| **Baseline requirement** | Requires clean baseline data before verification works                  | Uses baseline data but does not require manual baseline configuration        |
| **Configuration**        | Manual monitored service setup, health source queries, threshold tuning | Describe service in natural language, agent discovers and configures metrics |
| **Anomaly detection**    | Rule-based comparison against thresholds                                | Statistical and algorithmic detection with LLM-based contextualization       |
| **Failure explanations** | Generic alerts ("threshold exceeded")                                   | Natural language explanations of what went wrong and why                     |
| **Maintenance overhead** | Frequent threshold retuning as services evolve                          | Adapts automatically to changing behavior patterns                           |
| **False positives**      | Common with brittle thresholds                                          | Reduced by understanding context and relationships between signals           |
| **Setup time**           | Hours per service                                                       | Minutes per service                                                          |

## Feature parity between AI Verify (v1) and (v2) <a href="#feature-parity-ai-verify-v1-vs-v2" id="feature-parity-ai-verify-v1-vs-v2"></a>

Harness is developing AI Verify (v2) to reach feature parity with AI Verify (v1). The following table shows which capabilities are available and which are planned for future releases.

| Feature                                | AI Verify (v1) | AI Verify (v2) | Status                              |
| -------------------------------------- | -------------- | -------------- | ----------------------------------- |
| **Core verification**                  | ✓              | ✓              | Available                           |
| **Metrics analysis**                   | ✓              | ✓              | Available                           |
| **Logs analysis**                      | ✓              | ✓              | Available                           |
| **Variable support in health sources** | ✓              | ✓              | Available                           |
| **Multi-query support**                | ✓              | ✓              | Available                           |
| **User feedback on logs**              | ✓              | ✓              | Available                           |
| **Configurable start time**            | ✓              | ✗              | Future release                      |
| **Sub-task webhook notifications**     | ✓              | ✗              | Future release                      |
| **Pinned baseline**                    | ✓              | ✗              | Future release                      |
| **Multiple verification types**        | ✓              | N/A            | AI Verify (v2) uses single approach |

## Additional AI Verify capabilities <a href="#additional-ai-verify-capabilities" id="additional-ai-verify-capabilities"></a>

**Variable support for health sources**

Variable support for health sources lets you use Harness variables in health source queries to parameterize them dynamically based on deployment context. Health sources have inline variable support with a field to create variables. You can use these variables inside health source queries, for example `<+healthSourceVariables.var1>`. You can also use other pipeline, stage, and service variables along with health source variables.

**Multi-query support**

Multi-query support lets you analyze multiple queries configured in a single health source and provides an aggregated view of all the queries analyzed by AI Verify.

**User feedback on logs**

User feedback on log analysis allows you to provide feedback on the log analysis for better analysis based on your domain knowledge. This feature helps improve the accuracy of log anomaly detection.

For questions about the AI Verify roadmap or to request prioritization of specific features, go to [Harness Community](https://community.harness.io/) or contact [Harness Support](mailto:support@harness.io).

## How AI-powered verification works <a href="#how-ai-powered-verification-works" id="how-ai-powered-verification-works"></a>

AI-powered verification runs in two phases: health source configuration and deployment verification.

### Phase 1: Health source configuration <a href="#phase-1-health-source-configuration" id="phase-1-health-source-configuration"></a>

When you create a health source for AI Verify, the Configuration Agent:

1. **Discovers metrics**: Connects to your observability platform (Datadog or Dynatrace) and retrieves all available metrics for your connector. Filters out internal and billing metrics.
2. **Semantic grouping**: Analyzes your service description and monitoring goals to group related metrics into semantic categories like error tracking, performance latency, or infrastructure utilization. Ranks categories by golden signal importance.
3. **Query generation**: For each selected metric, fetches metadata, identifies appropriate tag filters based on your service and environment, constructs platform-specific queries, and validates them by retrieving sample data.
4. **Natural language refinement**: You can fine-tune any metric using plain English feedback (for example, "Use p95 instead of avg" or "Add filter for environment:production"). The agent regenerates the configuration with your feedback as context.

The Configuration Agent uses a three-phase pipeline (discovery, configuration, validation) with full transparency. You can view the complete agent timeline showing tool calls, LLM reasoning, and validation steps for each metric.

### Phase 2: Deployment verification <a href="#phase-2-deployment-verification" id="phase-2-deployment-verification"></a>

When the AI Verify (v2) step executes in your pipeline:

1. **Plugin deployment**: Deploys data collection plugins as pods in your Kubernetes cluster. Each health source creates an independent plugin for parallel analysis.
2. **Data collection**: Plugins query your observability platforms during a configurable time window (for example, 5 minutes) after deployment completes. Collects logs and metrics defined in your health sources.
3. **Pattern analysis and PII filtering**: Plugins aggregate and cluster similar log entries and metric patterns within your cluster. Deduplication eliminates redundant data points. Personally identifiable information is stripped before data leaves your cluster.
4. **Anomaly detection**: Statistical and algorithmic methods identify anomalies and regressions in the collected data. The system detects deviations from expected patterns without requiring historical baselines.
5. **LLM contextualization**: Large language models augment the detected anomalies by adding deployment context, filtering false positives based on business criticality, and understanding relationships between signals (for example, "latency increased because error rate spiked").
6. **Natural language insights**: If issues are detected, LLMs generate explanations of what went wrong, root cause analysis, and actionable remediation suggestions.
7. **Verdict**: The step passes or fails based on the issues that remain after analysis. [Sensitivity](#sensitivity) tunes how readily changes are flagged, but it does not decide the verdict on its own. Each health source provides an independent verdict.

### Sensitivity <a href="#sensitivity" id="sensitivity"></a>

Sensitivity controls how readily AI Verify (v2) flags a change as a problem. You can set it to **High**, **Medium** (default) or **Low**.

* **High**: Catches smaller changes. Detection windows are tighter, and borderline log clusters are classified as the more serious class.
* **Medium**: The default behaviour. Use it for most services.
* **Low**: Flags only clear changes. Detection windows are wider, and borderline log clusters are classified as the less serious class. Clear deployment-blocking errors are always reported.

For metrics, AI Verify (v2) still chooses the detection method from the metric type and baseline. Sensitivity adjusts how wide that method's window is. For logs, it adjusts how aggressively similar log lines are grouped, when a volume increase counts as a spike, and how borderline clusters are classified.

{% hint style="info" %}
This section covers AI Verify (v2). For sensitivity on the legacy Verify step (AI Verify v1), including the per-metric sensitivity override, go to [Sensitivity](/continuous-delivery/use-continuous-delivery/verify-deployments/configure-cv/ai-verify-v1/overview.md#sensitivity) in the Verification Overview.
{% endhint %}

## Limitations and considerations <a href="#limitations-and-considerations" id="limitations-and-considerations"></a>

**Network requirements**: The Configuration Agent and AI Verify data collection plugins require network access to both your observability platform and the LLM Gateway. Air-gapped deployments cannot use AI-powered verification and must use manual health source setup with traditional verification steps.

**Platform support**: Supports Datadog for metrics and Dynatrace for logs, and native Splunk Logs and New Relic Metrics health sources with manual setup. Manual configuration remains available for all other observability platforms including Prometheus, AppDynamics, and others.

**Infrastructure requirements**: Requires a Kubernetes cluster to run data collection plugin pods. Each health source creates one plugin pod that processes data within your cluster, stripping PII before transmission. Ensure sufficient resource quota for concurrent executions.

**Configuration scope**: Health source configuration changes made directly in your observability platform (outside Harness) are not reflected in the health source definition. The Configuration Agent only manages metrics it configures through Harness.

**Custom composite metrics**: Generated composites use only the base metrics you selected. If a composite requires metrics outside your selected categories, add those metrics manually before generating the composite.

**Service context changes**: If your service's tagging strategy changes, regenerate the health source or manually update queries to reflect the new tags. The agent cannot detect external tagging changes after initial configuration.

## Get started <a href="#getting-started" id="getting-started"></a>

To start using AI Verify (v2):

1. **Create AI-assisted health sources**: Use the Configuration Agent to automatically discover and configure metrics from your observability platforms. Go to [AI-assisted health source configuration](/continuous-delivery/use-continuous-delivery/verify-deployments/configure-cv/configure-verify-step-with-ai/ai-assisted-health-source.md) for step-by-step instructions.
2. **Add AI Verify (v2) step to pipelines**: Configure the AI Verify (v2) step in your deployment pipelines to analyze health sources during deployments. Go to [Configure the AI Verify (v2) step](/continuous-delivery/use-continuous-delivery/verify-deployments/configure-cv/configure-verify-step-with-ai/ai-verify.md) for detailed configuration options.
3. **Monitor executions**: View real-time agent events, data clustering, and natural language insights during pipeline execution. Review explanations and remediation suggestions when issues are detected.
4. **Refine over time**: Use natural language feedback to fine-tune metric configurations. Regenerate health sources as your service's monitoring needs evolve.

## Next steps

* Go to [AI-assisted health source configuration](/continuous-delivery/use-continuous-delivery/verify-deployments/configure-cv/configure-verify-step-with-ai/ai-assisted-health-source.md) to create health sources with the Configuration Agent.
* Go to [Configure the AI Verify (v2) step](/continuous-delivery/use-continuous-delivery/verify-deployments/configure-cv/configure-verify-step-with-ai/ai-verify.md) to add the step to your deployment pipeline.

{% @harness-feedback/feedback %}

[^1]: Off by default. If you want to try it and share feedback, request access and Harness will turn it on for your account. Beta features can change and may not be ready for production use.