Datadog Integration Guide

Send monitor alerts through webhooks.

Configure Datadog monitors to send webhook notifications to Harness AI SRE when alerts trigger.

Before you begin

• Harness webhook endpoint: Create a Datadog webhook in Harness AI SRE using the Datadog webhook template.

• Datadog permissions: Access to create or modify monitors and integrations in Datadog.

• Webhook URL: Copy the webhook URL from your Harness webhook configuration.

• Datadog webhook documentation: Go to Datadog Webhooks Integration to understand Datadog's webhook capabilities.

• Monitor notification syntax: Go to Datadog Monitor Notifications to learn about notification configuration and template variables.

---

Create webhook integration in Datadog

Navigate to Datadog integrations

Open the Webhooks integration to add a new webhook:

1. In Datadog, navigate to Integrations, then select Integrations

2. Search for and select Webhooks

3. Go to the Configuration tab

4. Click New

Configure the webhook

Configure these fields:

• Name: harness-ai-sre (or any descriptive name)

• URL: Your Harness webhook URL

  https://<your-harness-instance>/gateway/ai-sre/api/webhooks/<webhook-id>

• Encode as form: Leave unchecked (use JSON)

• Custom Headers: Add if your Harness webhook requires authentication

  X-Webhook-Secret: your-secret-key

Configure the payload

Use the Payload field to map Datadog variables to JSON:

Basic payload

{
  "alert_name": "$EVENT_TITLE",
  "alert_message": "$TEXT_ONLY_MSG",
  "alert_status": "$ALERT_STATUS",
  "priority": "$PRIORITY",
  "metric": "$ALERT_METRIC",
  "service": "$SERVICE",
  "env": "$ENV",
  "alert_url": "$LINK"
}

Advanced payload

{
  "alert_name": "$EVENT_TITLE",
  "alert_message": "$TEXT_ONLY_MSG",
  "alert_status": "$ALERT_STATUS",
  "priority": "$PRIORITY",
  "metric": "$ALERT_METRIC",
  "query": "$ALERT_QUERY",
  "service": "$SERVICE",
  "env": "$ENV",
  "host": "$HOSTNAME",
  "alert_id": "$ALERT_ID",
  "alert_url": "$LINK",
  "snapshot": "$SNAPSHOT",
  "last_updated": "$LAST_UPDATED",
  "tags": "$TAGS",
  "org_id": "$ORG_ID",
  "org_name": "$ORG_NAME"
}

With custom context

{
  "alert_name": "$EVENT_TITLE",
  "alert_message": "$TEXT_ONLY_MSG",
  "alert_status": "$ALERT_STATUS",
  "priority": "$PRIORITY",
  "service": "$SERVICE",
  "env": "$ENV",
  "alert_url": "$LINK",
  "custom_context": {
    "query": "$ALERT_QUERY",
    "runbook_url": "https://wiki.company.com/runbooks/$SERVICE"
  }
}

Save the webhook

Click Save to create the webhook integration.

---

Configure field mapping in Harness

In your Harness webhook configuration, map the Datadog payload fields to alert properties.

Basic field mapping example

Use Mustache templates for simple field mapping:

title: "{{webhook.alert_name}}"
message: "{{webhook.alert_message}}"
severity: "{{webhook.priority}}"
source: "datadog"
link: "{{webhook.alert_url}}"
tags:
  - "service:{{webhook.service}}"
  - "env:{{webhook.env}}"
  - "metric:{{webhook.metric}}"

Advanced field mapping with CEL

Use CEL expressions for conditional logic and transformations:

// Map Datadog priority to Harness severity
title: webhook.alert_name
message: webhook.alert_message
severity: webhook.priority == "P1" ? "critical" :
          webhook.priority == "P2" ? "high" :
          webhook.priority == "P3" ? "medium" : "low"
source: "datadog"
link: webhook.alert_url
tags: [
  "service:" + webhook.service,
  "env:" + webhook.env,
  "status:" + webhook.alert_status.toLowerCase()
]

// Filter: only process alerts (not warnings or recoveries)
filter: webhook.alert_status == "ALERT"

---

Add webhook to Datadog monitors

Option 1: Add to existing monitor

Add the webhook notification to a monitor you already have:

1. Open the Datadog monitor you want to integrate

2. Scroll to the Notify your team section

3. Add the webhook using the @webhook- syntax:

{{#is_alert}}
@webhook-harness-ai-sre
{{/is_alert}}

{{#is_recovery}}
@webhook-harness-ai-sre
{{/is_recovery}}

1. Click Save

Option 2: Add to monitor template

For monitors created from templates:

{{#is_alert}}
Alert on {{service.name}} in {{env}}
@webhook-harness-ai-sre
{{/is_alert}}

{{#is_warning}}
Warning on {{service.name}}
@webhook-harness-ai-sre
{{/is_warning}}

Option 3: Use notification policies

Configure Datadog notification policies to automatically add the webhook to matching monitors.

---

Test the integration

Test from Datadog

Send a test notification from a configured monitor:

1. Open a monitor configured with the webhook

2. Click Test Notifications

3. Select a test scenario (Alert, Warning, Recovery)

4. Click Run Test

Datadog sends a test webhook to Harness.

Verify in Harness

Confirm the test alert arrived and mapped correctly:

1. Navigate to Alerts in Harness AI SRE

2. Check that the test alert appears

3. Verify field mapping:

   • Alert title matches expected format

   • Severity is correct

   • Tags are populated

   • Link works

---

Available Datadog variables

Datadog provides these template variables for webhook payloads:

Variable	Description	Example
$EVENT_TITLE	Monitor name and status	[Alert] High CPU on prod-server-01
$TEXT_ONLY_MSG	Plain text message	CPU usage above 90% for 5 minutes
$ALERT_STATUS	Current status	ALERT, WARNING, RECOVERED, NO DATA
$PRIORITY	Alert priority	P1, P2, P3, P4, P5
$ALERT_METRIC	Primary metric name	system.cpu.user
$ALERT_QUERY	Monitor query	avg(last_5m):avg:system.cpu.user{*} > 0.9
$SERVICE	Service tag value	api-gateway
$ENV	Environment tag value	production
$HOSTNAME	Affected host	prod-server-01
$ALERT_ID	Unique alert identifier	123456789
$ALERT_CYCLE_KEY	Alert cycle identifier	abc123def456
$LINK	Link to alert in Datadog	https://app.datadoghq.com/...
$SNAPSHOT	Graph snapshot URL	https://p.datadoghq.com/...
$LAST_UPDATED	Timestamp of last update	2025-07-01 10:15:30 UTC
$TAGS	All monitor tags	service:api,env:prod
$ORG_ID	Datadog organization ID	12345
$ORG_NAME	Organization name	My Company

The current metric value and alert threshold are not exposed as webhook payload variables. Access them with the {{value}} and {{threshold}} Mustache variables in the monitor message body instead, then include the rendered message in the webhook payload through $TEXT_ONLY_MSG.

---

Advanced configuration

Filter by alert status

Send different payloads for alerts vs recoveries:

Alerts only

{{#is_alert}}
@webhook-harness-ai-sre
{{/is_alert}}

Recoveries only

{{#is_recovery}}
@webhook-harness-ai-sre-recovery
{{/is_recovery}}

Both alerts and recoveries

{{#is_alert}}
@webhook-harness-ai-sre
{{/is_alert}}

{{#is_recovery}}
@webhook-harness-ai-sre
{{/is_recovery}}

Route by priority

Create separate Harness webhooks for different priorities:

{{#is_alert}}
  {{#is_priority 'P1'}}
  @webhook-harness-critical
  {{/is_priority}}

  {{#is_priority 'P2'}}
  @webhook-harness-standard
  {{/is_priority}}
{{/is_alert}}

Include metric values and thresholds

The webhook payload does not carry the metric value or threshold directly. Datadog exposes those only as {{value}} and {{threshold}} template variables in the monitor message body, so include them there and pass the rendered message through to the webhook:

{
  "alert_name": "$EVENT_TITLE",
  "alert_message": "$TEXT_ONLY_MSG",
  "priority": "$PRIORITY",
  "service": "$SERVICE",
  "env": "$ENV",
  "alert_url": "$LINK",
  "context": {
    "query": "$ALERT_QUERY",
    "snapshot_url": "$SNAPSHOT"
  }
}

Monitor message body:

{{#is_alert}}
Current value: {{value}}
Threshold: {{threshold}}
{{/is_alert}}

Harness CEL mapping:

title: webhook.alert_name
message: webhook.alert_message
severity: webhook.priority == "P1" ? "critical" : "high"
link: webhook.context.snapshot_url

---

Troubleshooting

Datadog webhook is not triggering for Harness AI SRE

Verify the webhook appears under Integrations, then Webhooks, confirm the monitor notification includes @webhook-harness-ai-sre, test the notification manually from the monitor page, and review the Datadog Event Explorer for webhook calls.

Datadog webhook returns HTTP 4xx errors when calling the Harness AI SRE endpoint

Verify the webhook URL is correct with no typos in the webhook ID, check that custom headers match the Harness webhook configuration, ensure the Harness webhook is enabled, and test the URL with curl.

Datadog webhook variables are not mapping correctly in Harness AI SRE

Review the actual webhook payload sent by Datadog, ensure field names match exactly (case-sensitive), and use the CEL has() function to check whether fields exist.

High-frequency Datadog alerts are rate limiting the Harness AI SRE webhook

Configure the Datadog monitor to re-notify less frequently, use Harness alert routing rules to deduplicate similar alerts, and add debounce logic in the Harness webhook CEL.

---

Example: complete integration

This example shows a production-ready Datadog-to-Harness integration for a microservices platform.

Datadog webhook payload

{
  "alert_name": "$EVENT_TITLE",
  "alert_message": "$TEXT_ONLY_MSG",
  "alert_status": "$ALERT_STATUS",
  "priority": "$PRIORITY",
  "metric": "$ALERT_METRIC",
  "service": "$SERVICE",
  "env": "$ENV",
  "alert_url": "$LINK",
  "snapshot": "$SNAPSHOT",
  "timestamp": "$LAST_UPDATED",
  "tags": "$TAGS"
}

Harness webhook field mapping example

title: "{{webhook.alert_name}}"
message: |
  {{webhook.alert_message}}
  
  Metric: {{webhook.metric}}
severity: |
  webhook.priority == "P1" ? "critical" :
  webhook.priority == "P2" ? "high" :
  webhook.priority == "P3" ? "medium" : "low"
source: "datadog"
link: "{{webhook.alert_url}}"
tags:
  - "source:datadog"
  - "service:{{webhook.service}}"
  - "env:{{webhook.env}}"
  - "metric:{{webhook.metric}}"
filter: |
  webhook.alert_status == "ALERT" && 
  webhook.env in ["production", "staging"]

Datadog monitor notification

{{#is_alert}}
Production alert on {{service.name}}

Current: {{value}}
Threshold: {{threshold}}

@webhook-harness-ai-sre
@slack-platform-alerts
{{/is_alert}}

{{#is_recovery}}
✅ Recovered: {{service.name}}
@webhook-harness-ai-sre
{{/is_recovery}}

---

Next steps

• Route alerts: Route and deduplicate Datadog alerts.

• Use CEL in webhooks: Add advanced filtering logic.

• AI agent: Enable automated alert investigation.

---

Related documentation

Datadog official documentation

• Webhooks integration: Complete guide to Datadog webhook integration setup and configuration.

• Monitor notifications: Monitor notification configuration and @webhook syntax.

• Template variables reference: Complete list of available template variables ($EVENT_TITLE, $PRIORITY, $SERVICE, and others).

• Webhook template variables: Webhook-specific variables and payload customization.