> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/ai-sre/ai-sre-for-administrators/set-up-alert-management/webhooks/integration-guides/monitoring/grafana.md).

# Grafana Integration Guide

Configure Grafana Unified Alerting to send webhook notifications to Harness AI SRE when alerts fire.

### Before you begin <a href="#before-you-begin" id="before-you-begin"></a>

* **Harness webhook endpoint**: Create a Grafana webhook in Harness AI SRE using the [Grafana webhook template](/ai-sre/ai-sre-for-administrators/set-up-alert-management/webhooks/templates/monitoring/grafana.md).
* **Grafana access**: Admin permissions to configure contact points and notification policies.
* **Webhook URL**: Copy the webhook URL from your Harness webhook configuration.
* **Grafana version**: This guide covers Grafana 8.0+ with Unified Alerting. For legacy alerting, go to the Grafana documentation.
* **Unified Alerting documentation**: Go to [Grafana Unified Alerting](https://grafana.com/docs/grafana/latest/alerting/) to understand contact points and notification policies.
* **Webhook configuration reference**: Go to [Manage Contact Points](https://grafana.com/docs/grafana/latest/alerting/configure-notifications/manage-contact-points/) for webhook setup details.

***

### Create webhook contact point <a href="#create-webhook-contact-point" id="create-webhook-contact-point"></a>

#### Navigate to alerting configuration <a href="#navigate-to-alerting-configuration" id="navigate-to-alerting-configuration"></a>

Open the contact points configuration in Grafana:

1. In Grafana, go to **Alerting**, then select **Contact points**
2. Click **New contact point**

#### Configure webhook contact point <a href="#configure-webhook-contact-point" id="configure-webhook-contact-point"></a>

{% tabs %}
{% tab title="Basic configuration" %}

* **Name**: `Harness AI SRE`
* **Integration**: Select **Webhook**
* **URL**: Your Harness webhook URL

  ```
  https://<your-harness-instance>/gateway/ai-sre/api/webhooks/<webhook-id>
  ```
* **HTTP Method**: `POST`
  {% endtab %}

{% tab title="With authentication" %}

* **Name**: `Harness AI SRE`
* **Integration**: Select **Webhook**
* **URL**: Your Harness webhook URL
* **HTTP Method**: `POST`
* **Authorization**: Add custom header
  * **Header**: `X-Webhook-Secret`
  * **Value**: `your-secret-key`
    {% endtab %}

{% tab title="Advanced settings" %}

* **Name**: `Harness AI SRE`
* **Integration**: Select **Webhook**
* **URL**: Your Harness webhook URL
* **HTTP Method**: `POST`
* **Max alerts**: Leave blank to send all alerts
* **Disable resolved message**: Uncheck to send resolved notifications
* **HTTP settings**:
  * **Authorization**: Custom header if needed
  * **User**: Leave blank
  * **Password**: Leave blank
    {% endtab %}
    {% endtabs %}

#### Test contact point <a href="#test-contact-point" id="test-contact-point"></a>

Click **Test** to send a test notification and verify the webhook is reachable.

#### Save contact point <a href="#save-contact-point" id="save-contact-point"></a>

Click **Save contact point** to create the webhook integration.

***

### Configure notification policy <a href="#configure-notification-policy" id="configure-notification-policy"></a>

#### Edit default notification policy <a href="#edit-default-notification-policy" id="edit-default-notification-policy"></a>

Route alerts to the Harness contact point:

1. Go to **Alerting**, then select **Notification policies**
2. Edit the **Default policy** or create a new nested policy
3. Select **Harness AI SRE** as the contact point

#### Create specific routing policy <a href="#create-specific-routing-policy" id="create-specific-routing-policy"></a>

For selective alert routing:

1. Click **New nested policy**
2. Configure matching labels:
   * **Label**: `severity`
   * **Operator**: `=`
   * **Value**: `critical`
3. Select **Harness AI SRE** contact point
4. Click **Save policy**

***

### Configure field mapping in Harness <a href="#configure-field-mapping-in-harness" id="configure-field-mapping-in-harness"></a>

In your Harness webhook configuration, map Grafana payload fields to alert properties.

#### Grafana webhook payload structure <a href="#grafana-webhook-payload-structure" id="grafana-webhook-payload-structure"></a>

```json
{
  "receiver": "Harness AI SRE",
  "status": "firing",
  "alerts": [
    {
      "status": "firing",
      "labels": {
        "alertname": "High CPU Usage",
        "grafana_folder": "Production",
        "severity": "critical",
        "instance": "prod-server-01",
        "job": "node-exporter"
      },
      "annotations": {
        "summary": "CPU usage above 90%",
        "description": "Server prod-server-01 has CPU usage at 95%",
        "runbook_url": "https://wiki.company.com/runbooks/high-cpu"
      },
      "startsAt": "2025-07-01T10:30:00.000Z",
      "endsAt": "0001-01-01T00:00:00Z",
      "generatorURL": "https://grafana.example.com/alerting/grafana/abc123/view",
      "fingerprint": "a1b2c3d4e5f6",
      "silenceURL": "https://grafana.example.com/alerting/silence/new?...",
      "dashboardURL": "https://grafana.example.com/d/dashboard-id",
      "panelURL": "https://grafana.example.com/d/dashboard-id?viewPanel=1",
      "values": {
        "B": 95.2
      }
    }
  ],
  "groupLabels": {
    "alertname": "High CPU Usage"
  },
  "commonLabels": {
    "alertname": "High CPU Usage",
    "severity": "critical"
  },
  "commonAnnotations": {
    "summary": "CPU usage above 90%"
  },
  "externalURL": "https://grafana.example.com",
  "version": "1",
  "groupKey": "{}:{alertname=\"High CPU Usage\"}",
  "truncatedAlerts": 0,
  "orgId": 1,
  "title": "[FIRING:1] High CPU Usage (prod-server-01)",
  "state": "alerting",
  "message": "**Firing**\n\nValue: [ var='A' labels={instance=prod-server-01} value=95.2 ]\nLabels:\n - alertname = High CPU Usage\n - severity = critical\nAnnotations:\n - summary = CPU usage above 90%\n"
}
```

#### Map basic fields <a href="#map-basic-fields" id="map-basic-fields"></a>

Use Mustache templates:

```yaml
title: "{{webhook.title}}"
message: "{{webhook.message}}"
severity: "{{webhook.commonLabels.severity}}"
source: "grafana"
link: "{{webhook.externalURL}}"
tags:
  - "alertname:{{webhook.alerts[0].labels.alertname}}"
  - "grafana_folder:{{webhook.alerts[0].labels.grafana_folder}}"
  - "instance:{{webhook.alerts[0].labels.instance}}"
```

#### Advanced field mapping with CEL <a href="#advanced-field-mapping-with-cel" id="advanced-field-mapping-with-cel"></a>

```cel
// Use Grafana's pre-formatted title
title: webhook.title

// Use Grafana's pre-formatted message or build custom
message: size(webhook.alerts) > 0 
  ? webhook.alerts[0].annotations.description + "\n\n" +
    "Dashboard: " + webhook.alerts[0].dashboardURL + "\n" +
    "Panel: " + webhook.alerts[0].panelURL
  : webhook.message

// Map Grafana severity to Harness severity
severity: has(webhook.commonLabels.severity)
  ? (webhook.commonLabels.severity == "critical" ? "critical" :
     webhook.commonLabels.severity == "warning" ? "high" : "medium")
  : "medium"

source: "grafana"
link: size(webhook.alerts) > 0 ? webhook.alerts[0].generatorURL : webhook.externalURL

// Extract all labels as tags
tags: size(webhook.alerts) > 0
  ? webhook.alerts[0].labels.keys().map(k, k + ":" + string(webhook.alerts[0].labels[k]))
  : []

// Filter: only firing alerts
filter: webhook.status == "firing" && size(webhook.alerts) > 0
```

***

### Test the integration <a href="#test-the-integration" id="test-the-integration"></a>

#### Create test alert rule <a href="#create-test-alert-rule" id="create-test-alert-rule"></a>

Create a rule that fires immediately to confirm delivery:

1. Go to **Alerting**, then select **Alert rules**
2. Click **New alert rule**
3. Configure a simple test rule:
   * **Alert rule name**: `Test Harness Integration`
   * **Query**: Use a test query that always fires
   * **Condition**: Set threshold that will trigger
   * **Evaluate every**: `1m`
   * **For**: `0m`
4. Add labels:
   * `severity`: `warning`
   * `test`: `true`
5. Add annotations:
   * `summary`: `Test alert for Harness integration`
6. Select notification policy or contact point
7. Click **Save rule and exit**

#### Verify in Grafana <a href="#verify-in-grafana" id="verify-in-grafana"></a>

Confirm the rule fired and the notification was sent:

1. Go to **Alerting**, then select **Alert rules**
2. Wait for the rule to evaluate and fire
3. Check **Contact points**, then **Harness AI SRE**, for recent notifications

#### Verify in Harness <a href="#verify-in-harness" id="verify-in-harness"></a>

Confirm the alert arrived and mapped correctly:

1. Navigate to **Alerts** in Harness AI SRE
2. Check that the test alert appears
3. Verify field mapping is correct

***

### Available Grafana fields <a href="#available-grafana-fields" id="available-grafana-fields"></a>

| Field                   | Description                         | Example                                         |
| ----------------------- | ----------------------------------- | ----------------------------------------------- |
| `receiver`              | Contact point name                  | `Harness AI SRE`                                |
| `status`                | Alert status                        | `firing`, `resolved`                            |
| `alerts`                | Array of firing alerts              | See payload structure                           |
| `alerts[].labels`       | Alert labels                        | `{"alertname": "...", "severity": "critical"}`  |
| `alerts[].annotations`  | Alert annotations                   | `{"summary": "...", "description": "..."}`      |
| `alerts[].startsAt`     | Alert start time                    | `2025-07-01T10:30:00.000Z`                      |
| `alerts[].endsAt`       | Alert end time                      | `0001-01-01T00:00:00Z` (if not resolved)        |
| `alerts[].generatorURL` | Link to alert rule                  | `https://grafana.../alerting/.../view`          |
| `alerts[].dashboardURL` | Link to dashboard                   | `https://grafana.../d/dashboard-id`             |
| `alerts[].panelURL`     | Link to specific panel              | `https://grafana.../d/dashboard-id?viewPanel=1` |
| `alerts[].fingerprint`  | Unique alert ID                     | `a1b2c3d4e5f6`                                  |
| `alerts[].values`       | Query result values, keyed by refID | `{"B": 95.2}`                                   |
| `title`                 | Pre-formatted alert title           | `[FIRING:1] High CPU Usage`                     |
| `message`               | Pre-formatted alert message         | Multi-line formatted message                    |
| `groupLabels`           | Labels used for grouping            | `{"alertname": "..."}`                          |
| `commonLabels`          | Labels common to all alerts         | `{"severity": "critical"}`                      |
| `commonAnnotations`     | Annotations common to all           | `{"summary": "..."}`                            |
| `externalURL`           | Grafana instance URL                | `https://grafana.example.com`                   |

***

### Advanced configuration <a href="#advanced-configuration" id="advanced-configuration"></a>

#### Route by folder <a href="#route-by-folder" id="route-by-folder"></a>

Send alerts from specific Grafana folders to different Harness webhooks:

**Grafana notification policy**:

1. Create nested policy with label matcher:
   * **Label**: `grafana_folder`
   * **Operator**: `=`
   * **Value**: `Production`
2. Select appropriate contact point

#### Route by data source <a href="#route-by-data-source" id="route-by-data-source"></a>

Label alerts by data source and route accordingly:

**Alert rule labels**:

```yaml
labels:
  severity: critical
  datasource: prometheus
  team: platform
```

**Notification policy matcher**:

* **Label**: `datasource`
* **Operator**: `=`
* **Value**: `prometheus`

#### Silence alerts <a href="#silence-alerts" id="silence-alerts"></a>

Use Grafana's silence URL in Harness:

```cel
// Include silence link in message
message: webhook.alerts[0].annotations.description + "\n\n" +
         "Silence this alert: " + webhook.alerts[0].silenceURL
```

#### Multi-dimensional alerts <a href="#multi-dimensional-alerts" id="multi-dimensional-alerts"></a>

Grafana supports multi-dimensional alerting. Access all dimensions:

```cel
// Extract all label dimensions
tags: size(webhook.alerts) > 0
  ? webhook.alerts[0].labels.keys().map(k, k + ":" + string(webhook.alerts[0].labels[k]))
  : []
```

***

### Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

<details>

<summary>Grafana webhook is not sending notifications to Harness AI SRE</summary>

Test the contact point from the Grafana UI, check the Grafana logs, verify the webhook URL is accessible from the Grafana server, and check for a firewall or proxy blocking outbound requests.

</details>

<details>

<summary>Grafana alerts are not appearing in Harness AI SRE</summary>

Check the Harness webhook logs for errors, inspect the raw payload in the Grafana contact point test, verify the CEL filter logic by temporarily removing the filter, and ensure the required fields exist in the payload.

</details>

<details>

<summary>Resolved Grafana alerts are not clearing in Harness AI SRE</summary>

Ensure 'Disable resolved message' is unchecked in the contact point, and handle the resolved status in the Harness CEL mapping.

</details>

<details>

<summary>Grafana is creating duplicate alerts in Harness AI SRE</summary>

Review the notification policy routes to ensure no overlapping matchers, and adjust the group wait, group interval, and repeat interval in the notification policy.

</details>

***

### Example: complete integration <a href="#example-complete-integration" id="example-complete-integration"></a>

#### Grafana contact point configuration <a href="#grafana-contact-point-configuration" id="grafana-contact-point-configuration"></a>

Configure the contact point with these settings:

* **Name**: `Harness AI SRE`
* **Integration**: Webhook
* **URL**: `https://app.harness.io/gateway/ai-sre/api/webhooks/wh_abc123`
* **HTTP Method**: POST
* **Disable resolved message**: Unchecked

#### Grafana notification policy <a href="#grafana-notification-policy" id="grafana-notification-policy"></a>

```
Root policy
├─ Default contact point: Harness AI SRE
├─ Group by: [alertname, grafana_folder]
├─ Group wait: 10s
├─ Group interval: 5m
├─ Repeat interval: 4h
└─ Nested policies:
   ├─ severity = critical → Harness AI SRE (immediate)
   └─ grafana_folder = Production → Harness AI SRE
```

#### Map fields in the Harness webhook <a href="#map-fields-in-the-harness-webhook" id="map-fields-in-the-harness-webhook"></a>

```yaml
title: "{{webhook.title}}"

message: |
  **Alert**: {{webhook.alerts[0].labels.alertname}}
  **Status**: {{webhook.status}}
  **Severity**: {{webhook.commonLabels.severity}}
  **Folder**: {{webhook.alerts[0].labels.grafana_folder}}
  
  **Summary**: {{webhook.alerts[0].annotations.summary}}
  {{webhook.alerts[0].annotations.description}}
  
  **Value**: {{webhook.alerts[0].values.B}}
  **Started At**: {{webhook.alerts[0].startsAt}}
  
  **Dashboard**: {{webhook.alerts[0].dashboardURL}}
  **Panel**: {{webhook.alerts[0].panelURL}}
  **Alert Rule**: {{webhook.alerts[0].generatorURL}}

severity: |
  webhook.status == "resolved" ? "info" :
  webhook.commonLabels.severity == "critical" ? "critical" :
  webhook.commonLabels.severity == "warning" ? "high" : "medium"

source: "grafana"
link: "{{webhook.alerts[0].dashboardURL}}"

tags:
  - "source:grafana"
  - "alertname:{{webhook.alerts[0].labels.alertname}}"
  - "severity:{{webhook.commonLabels.severity}}"
  - "folder:{{webhook.alerts[0].labels.grafana_folder}}"
  - "status:{{webhook.status}}"
  - "fingerprint:{{webhook.alerts[0].fingerprint}}"

filter: |
  size(webhook.alerts) > 0 &&
  webhook.status == "firing"

custom_fields:
  fingerprint: "{{webhook.alerts[0].fingerprint}}"
  dashboard_url: "{{webhook.alerts[0].dashboardURL}}"
  panel_url: "{{webhook.alerts[0].panelURL}}"
  generator_url: "{{webhook.alerts[0].generatorURL}}"
  grafana_instance: "{{webhook.externalURL}}"
```

***

### Next steps <a href="#next-steps" id="next-steps"></a>

* [Route alerts](/ai-sre/ai-sre-for-administrators/set-up-alert-management/alert-rules/overview.md): Route and deduplicate Grafana alerts.
* [Use CEL in webhooks](/ai-sre/ai-sre-for-administrators/set-up-alert-management/webhooks/use-cel-webhooks.md): Add custom filtering logic.
* [AI agent](/ai-sre/ai-sre-for-incident-responders/use-ai-agents/ai-agent.md): Enable automated alert investigation.
* [Grafana template](/ai-sre/ai-sre-for-administrators/set-up-alert-management/webhooks/templates/monitoring/grafana.md): Use the pre-configured template.

***

### Related documentation <a href="#related-documentation" id="related-documentation"></a>

#### Grafana official documentation <a href="#grafana-official-documentation" id="grafana-official-documentation"></a>

* [Unified Alerting](https://grafana.com/docs/grafana/latest/alerting/): Complete guide to Grafana Unified Alerting, contact points, and notification policies.
* [Manage contact points](https://grafana.com/docs/grafana/latest/alerting/configure-notifications/manage-contact-points/): Webhook contact point setup and configuration options.
* [Template notifications](https://grafana.com/docs/grafana/latest/alerting/configure-notifications/template-notifications/): Webhook payload structure and available fields.
* [Webhook notifier](https://grafana.com/docs/grafana/latest/alerting/configure-notifications/webhook-notifier/): Complete payload structure reference (alerts array, labels, annotations, URLs).