Skip to main content

datadog-cpu-check

Last updated on

Datadog CPU check validates the CPU utilisation of a service using Datadog container metrics. The target service name and Kubernetes namespace are supplied at runtime via probe variables, allowing this probe to be reused across services.

Infrastructure type

  • Kubernetes

Use cases

Datadog CPU Check probe helps you:

  • Verify CPU stays within limits during pod CPU hog faults
  • Monitor container CPU after resource stress experiments
  • Validate autoscaling behavior under load chaos
  • Detect CPU saturation before it affects latency

Overview

This probe queries Datadog for sum:container.cpu.usage filtered by kube_deployment and kube_namespace, converts nanoseconds to CPU cores using the formula cpu/1000000000, and compares the mean aggregated value against your threshold.

Probe type

APM Probe (Datadog)

Prerequisites

  • Kubernetes chaos infrastructure with Harness Delegate
  • A configured Datadog connector
  • Datadog agent reporting container.cpu.usage for the target deployment
  • Probe variables SERVICE_NAME and NAMESPACE set at experiment runtime

Probe properties

Comparator

TypeCriteriaDefault value
float<=0.50

The probe passes when the mean CPU usage (in cores) is less than or equal to the comparator value.

Datadog APM probe inputs

FieldDescriptionDefault
connectorIDIdentifier of the Datadog connectorRequired at runtime
durationInMinLook-back window in minutes5
queryTypeDatadog query API versionv2
queries[0].nameQuery alias used in the formulacpu
queries[0].dataSourceMetric source typemetrics
queries[0].params.queryDatadog metric querysum:container.cpu.usage{kube_deployment:<+probe.variables.SERVICE_NAME>,kube_namespace:<+probe.variables.NAMESPACE>}
formulaConverts query result to CPU corescpu/1000000000
aggregationAggregation over the time windowmean

Probe variables

VariableDescriptionRequired
SERVICE_NAMEKubernetes deployment name of the target serviceYes
NAMESPACEKubernetes namespace of the target serviceYes

Configurable inputs

InputDescriptionRequiredDefault
CONNECTOR_IDDatadog connector identifierYes-
DURATION_IN_MINLook-back window in minutesNo5
COMPARATOR_VALUECPU threshold in coresYes0.50

Run properties

PropertyDescriptionTypeDefault
timeoutMaximum time to wait for the probe to completeString30s
intervalTime between probe executionsString5s
attemptNumber of retry attempts before marking as failedInteger1
pollingIntervalTime between retry attemptsString30s
initialDelayInitial delay before starting the probeString1s
stopOnFailureStop the experiment if the probe failsBooleanfalse
verbosityLog verbosity levelStringdebug

Probe definition

You can define this probe in your chaos experiment as follows:

probe:
- name: datadog-cpu-check
type: apmProbe
mode: Continuous
apmProbe/inputs:
type: Datadog
comparator:
type: float
value: "0.50"
criteria: <=
datadogApmProbeInputs:
connectorID: <+connector.identifier>
durationInMin: 5
queryType: v2
queries:
- name: cpu
dataSource: metrics
params:
query: sum:container.cpu.usage{kube_deployment:<+probe.variables.SERVICE_NAME>,kube_namespace:<+probe.variables.NAMESPACE>}
formula: cpu/1000000000
aggregation: mean
runProperties:
timeout: 30s
interval: 5s
attempt: 1
pollingInterval: 30s
initialDelay: 1s
stopOnFailure: false
verbosity: debug

Next steps