Skip to main content

AI Cost Management Quickstart

Last updated on

This quickstart walks you through setting up AI trace attribution so you can see exactly which agent, session, or request drove your AI spend. If you have not connected a billing provider yet, start with Get Started first.

Before You Begin​

  • AI Cost Management enabled: Confirm that AI Cloud Providers appears under Cloud & AI Cost Management > Account Settings. If it does not, contact Harness Support to enable it for your account.
  • A provider connector (recommended): Needed for invoice-accurate costs to compare against trace estimates. Go to the CACM Get Started to connect one.
  • Permission to create a service account: You generate the ingestion token from a service account. Go to RBAC in Harness to confirm your role.
  • A runtime: You need one of the following: Python 3.8+ to instrument your app with the Harness SDK, or curl to send a quick test trace and verify the setup works.
  • Network access: From your app or shell to the Harness OTLP endpoint (https://app.harness.io/udp-ingest/otel/v1/traces, or your cluster's equivalent).

Set Up Trace Attribution​

Step 1: Generate an Ingestion Token​

Harness authenticates trace ingestion with a bearer token against the account's OTLP endpoint. Create a dedicated service account for ingestion so the credential is isolated and easy to rotate.

  1. Create a service account (example: ai-telemetry-ingest) with the minimum permissions required for ingestion.
  2. Create a service account API key and token.
  3. Select Generate Token and copy it.
Store the token securely

The token is displayed once. Save it to a secret manager or environment variable and do not commit it to source control.

Step 2: Choose Your Path​

Select your onboarding path based on whether your application or gateway currently emits GenAI OpenTelemetry traces:

  • Traces are already emitted: Select Route Existing Traces to forward your telemetry to Harness without code changes.
  • Application requires instrumentation: Instrument your code to emit GenAI traces, then route them to Harness. Use the Harness SDK for direct Python model SDK calls, or a compatible open-source SDK for orchestration frameworks.

Not sure whether your stack emits GenAI traces? Go to Compatibility Matrix to check.

PathUse whenWhat you do
Route Existing TracesYour app or gateway already emits GenAI OpenTelemetry traces (LangSmith, OpenInference, a LiteLLM proxy, or native framework export).Generate a token and point the existing exporter at the Harness endpoint. No code changes.
Instrument Your ApplicationYour app does not emit GenAI traces yet.Add the Harness SDK (direct Python SDK calls) or an open-source SDK (orchestration frameworks), then route the traces to Harness.

Step 3: Instrument Your Application​

GenAI semantic conventions required

Trace attribution depends on OpenTelemetry traces with GenAI semantic conventions, not just standard OpenTelemetry traces. Standard HTTP, database, or function spans do not carry the model name or token counts CACM needs to calculate cost. Go to the GenAI Span Attribute Reference to review the attributes CACM reads.

Follow the path you chose in Step 2.

Use this path if your app or gateway already emits OpenTelemetry traces with GenAI semantic conventions (from LangSmith, OpenInference, a LiteLLM proxy, or a framework with native export). You repoint the existing exporter at Harness, with no code changes.

Set these OpenTelemetry environment variables, then restart the application or gateway:

export OTEL_EXPORTER_OTLP_ENDPOINT=https://app.harness.io/udp-ingest/otel
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer <YOUR_TOKEN>"
export OTEL_TRACES_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf

Replace app.harness.io with your cluster if different (example: app3.harness.io).

What each variable does
VariablePurpose
OTEL_EXPORTER_OTLP_ENDPOINTHarness OTLP trace ingestion endpoint. Replace app.harness.io with the account's cluster if different (example: app3.harness.io). Find the cluster in the URL when logged in to Harness.
OTEL_EXPORTER_OTLP_HEADERSBearer token for authentication. Use the literal token value or reference it from a secret or environment variable.
OTEL_TRACES_EXPORTERSelects the OTLP exporter for traces.
OTEL_EXPORTER_OTLP_PROTOCOLSelects HTTP/protobuf OTLP transport (Harness expects this format).

Some frameworks and tools need one extra flag to turn telemetry on before the variables above take effect. Go to the matching integration page for the exact flag:

Framework or toolExtra step
LangChain / LangGraphSet LANGSMITH_OTEL_ENABLED=true
LiteLLM ProxyEnable the otel callback in config.yaml
Claude CodeSet CLAUDE_CODE_ENABLE_TELEMETRY=1

Step 4: Verify Traces in Cost Explorer​

  1. Run the application, or restart the exporter, so traces flow. They usually appear within a few minutes; allow up to about 20 minutes.
  2. Go to Cloud & AI Cost Management > Cost Explorer.
  3. Select the AI Traces view or group by Service Name, and find your service.
  4. Select a service row to open the Service Traces drawer, and drill from a session down to the exact LLM call, retry, or tool loop that drove the cost.

Once traces are flowing and you can see cost attributed by service, you have full AI cost visibility: invoice-accurate totals from the connector and a code-level breakdown from traces.

Step 5: Send a test trace​

To confirm the endpoint and token work without instrumenting an application, send a single test span with curl. This is useful before you wire up a full app, or to isolate whether a problem is with the ingestion setup or the instrumentation.

Replace <ACCOUNT_ID> with your account identifier and <YOUR_TOKEN> with the token from Step 1, then run:

Test trace curl command
curl --request POST \
  --url 'https://app.harness.io/udp-ingest/otel/v1/traces?accountIdentifier=<ACCOUNT_ID>&routingId=<ACCOUNT_ID>' \
  --header 'content-type: application/json' \
  --header 'Authorization: Bearer <YOUR_TOKEN>' \
  --data '{
  "resourceSpans": [
    {
      "resource": {
        "attributes": [
          { "key": "harness.account.id", "value": { "stringValue": "<ACCOUNT_ID>" } },
          { "key": "service.name", "value": { "stringValue": "test-service" } }
        ]
      },
      "scopeSpans": [
        {
          "scope": { "name": "otlp-test-client" },
          "spans": [
            {
              "traceId": "aabbccdd11223344aabbccdd11223344",
              "spanId": "0000000000000001",
              "name": "test-span",
              "startTimeUnixNano": "1786104000000000000",
              "endTimeUnixNano": "1786104000500000000"
            }
          ]
        }
      ]
    }
  ]
}'

Replace startTimeUnixNano and endTimeUnixNano with current values by running date +%s%N.

Open the AI Traces view in Cost Explorer and look for test-service. It usually appears within a few minutes; allow up to about 20 minutes.

Which number to trust

Trace cost is computed from tokens and list price, so it is an estimate, typically 95 to 98% accurate against the invoice. Use the connector for any billed figure and traces for attribution. Go to Which number to trust when they differ for the full explanation.

Next Steps​