Skip to main content
Unlisted page
This page is unlisted. Search engines will not index it, and only users having a direct link can access it.

Set Up AI Cost Traces

Last updated on

After you understand how AI traces work, instrument your application to send OpenTelemetry traces with GenAI semantic conventions to Cloud & AI Cost Management (CACM). This page walks you through every setup path, from generating an ingestion token to verifying traces in Cost Explorer.

GenAI semantic conventions required

This depends on OpenTelemetry traces with GenAI semantic conventions, not just standard OpenTelemetry traces. Standard HTTP, database, or function spans do not carry the model name or token counts CACM needs to calculate cost. Go to the GenAI Span Attribute Reference to review the attributes CACM reads.

Before You Begin​

  • AI Cost Management enabled: Confirm that AI Cloud Providers appears under Cloud & AI Cost Management > Account Settings. Go to AI Cost Troubleshooting if it does not.
  • Permission to create a service account: Route Existing Traces generates the ingestion token as its first step. Go to RBAC in Harness to confirm your role.
  • A provider connector (recommended): Needed for invoice-accurate costs. Go to AI Cost Management Quickstart to connect one.
  • A runtime: Python 3.8+ for the Harness SDK, or curl for a test trace.
  • Network egress: From your app or shell to the Harness OTLP endpoint (https://app.harness.io/udp-ingest/otel/v1/traces, or your cluster's equivalent).

Choose Your Path​

Your setup path depends on one question: Does your app or gateway already emit GenAI OpenTelemetry traces?

  • Yes, my stack already emits GenAI traces: Route those traces to Harness with no code changes. Use Route Existing Traces.
  • No, I need to instrument my application: Add instrumentation that emits GenAI traces, then route them to Harness. Use the Harness SDK if you call a model SDK directly in Python, or an open-source SDK if an orchestration framework runs your calls.

Not sure whether your stack emits GenAI traces? Go to Supported Providers and Frameworks to check the compatibility matrix.

PathUse whenWhat you do
Route Existing TracesYour app or gateway already emits GenAI OpenTelemetry traces (LangSmith, OpenInference, a LiteLLM proxy, or native framework export).Generate a token and point the existing exporter at the Harness endpoint. No code changes.
Instrument Your ApplicationYour app does not emit GenAI traces yet.Add the Harness SDK (direct Python SDK calls) or an open-source SDK (orchestration frameworks), then route the traces to Harness.

Instrument Your Application​

Select the tab that matches your stack. Each tab is a complete, self-contained setup, from the ingestion token to verification in Cost Explorer.

Route Existing Traces​

Use this path if your app or gateway already emits OpenTelemetry traces with GenAI semantic conventions (from LangSmith, OpenInference, a LiteLLM Proxy, LangChain, or a framework with native export). No code changes: you repoint the existing exporter at Harness.

Step 1: Generate an Authentication Token​

Harness uses a bearer token to authenticate trace ingestion against the account's OTLP endpoint.

Recommended: Create a dedicated service account for telemetry ingestion and generate an API key under that account. This isolates the credential, makes it easy to rotate, and keeps trace ingestion working independently of any individual user.

  1. Create a service account (example: ai-telemetry-ingest) and assign it a role with the minimum permissions required for ingestion.
  2. Create a service account API key and token.
  3. Select Generate Token and copy the token.
Store the token securely
  • The token is only displayed once. Store it securely (secret manager, environment variable, or vault).
  • Treat it like a password. Never commit it to source control.
  • Rotate it periodically (every 90 days recommended).

Step 2: Configure the OTLP Exporter​

Point the existing OTLP exporter at the Harness endpoint by setting these OpenTelemetry environment variables:

export OTEL_EXPORTER_OTLP_ENDPOINT=https://app.harness.io/udp-ingest/otel
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer <YOUR_TOKEN>"
export OTEL_TRACES_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
What each variable does
VariablePurpose
OTEL_EXPORTER_OTLP_ENDPOINTHarness OTLP trace ingestion endpoint. Replace app.harness.io with the account's cluster if different (example: app3.harness.io). Find the cluster in the URL when logged in to Harness.
OTEL_EXPORTER_OTLP_HEADERSBearer token for authentication. Use the literal token value or reference it from a secret/environment variable.
OTEL_TRACES_EXPORTERSelects the OTLP exporter for traces.
OTEL_EXPORTER_OTLP_PROTOCOLSelects HTTP/protobuf OTLP transport (Harness expects this format).

Framework-specific configuration

The four variables above are the standard OpenTelemetry exporter settings and work for most stacks. Some frameworks and tools also require a framework-specific flag to turn telemetry on. Pick your framework below.

No extra flag is needed. The standard variables are enough:

export OTEL_EXPORTER_OTLP_ENDPOINT=https://app.harness.io/udp-ingest/otel
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer ${HARNESS_OTEL_TOKEN}"
export OTEL_TRACES_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf

After setting the variables, restart the application or gateway so they take effect.

Step 3: Verify Traces in Cost Explorer​

  1. Wait for traces to flow. They usually appear within a few minutes; allow up to about 20 minutes.
  2. Go to Cloud & AI Cost Management > Cost Explorer.
  3. Select the AI Traces view or group by Service Name.
  4. Look for the service name (from the service.name attribute in traces).
  5. Select a service row to open the Service Traces drawer.
  6. Inspect recent runs, span waterfalls, and per-span cost attribution.

Step 4: (Optional) Send a Test Trace​

Before wiring up the full application, confirm the endpoint and token work by sending a single test span with curl.

Replace <ACCOUNT_ID> with the account identifier and <YOUR_TOKEN> with the token from Step 1, then run:

curl --request POST \
  --url 'https://app.harness.io/udp-ingest/otel/v1/traces?accountIdentifier=<ACCOUNT_ID>&routingId=<ACCOUNT_ID>' \
  --header 'content-type: application/json' \
  --header 'Authorization: Bearer <YOUR_TOKEN>' \
  --data '{
  "resourceSpans": [
    {
      "resource": {
        "attributes": [
          { "key": "harness.account.id", "value": { "stringValue": "<ACCOUNT_ID>" } },
          { "key": "service.name", "value": { "stringValue": "test-service" } }
        ]
      },
      "scopeSpans": [
        {
          "scope": { "name": "otlp-test-client" },
          "spans": [
            {
              "traceId": "aabbccdd11223344aabbccdd11223344",
              "spanId": "0000000000000001",
              "name": "test-span",
              "startTimeUnixNano": "1786104000000000000",
              "endTimeUnixNano": "1786104000500000000"
            }
          ]
        }
      ]
    }
  ]
}'

Note: startTimeUnixNano and endTimeUnixNano are example nanosecond timestamps (they resolve to a 2026 date). If the test span does not appear, regenerate current values. For example, run date +%s and append nine zeros for nanoseconds.

Then open the AI Traces view in Cost Explorer and look for test-service. It usually appears within a few minutes; allow up to about 20 minutes.

Manual Instrumentation (Advanced)​

Use manual instrumentation for languages the Harness SDK does not cover (Go, Java, .NET), custom agents, or when you need full control over span structure. You create spans with a standard OpenTelemetry SDK and set the required GenAI attributes yourself.

At minimum, each LLM span must set gen_ai.provider.name (preferred; the legacy gen_ai.system is also supported), gen_ai.request.model, gen_ai.usage.input_tokens, and gen_ai.usage.output_tokens, and export over OTLP to the Harness endpoint. Set gen_ai.agent.name if you want cost grouped by agent in Cost Explorer. Reuse the shared OpenTelemetry exporter setup from the Framework Instrumentation tab (which registers the global tracer provider that trace.get_tracer() reads from), then wrap each LLM call:

from opentelemetry import trace

tracer = trace.get_tracer(__name__)

with tracer.start_as_current_span("llm.call") as span:
    span.set_attribute("gen_ai.provider.name", "openai")  # preferred; "gen_ai.system" also supported
    span.set_attribute("gen_ai.agent.name", "support-copilot")  # optional: groups cost by agent in Cost Explorer
    span.set_attribute("gen_ai.request.model", "gpt-4-turbo")
    # ... make the LLM call ...
    span.set_attribute("gen_ai.usage.input_tokens", resp.usage.prompt_tokens)
    span.set_attribute("gen_ai.usage.output_tokens", resp.usage.completion_tokens)

For other languages, use the equivalent OpenTelemetry SDK (Go, Java, .NET) and set the same attributes. Go to the GenAI Span Attribute Reference to review the full list.

Reduce Trace Data Volume​

Large prompt and response payloads and over-instrumentation inflate span volume and storage cost. To keep trace data manageable in high-traffic production:

  • Disable payload capture: Set HA_GEN_AI_PAYLOAD_CAPTURE_ENABLED=false for the Harness SDK so raw prompt and response text is not stored on spans.
  • Scope instrumentation to LLM calls: Instrument the model calls that carry cost, not every function in the application.
  • Sample a percentage of traces: In high-traffic production, export a representative sample rather than every trace.

Next Steps​