> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/web-application-and-api-protection-waap-discovery-discovery-discovery-discovery/data-collection/gateways-load-balancers/aws/aws-cloudwatch-agent-overview/troubleshooting-the-aws-cloudwatch-agent.md).

# Troubleshooting the AWS CloudWatch agent

Two things tend to go wrong with this agent. Either the install fails outright, or it finishes without complaint and your spans never show up in Harness. This topic walks you through both.

The sections are organized by what you see, not by what caused it, so find the symptom that matches yours and start there. Some fixes depend on how you deployed the agent, and those sections are labeled. `ec2` covers the manual install and `ec2` mode in Terraform and CloudFormation. `ecs` covers `ecs` mode.

***

## The installation or deployment fails

Use this part when a failure occurs during agent installation or deployment, before the agent has collected any data. The error appears in your terminal or in the AWS console, so these sections start from what you see there.

### No collector is enabled

**Cause**: The deployment or install fails immediately with an error that contains "at least one collector must be enabled". All three collectors are off by default, and the agent will not start with any of them on.

**Resolution**: Turn on at least one collector. The following table lists the settings that turn on each collector for each installation method.

| Installation Method   | Settings                                                                                 |
| --------------------- | ---------------------------------------------------------------------------------------- |
| Terraform             | `enable_api_gateway_collector`, `enable_bedrock_collector`, `enable_agentcore_collector` |
| CloudFormation        | `EnableApiGatewayCollector`, `EnableBedrockCollector`, `EnableAgentCoreCollector`        |
| Manual install on EC2 | `--enable-api-gateway`, `--enable-bedrock`, `--enable-agentcore`                         |

Set at least one to `true`, then run the deployment or installer again.

***

### The CloudFormation stack rolls back

**Cause**: One resource in the stack failed to create, and CloudFormation rolled the whole stack back to leave nothing half-built.

**Resolution**:

1. Open the **CloudFormation** console and go to **Stacks**.
2. Select your stack and open its events.
3. Find the resource whose creation failed, and read the reason next to it.

The reason usually points straight at the cause, such as a missing parameter, a subnet in the wrong VPC, or a permission your credentials lack.

***

### The ECSCustomResource fails (ecs mode, CloudFormation only)

**Cause**: `ECSCustomResource` is a helper AWS Lambda function in the CloudFormation template. It creates the EventBridge schedules that start the agent. When it fails, the stack fails with it. The Terraform module does this work itself and has no such function.

**Resolution**:

1. In **CloudWatch**, go to **Log groups** and search for your stack name.
2. Open the log group for the Lambda function.
3. Open the log event whose timestamp matches the time you created the stack. It shows what caused the failure.

***

## Spans do not reach Harness

Start here when the install succeeded but your traffic does not appear. Work through the checks that match your deployment, in order. Most problems come from the first one.

### API Gateway logging is not configured correctly

**Cause**: The API Gateway collector reads access logs in a specific format. If the stage does not write them, or writes them in another format, there is nothing for the agent to read.

**Resolution**: Go back to the "Prepare your AWS services" step in your installation topic and check each stage: **Custom Access Logging** is on, the log group ARN is correct, and the log format matches the JSON in that step.

***

### Bedrock data does not appear

**Cause**: The Bedrock collector does not discover its log group. It reads only the log groups you name. If model invocation logging is off, or the log group is missing from your configuration, there is nothing for the agent to read.

**Resolution**:

1. Confirm that model invocation logging is turned on in Amazon Bedrock, and that it writes to a CloudWatch Logs log group. AWS describes the steps in its [model invocation logging documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-invocation-logging.html).
2. Confirm that the log group name is in the `log_groups` field of the matching account entry.
3. If the API Gateway collector is also on and you list log groups for Bedrock, confirm that you list the API Gateway log groups in the same field too.

***

### The cron job is missing or not running (ec2)

**Cause**: The agent runs from a cron job on the host. If the job does not exist, or the system does not run it, no new data leaves the host.

**Resolution**:

1. Check the agent logs in `/var/traceable/log/aws-cloudwatch`, or in the directory you set with `--log-dir`. If a healthy agent is running, new log entries appear on every run. If nothing new appears, the cron job is not running.
2. Check whether the cron file exists:

```
cat /etc/cron.d/traceable-awscloudwatch
```

This prints the schedule the installer registered. If the command reports that the file does not exist, run the installer again.

3. If the file exists but the job still does not run, another system process may be stopping it from running as a system cron job. Register it as a cron job for the `traceable` user instead:

```
crontab -u traceable /etc/cron.d/traceable-awscloudwatch
```

This loads the schedule from the cron file into the `traceable` user's own crontab, which bypasses the system-level problem.

***

### The IAM permissions are wrong, or the TPA is unreachable (ec2)

**Cause**: The cron job runs, but the agent logs show errors, and your API Gateway and log group settings are correct. In that case, the usual cause is a missing IAM permission or a TPA that the agent cannot reach.

**Resolution**: Check the following in order.

* **The IAM role is missing a permission.** Make sure that the role attached to the instance has `apigateway:GET`, `logs:DescribeLogGroups`, and `logs:FilterLogEvents`. The agent cannot find or read the logs without them. If you collect from another account, the role also needs `sts:AssumeRole`.
* **The agent cannot reach the TPA.** Make sure that the TPA is running and reporting, and that the host running the agent can reach the TPA endpoint. The agent sends everything it collects to the TPA, so nothing reaches Harness while the endpoint is unreachable.

***

### The EventBridge schedules are missing (ecs)

**Cause**: In `ecs` mode, EventBridge schedules start the agent as tasks. If the schedules do not exist, nothing ever starts. The usual reason is an incorrect IAM policy on the role that creates them.

**Resolution**:

1. Open **Amazon EventBridge** → **Schedules** and confirm that schedules exist for your deployment.
2. If none exist, open AWS CloudTrail and find the `CreateSchedule` event. A successful event has no error code. A failed one shows the reason.

***

### The schedules exist, but the tasks do not run (ecs)

**Cause**: A schedule can exist and still fail to start a task, for example when the role it uses cannot run the task.

**Resolution**: In AWS CloudTrail, find the `RunTask` events and check each one for an error code. A successful event has none. The error code identifies why ECS could not start the task, such as a permission that the role lacks.

***

### AgentCore spans do not appear

**Cause**: The AgentCore collector needs spans in `aws/spans` and, for the full trace, readable runtime logs.

**Resolution**:

1. Confirm that AgentCore observability is turned on and that spans exist in the `aws/spans` log group.
2. Confirm that the enrichment log group, `/aws/bedrock-agentcore/runtimes/*/otel-rt-logs`, is reachable by the agent's role.
3. Search the agent logs, or the task logs in `ecs` mode, for `Dropped N span(s)`. The collector drops a leaf span when it cannot enrich it, and logs the number it dropped each time. It never drops the parent of a dropped span.

***

For more information, see [Configure the AgentCore collector](https://developer.harness.io/web-application-and-api-protection-waap-discovery-discovery-discovery-discovery/data-collection/gateways-load-balancers/aws/aws-cloudwatch-agent-overview/configure-the-agentcore-collector.md) and [AWS CloudWatch agent overview](https://developer.harness.io/web-application-and-api-protection-waap-discovery-discovery-discovery-discovery/data-collection/gateways-load-balancers/aws/aws-cloudwatch-agent-overview.md).
