> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/web-application-and-api-protection-waap-discovery-discovery-api-security-testing/data-collection/gateways-load-balancers/aws/aws-cloudwatch-agent-overview.md).

# AWS CloudWatch agent overview

The AWS CloudWatch agent collects API traffic data from AWS services that write logs to Amazon CloudWatch Logs and sends it to Harness. Today it covers three services: API Gateway, Amazon Bedrock, and Amazon Bedrock AgentCore. The agent reads the CloudWatch logs that these services already write. You do not need to change the applications behind them.

If you already run the AWS API Gateway agent, this agent replaces it. For the migration steps, see [Migrate from the AWS API Gateway agent](https://developer.harness.io/web-application-and-api-protection-waap-discovery-discovery-api-security-testing/data-collection/gateways-load-balancers/aws/aws-cloudwatch-agent-overview/migrate-from-the-aws-api-gateway-agent.md).

## What will you learn in this topic?

By the end of this topic, you will understand:

* The purpose of the AWS CloudWatch agent and the AWS services it collects from.
* The way the agent moves data from CloudWatch Logs to Harness.
* The purpose of each collector and the setting that turns it on.
* The steps to choose the installation method that fits your environment.

***

## Before you begin

Before you install the agent, whichever method you use, complete the following:

* Turn on logging for the AWS services you want to monitor. The agent does not generate data itself. It reads the logs that API Gateway, Amazon Bedrock, and Amazon Bedrock AgentCore write to Amazon CloudWatch Logs, so a service that does not write logs there gives the agent nothing to collect.
* Decide which collectors you need: API Gateway, Bedrock, AgentCore, or any mix. All three are off by default, and you must turn on at least one.
* [Set up a TPA](https://developer.harness.io/web-application-and-api-protection-waap-discovery-discovery-api-security-testing/installation/platform-agent.md), make sure the host or task is running, and ensure the host or task that runs the agent can reach its endpoint. The agent sends everything it collects to the TPA, which passes it on to Harness.
* Confirm that you have permission to deploy resources in the AWS account that hosts the agent. Terraform and CloudFormation create the instance or tasks, and the IAM roles for you, and the manual install needs access to the host.

Each installation topic lists the extra prerequisites for that method.

***

## How the agent works

The agent does not sit in the path of your API traffic. It reads the logs your AWS services have already written, so it sees each request only after it has finished.

The flow has four stages:

1. Your AWS services write logs to CloudWatch log groups.
2. On a schedule, the agent reads the log groups for every collector you enabled. The default interval is 5 minutes.
3. The agent sends the data to the TPA endpoint that you configure. The default endpoint is `http://localhost:5442`.
4. The TPA passes the data to the Harness platform. You see it under **Insights** → **Explore Data** and **Discovery** → **Inventory**.

```mermaid
flowchart TD
    A["API Gateway logs<br/>Access, execution logs"] --> D
    B["Bedrock logs<br/>Model-invocation logs"] --> D
    C["AgentCore logs<br/>aws/spans, runtime logs"] --> D
    D["AWS CloudWatch agent<br/>Default run: every 5 minutes"] --> E
    E["TPA<br/>Receives data from the agent"] --> F
    F["Harness platform<br/>Data appears in the UI"]
```

***

## Understand the collectors

The agent has three collectors. Each one reads a different set of log groups, and each one is independent of the others. You turn on only the ones you need.

The following table summarizes what each collector reads and the TPA path it sends the data to.

| Collector   | What It Reads                                                                                                          | Where It Sends Data                         |
| ----------- | ---------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- |
| API Gateway | API Gateway access and execution log groups                                                                            | `$TPA_ENDPOINT/ext_cap/v1/req_res_list_cap` |
| Bedrock     | The Bedrock model-invocation log group, which you name in the `log_groups` field                                       | `$TPA_ENDPOINT/ext_cap/v1/req_res_list_cap` |
| AgentCore   | The `aws/spans` log group, plus the AgentCore runtime log groups for enrichment. The agent discovers these on its own. | `$TPA_ENDPOINT/v1/traces` (OTLP)            |

All three collectors are off by default, and you must turn on at least one. If you turn on none, the deployment or installation fails with the error "at least one collector must be enabled".

Each installation method has its own setting for each collector. The following table lists the setting names for every method and the type of value each one takes.

| Installation Method   | Setting Names                                                                            | Value Type                                 |
| --------------------- | ---------------------------------------------------------------------------------------- | ------------------------------------------ |
| Terraform             | `enable_api_gateway_collector`, `enable_bedrock_collector`, `enable_agentcore_collector` | Boolean (`true` or `false`)                |
| CloudFormation        | `EnableApiGatewayCollector`, `EnableBedrockCollector`, `EnableAgentCoreCollector`        | String (`"true"` or `"false"`)             |
| Manual install on EC2 | `--enable-api-gateway`, `--enable-bedrock`, `--enable-agentcore`                         | Explicit `true` or `false` after each flag |

{% hint style="info" %}
**Note**: The service name that Harness shows for your traffic depends on the collector. The service name setting (`traceable_service_name`, `TraceableServiceName`, or `--service-name`) applies to the API Gateway and Bedrock collectors only. The AgentCore collector always takes its service name from the AgentCore resource (the AgentCore agent or runtime name), so the setting has no effect on it.
{% endhint %}

{% hint style="info" %}
**Note**: The Bedrock collector reads model invocation logs, which AWS provides for Amazon Bedrock Runtime. It does not collect data from Bedrock Mantle, because AWS does not support model invocation logging for it.
{% endhint %}

For more information about the AgentCore collector, see [Configure the AgentCore collector](https://developer.harness.io/web-application-and-api-protection-waap-discovery-discovery-api-security-testing/data-collection/gateways-load-balancers/aws/aws-cloudwatch-agent-overview/configure-the-agentcore-collector.md).

***

## Choose an installation method

You can install the agent in three ways. They all run the same agent, so the choice comes down to how your team already manages infrastructure.

The following table compares the three installation methods by who they suit best and how each one runs the agent.

| Installation Method   | Best For                                                                                              | How It Runs                                                                                         |
| --------------------- | ----------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
| Terraform             | Teams that manage AWS with infrastructure as code                                                     | Creates the resources for you, as an EC2 instance (`ec2` mode) or as ECS Fargate tasks (`ecs` mode) |
| CloudFormation        | AWS-native deployments that you can repeat from a template                                            | Creates the resources for you, in the same two modes                                                |
| Manual install on EC2 | An EC2 or other Linux host you already manage, or an airgapped host that cannot pull container images | Registers the agent as a cron job on that host                                                      |

In `ecs` mode, an Amazon EventBridge schedule starts an ECS task on each run, and you provide the ECS Fargate cluster. In `ec2` mode, the stack creates an EC2 instance that runs the agent. The manual install skips both and runs the agent on a host you choose.

```mermaid
flowchart LR
    subgraph EC2["ec2 mode"]
        A1["EC2 instance created for you<br/>Agent runs on a schedule"]
    end
    subgraph ECS["ecs mode"]
        B1["Amazon EventBridge schedule"] --> B2["ECS Fargate task<br/>Runs on your existing cluster"]
    end
    subgraph MAN["Manual install"]
        C1["Your own Linux host<br/>Cron job runs the agent"]
    end
    A1 --> T["TPA endpoint"]
    B2 --> T
    C1 --> T
```

For the steps, see:

* [Deployment using Terraform](https://developer.harness.io/web-application-and-api-protection-waap-discovery-discovery-api-security-testing/data-collection/gateways-load-balancers/aws/aws-cloudwatch-agent-overview/deployment-using-terraform.md)
* [Deployment using CloudFormation](https://developer.harness.io/web-application-and-api-protection-waap-discovery-discovery-api-security-testing/data-collection/gateways-load-balancers/aws/aws-cloudwatch-agent-overview/deployment-using-cloudformation.md)
* [Manual install on EC2](https://developer.harness.io/web-application-and-api-protection-waap-discovery-discovery-api-security-testing/data-collection/gateways-load-balancers/aws/aws-cloudwatch-agent-overview/manual-install-on-ec2.md)

***

## Collect from multiple accounts and regions

One host account can collect from API gateways in other AWS accounts. The agent assumes a cross-account IAM role in each account it monitors, and that role gives it read access to the logs and API Gateway details it needs.

```mermaid
flowchart LR
    H["Host account<br/>Where the agent runs"] -- "assumes role" --> R1["Monitored account A<br/>traceable-cross-account-role"]
    H -- "assumes role" --> R2["Monitored account B<br/>traceable-cross-account-role"]
    R1 --> L1["CloudWatch logs<br/>and API Gateway details"]
    R2 --> L2["CloudWatch logs<br/>and API Gateway details"]
```

Each region needs its own entry, even within a single account, so add one entry per account and region to the `accounts` setting.

***

## Understand the AWS permissions the agent uses

The agent needs read access to CloudWatch Logs and API Gateway. The Terraform and CloudFormation methods create the required IAM roles and policies for you. For a manual install, you attach the permissions to the host’s IAM role or instance profile yourself, as described in the [Manual install on EC2](https://developer.harness.io/web-application-and-api-protection-waap-discovery-discovery-api-security-testing/data-collection/gateways-load-balancers/aws/aws-cloudwatch-agent-overview/manual-install-on-ec2.md) topic. To collect from another AWS account, the cross-account role in that account needs the same read permissions.

The following table lists each permission the agent uses and the reason it needs it.

| Permission               | Why the Agent Needs It                                                                                                         |
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------ |
| `logs:DescribeLogGroups` | Lists log groups, so the agent can find the ones to read. The AgentCore collector also uses it to discover runtime log groups. |
| `logs:FilterLogEvents`   | Reads log events from the log groups.                                                                                          |
| `apigateway:GET`         | Reads API Gateway details for the APIs and stages you listed.                                                                  |
| `sts:AssumeRole`         | Lets the agent assume the cross-account role. You need it only when you monitor another account.                               |

For more information, see [Deployment using Terraform](https://developer.harness.io/web-application-and-api-protection-waap-discovery-discovery-api-security-testing/data-collection/gateways-load-balancers/aws/aws-cloudwatch-agent-overview/deployment-using-terraform.md), [Deployment using CloudFormation](https://developer.harness.io/web-application-and-api-protection-waap-discovery-discovery-api-security-testing/data-collection/gateways-load-balancers/aws/aws-cloudwatch-agent-overview/deployment-using-cloudformation.md), and [Manual install on EC2](https://developer.harness.io/web-application-and-api-protection-waap-discovery-discovery-api-security-testing/data-collection/gateways-load-balancers/aws/aws-cloudwatch-agent-overview/manual-install-on-ec2.md).
