> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/web-application-and-api-protection-waap-discovery-discovery-api-security-testing/data-collection/gateways-load-balancers/aws/aws-cloudwatch-agent-overview/deployment-using-terraform.md).

# Deployment using Terraform

The Terraform module in this topic deploys the AWS CloudWatch agent into your AWS account. You choose whether the agent runs on an EC2 instance or as ECS Fargate tasks, and you choose which collectors it runs: API Gateway, Bedrock, AgentCore, or any mix of the three.

## What will you learn in this topic?

By the end of this topic, you will understand:

* The steps to prepare your AWS services and credentials for the agent.
* The steps to configure and apply the Terraform module.
* The purpose of each Terraform variable that you are likely to set.
* The way to verify, change, and remove the deployment.

***

## Before you begin

* Make sure that Terraform is installed on the machine where you will run the module, because you apply the module's configuration files from that machine.
* Make sure that the AWS CLI is configured with credentials that can deploy the resources in the target account and region. Step 2 covers this.
* Make sure that you know which collectors you want. You must turn on at least one, or `terraform apply` stops with an error before it creates anything.
* Make sure that you have a TPA set up. The agent sends all its data to the TPA, so follow the [TPA installation guide](https://developer.harness.io/web-application-and-api-protection-waap-discovery-discovery-api-security-testing/installation/platform-agent.md) first, then note the TPA's URL. The URL must be reachable from the place where the agent runs. You enter it as `traceable_agent_endpoint` in Step 6.
* Make sure that you have a refresh token for your Harness account. In `ec2` mode, the module passes it as `traceable_refresh_token` so that the instance can connect to Harness.
* Make sure that API Gateway logging, Bedrock model-invocation logging, or AgentCore observability is turned on for the services you want to collect. Step 1 covers all three.

***

## Understand the deployment modes

The `deployment_mode` variable decides where the agent runs. Pick it first, because the two modes use different variables.

The following table describes the two modes, what Terraform creates in each, and when to choose one over the other.

| Mode            | What Terraform Creates                                                                  | Use It When                                                                                 |
| --------------- | --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- |
| `ec2` (default) | An EC2 instance in the subnet you specify, with the agent running on it                 | You want the agent to run on one instance and you do not want to manage a container cluster |
| `ecs`           | Resources that start the agent as an ECS task on a schedule, through Amazon EventBridge | You already run an ECS Fargate cluster and want the agent to run as a task                  |

In `ecs` mode, you must supply an existing ECS Fargate cluster through `cluster_arn`. The module does not create the cluster.

***

## Step 1: Prepare your AWS services

The agent does not create logs. It reads logs that your AWS services already write to CloudWatch Logs, so each service you want to collect from has to be logging first. The three collectors read different logs, so the preparation differs for each. API Gateway needs access logs in a specific JSON format. Bedrock needs model invocation logging enabled, plus the name of its log group, in your agent configuration. AgentCore needs observability turned on and nothing more.

Do only the parts for the collectors you plan to turn on. If you turn on more than one, do each part.

### API Gateway (REST APIs)

Use this section if the API Gateway collector will read REST APIs. The collector reads each stage's access logs, so every stage you want to collect from must write them in a format the agent can parse.

Complete the following for every stage you want to collect from:

1. Open the API Gateway console and select your API, then select the stage.
2. Go to **Logs/tracing**.
3. In the **CloudWatch Logs** drop-down, select **Error and info logs**.
4. Turn on **Custom Access Logging**.
5. In **Access log destination ARN**, enter the ARN of a CloudWatch log group. This is where API Gateway writes the access logs that the agent reads.
6. In **Log Format**, paste the following JSON:

```json
{
  "requestId": "$context.requestId",
  "ip": "$context.identity.sourceIp",
  "httpMethod": "$context.httpMethod",
  "path": "$context.path",
  "status": "$context.status",
  "responseLength": "$context.responseLength",
  "domainName": "$context.domainName"
}
```

Each field maps to an API Gateway `$context` variable. API Gateway fills the variables in for every request, so each log line carries the request ID, the caller's IP address, the HTTP method, the path, the status code, the response size, and the domain name. The agent expects these fields in every log line, so leave none out.

### API Gateway (HTTP APIs)

Use this section if the API Gateway collector will read HTTP APIs. The agent reads them the same way it reads REST APIs, so the log format is the same. Only the place where you turn logging on differs.

Turn on access logging for the stage and use the same JSON format as in the REST APIs section. That way, every log line carries the fields the agent needs.

### Bedrock

Use this section if you turn on the Bedrock collector. The collector reads Bedrock's model invocation logs from CloudWatch Logs, so you need to complete two tasks: turn on logging in Amazon Bedrock, and tell the agent which log group to read.

{% hint style="info" %}
**Note**: The Bedrock collector reads model invocation logs, which AWS provides for Amazon Bedrock Runtime. It does not collect data from Bedrock Mantle, because AWS does not support model invocation logging for it.
{% endhint %}

1. Turn on model invocation logging in Amazon Bedrock, and send the logs to a CloudWatch Logs log group. AWS describes the steps in its [model invocation logging documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-invocation-logging.html).
2. Add the name of that log group to the `log_groups` field of the matching entry in `accounts`, which [Step 6](#step-6-create-the-terraform.tfvars-file) explains. The Bedrock collector does not discover this log group on its own, so it reads only the log groups that you list.

If the API Gateway collector is also on, list the API Gateway log groups in the same `log_groups` field as well.

### AgentCore

Use this section if you turn on the AgentCore collector. It needs the least setup on your side, because the agent finds its own log groups. All it needs is for AgentCore to produce observability data.

Turn on AgentCore observability by following the [AWS documentation](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/observability-configure.html#observability-configure-builtin). The agent discovers all AgentCore log groups on its own, so you do not list any.

***

## Step 2: Configure AWS credentials

Terraform uses your AWS credentials to create resources, and it deploys into the region those credentials point . Configure them with the AWS CLI:

```
aws configure
```

This prompts for your AWS access key ID, secret access key, default region name, and default output format (use `json`). The CLI stores the answers so that Terraform and other AWS tools can find them.

If you use named profiles, tell the CLI which one to use:

```
export AWS_PROFILE=<profile-name>
```

This sets the profile for your current shell session. The profile must hold the credentials you use to deploy the Harness resources.

Then check the region:

```
aws configure get region
```

The command prints the default region. It should match the region where you want to install the agent, because a mismatch deploys the resources somewhere you did not intend.

If you prefer environment variables to `aws configure`, set `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION` (or `AWS_DEFAULT_REGION`), and `AWS_DEFAULT_OUTPUT` instead.

***

## Step 3: Create a cross-account IAM role (multiple accounts only)

Skip this step if every API gateway you want to monitor lives in the same account as the agent.

To collect from another account, the agent needs permission to read that account's logs. It gets that permission by assuming an IAM role that you create in the monitored account. You create one role per monitored account.

Use the following Terraform template to create the role. Run it with credentials for the monitored account, because the role must exist in that account.

```hcl
data "aws_iam_policy_document" "traceable" {
  statement {
    sid = "readAccess"
    actions = [
      "logs:describeLogGroups",
      "logs:FilterLogEvents",
      "apigateway:GET"
    ]
    effect    = "Allow"
    resources = ["*"]
  }
}

resource "aws_iam_policy" "traceable" {
  name        = "traceable-cross-account-policy"
  description = "IAM policy for traceable instance"
  path        = "/"
  policy      = data.aws_iam_policy_document.traceable.json
}

resource "aws_iam_role" "traceable" {
  name                  = "traceable-cross-account-role"
  description           = "IAM role for traceable instance"
  path                  = "/"
  force_detach_policies = true
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          AWS = "arn:aws:iam::${var.account_id}:root"
        }
        Condition = {}
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "traceable" {
  depends_on = [aws_iam_policy.traceable, aws_iam_role.traceable]
  role       = aws_iam_role.traceable.name
  policy_arn = aws_iam_policy.traceable.arn
}

variable "account_id" {
  type = string
}
```

Here is what each piece does:

* The policy document grants read-only access: it can list log groups, read log events, and read API Gateway details. It applies to all resources (`*`) in the account.
* The role carries the trust policy. The `Principal` line decides who may assume the role, and `${var.account_id}` is the account you trust.
* The attachment links the policy to the role.
* `account_id` is the one value you supply. It is the ID of the AWS account that assumes the role, which is the **host account**, the account where the agent runs.

{% hint style="info" %}
**Note**: The trust policy names the whole account (`:root`), so any identity in that account with `sts:AssumeRole` permission can assume this role. If your security team wants a narrower trust, restrict the principal or add a condition.
{% endhint %}

After you apply the template, copy the ARN of the new role. You enter it as `cross_account_role_arn` in Step 6.

***

## Step 4: Download and extract the module

Download the Terraform module:

```
curl --remote-name "https://downloads.traceable.ai/install/aws-cloudwatch/terraform/latest/aws-cloudwatch-tf.tar.gz"
```

The `--remote-name` option (short form `-O`) saves the file under its original name, `aws-cloudwatch-tf.tar.gz`, in your current directory. The `latest` segment of the path selects the most recent release of the module.

Extract the archive and move into the directory:

```
tar xvzf aws-cloudwatch-tf.tar.gz && cd aws-cloudwatch-tf
```

`tar xvzf` unpacks the gzip archive and lists each file as it goes. The `&& cd` part moves you into the new directory only if the extraction succeeded. Run every remaining command from this directory.

***

## Step 5: Initialize Terraform

Before Terraform can create anything, it needs to download the providers the module uses. Run:

```
terraform init
```

This downloads those providers and prepares the directory for the next command. You run it once per directory, and again if you change providers. Make a note of this directory. Terraform stores its state here, so you must run `terraform destroy` from the same directory when you uninstall the module.

***

## Step 6: Create the terraform.tfvars file

Terraform reads your settings from a file named `terraform.tfvars` in the module directory. The example below is for `ec2` mode with the API Gateway and Bedrock collectors turned on. Replace each placeholder with your own value.

```hcl
subnet_id                = "<subnet-id>"
key_name                 = "<key-pair-name>"
traceable_refresh_token  = "<traceable-refresh-token>"
traceable_agent_endpoint = "<tpa-endpoint>"
traceable_environment    = "<environment-name>"
traceable_service_name   = "aws-cloudwatch"
traceable_api_endpoint   = "api.traceable.ai"

# Turn on at least one collector
enable_api_gateway_collector = true
enable_bedrock_collector     = true
enable_agentcore_collector   = false

# One entry for each account and region you want to collect from
accounts = [
  {
    region                 = "us-east-1"
    api_list               = ["<api-id>/<stage-name>"]
    cross_account_role_arn = ""
    exclude                = false
  }
]

tags = {
  owner  = "<team-name>"
  reason = "aws cloudwatch"
}

service_version = "latest"
```

The comment in the example marks the setting that most often causes errors. You must set at least one collector setting to `true`, because the module refuses to apply with all three off. The `accounts` list has one entry per account and region, so a second region needs a second entry.

The variables are grouped by when you need them. The sections below cover the variables that apply to both modes, the ones specific to `ec2` or `ecs`, the ones that send credentials to a hosted TPA, and the `accounts` variable in detail.

### Variables that apply to both modes

These variables apply whichever mode you choose, so start here. The following table lists each one with its default and the reason you might change it.

| Variable                         | Required | Default                 | What It Does and Why You Might Change It                                                                           |
| -------------------------------- | -------- | ----------------------- | ------------------------------------------------------------------------------------------------------------------ |
| `accounts`                       | Yes      | None                    | The accounts and regions to collect from. See [Configure the accounts variable](#configure-the-accounts-variable). |
| `subnet_id`                      | Yes      | `""`                    | The subnet where the agent runs. Choose one that can reach your TPA endpoint and the AWS APIs.                     |
| `deployment_mode`                | No       | `ec2`                   | Where the agent runs: `ec2` or `ecs`. Set it to `ecs` to run the agent as ECS tasks.                               |
| `enable_api_gateway_collector`   | See note | `false`                 | Turns on the API Gateway collector.                                                                                |
| `enable_bedrock_collector`       | See note | `false`                 | Turns on the Bedrock collector.                                                                                    |
| `enable_agentcore_collector`     | See note | `false`                 | Turns on the AgentCore collector.                                                                                  |
| `traceable_agent_endpoint`       | No       | `http://localhost:5442` | Where the agent sends data. Change it when your TPA runs somewhere else.                                           |
| `traceable_agent_max_batch_size` | No       | Set by the module       | The maximum number of spans in one request to the TPA. Lower it if requests to your TPA time out.                  |
| `traceable_service_name`         | No       | `aws-cloudwatch`        | The service name Harness shows. It applies to the API Gateway and Bedrock collectors only.                         |
| `cron_interval_mins`             | No       | `5`                     | How often the agent runs, in minutes. A shorter interval brings data in sooner, and a longer one means fewer runs. |
| `max_concurrency`                | No       | `8`                     | The maximum number of agent executions that can run at the same time.                                              |
| `log_group_arns`                 | No       | `*`                     | The log groups the agent may read. Narrow it to limit what the agent's IAM policy allows.                          |
| `api_gateway_stages_arns`        | No       | `*`                     | The API Gateway stages the agent may read, in the format `restapis/<api_id>/stages/<stage_name>`.                  |
| `tags`                           | No       | None                    | Tags to add to every resource the module creates.                                                                  |

{% hint style="info" %}
**Note**: You must set at least one of the three collector variables to `true`.
{% endhint %}

### Variables for ec2 mode

Use these variables only when `deployment_mode` is `ec2`. They control the instance the module creates and where the agent installs on it. The following table lists each one.

| Variable                  | Required | Default                                                                 | What It Does and Why You Might Change It                                                                    |
| ------------------------- | -------- | ----------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
| `key_name`                | No       | `""`                                                                    | The key pair for the instance. Set it if you need SSH access to troubleshoot.                               |
| `instance_type`           | No       | `m4.xlarge`                                                             | The EC2 instance type. Pick a larger type if you collect heavy traffic.                                     |
| `traceable_refresh_token` | Yes      | None                                                                    | The refresh token for your Harness account. The instance uses it to authenticate with the Harness platform. |
| `traceable_api_endpoint`  | No       | `api.traceable.ai`                                                      | The Harness platform API endpoint. Change it if your account lives in another region.                       |
| `tpa_version`             | No       | `<TPA_VERSION>`                                                         | The TPA version to install.                                                                                 |
| `service_version`         | No       | `latest`                                                                | The agent version to install.                                                                               |
| `package_url`             | No       | `""`                                                                    | A custom location for the install package.                                                                  |
| `root_directory`          | No       | `/var/traceable/aws-cloudwatch`                                         | Where the agent is installed on the instance.                                                               |
| `log_directory`           | No       | `/var/traceable/log/aws-cloudwatch`                                     | Where the agent writes its logs. You look here when you troubleshoot.                                       |
| `ami_name_ssm`            | No       | `/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64` | The SSM parameter that holds the AMI ID. The default resolves to the latest Amazon Linux 2023 image.        |

### Variables for ecs mode

Use these variables only when `deployment_mode` is `ecs`. They describe the cluster and the task the agent runs as. The following table lists each one.

| Variable                | Required | Default                            | What It Does and Why You Might Change It                                                                          |
| ----------------------- | -------- | ---------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
| `cluster_arn`           | Yes      | None                               | The ARN of the ECS Fargate cluster that runs the tasks.                                                           |
| `agent_image`           | Yes      | `traceableai/awscloudwatch:latest` | The container image that ECS runs for each task. Change it only if you mirror the image into your own registry.   |
| `ecs_task_resources`    | No       | `{ cpu = 1024, memory = 2048 }`    | CPU (in millicores) and memory (in MB) for each task.                                                             |
| `enable_logging`        | No       | `false`                            | Saves the task logs to CloudWatch. Turn it on while you troubleshoot.                                             |
| `allow_images_from_ecr` | No       | `true`                             | Lets the task's IAM role pull images from Amazon ECR. Leave it on if you mirror the image into your own registry. |

### Variables that send credentials to a hosted TPA

Set these only if your agent sends data to a Harness-hosted (SaaS) TPA. The TPA uses them to identify your environment and authenticate the agent. The following table lists the three variables and the request header each one becomes.

| Variable                | Default | What It Does                                                  |
| ----------------------- | ------- | ------------------------------------------------------------- |
| `traceable_environment` | `""`    | Sent to the TPA as the `traceableai-environment-name` header. |
| `traceable_token`       | `""`    | Sent to the TPA as the `traceableai-agent-token` header.      |
| `traceable_guid`        | `""`    | Sent to the TPA as the `x-traceable-guid` header.             |

The agent leaves out any header whose value is empty. You need these three only when `traceable_agent_endpoint` points to a Harness-hosted (SaaS) TPA. For a self-hosted TPA, leave them empty.

### Configure the accounts variable

The `accounts` variable is the one that tells the agent where to look, so it deserves a closer read. Each item in the list describes one account and region pair. The following table describes each key you can set inside an item.

| Key                      | Required | What It Does                                                                                                                                                                                                                            |
| ------------------------ | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `region`                 | Yes      | The AWS region to collect from.                                                                                                                                                                                                         |
| `api_list`               | Yes      | A list of API IDs. To target one stage, write `<api_id>/<stage_name>`.                                                                                                                                                                  |
| `exclude`                | Yes      | Decides how the agent treats the `api_list` entries. It applies to the API Gateway collector only. Set it to `false` to collect only the listed APIs. Set it to `true` to collect everything except the listed APIs.                    |
| `cross_account_role_arn` | Yes      | The ARN of the role from Step 3. Use an empty string when the APIs are in the same account as the agent.                                                                                                                                |
| `log_groups`             | No       | A list of log group names to read directly. For the Bedrock collector, list the model-invocation log group here. If the API Gateway collector is also on and you list log groups for Bedrock, list the API Gateway log groups here too. |
| `access_log_format`      | No       | A map that describes the access log format your API Gateway instances use. It is required when you set `log_groups`.                                                                                                                    |

{% hint style="info" %}
**Note**: The AgentCore collector uses only `region` and `cross_account_role_arn` from each entry, and it discovers its own log groups.
{% endhint %}

***

## Step 7: Apply the module

Applying the module is the step that creates the resources. Run:

```
terraform apply
```

Terraform first shows you the plan, which is the list of resources it will create, and waits for you to confirm. When you do, it creates them. Review the plan the first time you apply, because it is the quickest way to catch a wrong region, subnet, or mode before any resource exists.

***

## Step 8: Verify the deployment

Give the agent one run interval (5 minutes by default), then check Harness for your traffic:

* Go to **Insights** → **Explore Data** to view the traces and spans directly.
* Go to **Discovery** → **Inventory** to view the services and endpoints the agent discovered.

API Gateway and Bedrock traffic appear under the service name you set in `traceable_service_name`, which is `aws-cloudwatch` by default. AgentCore traffic appears under the name of the AgentCore resource.

If nothing arrives, go through the checks in [Troubleshooting the AWS CloudWatch agent](https://developer.harness.io/web-application-and-api-protection-waap-discovery-discovery-api-security-testing/data-collection/gateways-load-balancers/aws/aws-cloudwatch-agent-overview/troubleshooting-the-aws-cloudwatch-agent.md).

***

## Change or remove the deployment

To change a setting, edit `terraform.tfvars` and run `terraform apply` again. Terraform compares the new settings with what exists and changes only the difference.

To remove everything the module created, run the following from the directory where you ran `terraform init`:

```
terraform destroy
```

Terraform reads its saved state from that directory, so running the command anywhere else leaves the resources in place.

For more information, see [AWS CloudWatch agent overview](https://developer.harness.io/web-application-and-api-protection-waap-discovery-discovery-api-security-testing/data-collection/gateways-load-balancers/aws/aws-cloudwatch-agent-overview.md) and [Configure the AgentCore collector](https://developer.harness.io/web-application-and-api-protection-waap-discovery-discovery-api-security-testing/data-collection/gateways-load-balancers/aws/aws-cloudwatch-agent-overview/configure-the-agentcore-collector.md).
