> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/resilience-testing/chaos-engineering/faults/chaos-fault-categories/aws/ec2-http-modify-header.md).

# EC2 HTTP modify header

EC2 HTTP modify header is an AWS chaos fault that adds, changes, or removes HTTP headers on a configurable port of a target EC2 instance for a configurable duration. The fault can rewrite headers on requests entering the service or on responses leaving the service (controlled by `HEADERS_MODE`). The fault interposes a transparent HTTP proxy on the instance, scoped to `TARGET_SERVICE_PORT` and `NETWORK_INTERFACE`, and dispatches the proxy via AWS Systems Manager Run Command.

Use this fault to test how clients and servers react when expected headers are missing or malformed: do auth checks fail safely when the auth header is stripped, do cache layers misbehave when cache headers change, does the application surface a clean error or behave erratically?

{% hint style="info" %}
**RUN YOUR FIRST EXPERIMENT**

If you have not configured the chaos infrastructure yet, go to [Quickstart](/resilience-testing/chaos-engineering/new-to-chaos-engineering/quickstart.md) to install the chaos infrastructure and run an experiment end to end.
{% endhint %}

***

### Use cases <a href="#use-cases" id="use-cases"></a>

Run this fault when you want to answer concrete questions like:

* **Missing auth header on requests:** When `Authorization` is stripped from incoming requests, does the server return `401` cleanly or crash?
* **Missing content-type on responses:** When `Content-Type` is stripped from responses, do clients fall back gracefully?
* **Invalid CORS headers:** When CORS headers are changed, do browser clients honour the changes correctly?
* **Cache control churn:** When `Cache-Control` is set to `no-store`, do downstream caches honour it and how does this affect load?
* **Tracing-header propagation:** When tracing headers are removed, does the trace break exactly where expected (a useful negative test)?

***

### Prerequisites <a href="#prerequisites" id="prerequisites"></a>

* **Kubernetes version:** 1.21 or later for the chaos infrastructure cluster.
* **Target instance is reachable via SSM:** The instance has the SSM Agent running and an instance profile with `AmazonSSMManagedInstanceCore`.
* **Selector provided:** Either `EC2_INSTANCE_ID` or `EC2_INSTANCE_TAG` is set.
* **HTTP service runs on TARGET\_SERVICE\_PORT:** A plaintext HTTP service is listening on `TARGET_SERVICE_PORT`.
* **AWS credentials available:** Either an AWS credentials file uploaded as a **File Secret in Harness Secret Manager** (see Authentication below) or IRSA on the chaos infrastructure service account.

***

### Supported environments <a href="#supported-environments" id="supported-environments"></a>

| Platform                                    | Support status                                           |
| ------------------------------------------- | -------------------------------------------------------- |
| Amazon EC2 (Linux instances with SSM Agent) | Supported                                                |
| Amazon EKS managed worker nodes             | Supported (if SSM Agent is installed)                    |
| Amazon EKS self-managed worker nodes        | Supported (if SSM Agent is installed)                    |
| Targeting by tag                            | Supported via `EC2_INSTANCE_TAG`                         |
| Targeting by ID                             | Supported via `EC2_INSTANCE_ID`                          |
| HTTPS traffic (TLS-encrypted)               | Not supported on the target port; terminate TLS upstream |
| Windows instances                           | Not supported (Linux-only proxy)                         |

***

### Permissions required <a href="#permissions-required" id="permissions-required"></a>

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeInstances",
        "ec2:DescribeInstanceStatus"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "ssm:SendCommand",
        "ssm:CancelCommand",
        "ssm:GetCommandInvocation",
        "ssm:DescribeInstanceInformation",
        "ssm:GetDocument",
        "ssm:DescribeDocument"
      ],
      "Resource": "*"
    }
  ]
}
```

Go to [common policy for all AWS faults](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/aws/security-configurations/policy-for-all-aws-faults.md) to use a single superset IAM policy.

***

### Authentication <a href="#authentication" id="authentication"></a>

The fault supports three credential delivery models. Pick one based on how your chaos infrastructure is deployed.

| Method                                | When to use it                                                                 | How to configure                                                                                                                                                                                                                    |
| ------------------------------------- | ------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Harness Secret Manager file secret    | Chaos infrastructure runs outside EKS, or you want explicit static credentials | Upload the AWS credentials file as a **File Secret** in Harness Secret Manager and reference its identifier via `AWS_AUTHENTICATION_SECRET`                                                                                         |
| IAM Roles for Service Accounts (IRSA) | Chaos infrastructure runs in EKS and uses an OIDC-bound service account        | No tunable changes; the chaos pod inherits the role automatically. Go to [AWS IAM integration](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/aws/security-configurations/aws-iam-integration.md) to set it up |
| Assume role                           | The fault needs to act in a different account or with elevated permissions     | Set `ASSUME_ROLE_ARN` to the role ARN; the chaos pod assumes the role on top of its base credentials                                                                                                                                |

When using the Harness Secret Manager method, the File Secret should contain an AWS credentials file in the standard `~/.aws/credentials` format:

```ini
[default]
aws_access_key_id = REPLACE_WITH_ACCESS_KEY_ID
aws_secret_access_key = REPLACE_WITH_SECRET_ACCESS_KEY
```

Upload this file as a **File Secret** in Harness Secret Manager (Project Setup → Secrets → New File Secret), and pass the secret identifier in `AWS_AUTHENTICATION_SECRET`.

***

### Fault tunables <a href="#fault-tunables" id="fault-tunables"></a>

**Required parameters**

| Tunable                                   | Description                                                                                                                                                     | Default            |
| ----------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ |
| `REGION`                                  | AWS region that hosts the target instance.                                                                                                                      | (required)         |
| `EC2_INSTANCE_ID` *or* `EC2_INSTANCE_TAG` | One of these must be set to select the target instance(s).                                                                                                      | `""`               |
| `TARGET_SERVICE_PORT`                     | Port the target HTTP service listens on.                                                                                                                        | `80`               |
| `HEADERS_MAP`                             | JSON object mapping header names to values to inject. Setting a value to an empty string removes the header. Example: `{"X-Auth": "fake", "Content-Type": ""}`. | `{"key": "value"}` |
| `HEADERS_MODE`                            | Side to apply the rewrite to: `request` (incoming requests to the service) or `response` (outgoing responses from the service).                                 | `response`         |

**Chaos parameters**

| Tunable                  | Description                                                                                          | Default    |
| ------------------------ | ---------------------------------------------------------------------------------------------------- | ---------- |
| `NETWORK_INTERFACE`      | Network interface where the HTTP proxy intercepts traffic.                                           | `eth0`     |
| `TOTAL_CHAOS_DURATION`   | Duration of the fault in seconds.                                                                    | `30`       |
| `INSTANCE_AFFECTED_PERC` | Percentage of matching instances to target (only with `EC2_INSTANCE_TAG`). `0` targets one instance. | `0`        |
| `INSTALL_DEPENDENCIES`   | Install the in-instance HTTP proxy if missing. Set to `False` to skip.                               | `True`     |
| `PROXY`                  | HTTP/HTTPS proxy used by the in-instance installer.                                                  | `""`       |
| `SEQUENCE`               | Order in which multiple instances are processed: `parallel` or `serial`.                             | `parallel` |
| `RAMP_TIME`              | Wait period in seconds before and after the fault.                                                   | `0`        |

**Authentication**

| Tunable                     | Description                                                                                                                       | Default |
| --------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | ------- |
| `ASSUME_ROLE_ARN`           | ARN of an IAM role to assume on top of the base credentials.                                                                      | `""`    |
| `AWS_AUTHENTICATION_SECRET` | Identifier of the **File Secret in Harness Secret Manager** that contains the AWS credentials file. Not required when using IRSA. | `""`    |

{% hint style="info" %}
**REMOVE HEADERS WITH THE EMPTY STRING**

To remove a header rather than overwrite it, set its value to `""` in `HEADERS_MAP`. For example, `{"Authorization": ""}` strips the auth header from every request when `HEADERS_MODE=request`.
{% endhint %}

***

### Fault execution in brief <a href="#fault-execution-in-brief" id="fault-execution-in-brief"></a>

Sends an SSM Run Command to the selected instance(s) in `REGION` that interposes an HTTP proxy on `NETWORK_INTERFACE` for traffic destined to `TARGET_SERVICE_PORT`; the proxy applies the header rewrites in `HEADERS_MAP` on either incoming requests or outgoing responses (per `HEADERS_MODE`) for `TOTAL_CHAOS_DURATION` seconds before being removed.

***

### Expected behavior during fault execution <a href="#expected-behavior-during-fault-execution" id="expected-behavior-during-fault-execution"></a>

* Every HTTP request (`HEADERS_MODE=request`) or response (`HEADERS_MODE=response`) is modified by the rewrites in `HEADERS_MAP`.
* Headers explicitly set in `HEADERS_MAP` are added or overwritten.
* Headers set to an empty string are removed.
* Other ports and non-HTTP traffic are unaffected.
* Application behavior depends on which header was modified: auth changes typically cause `401`/`403`, content-type changes typically cause parse errors, cache-control changes affect downstream caches.

{% hint style="info" %}
**WHEN THE FAULT ENDS**

The chaos pod removes the HTTP proxy. Headers return to normal immediately. Clients or downstream caches that captured the modified headers continue using them until they refresh.
{% endhint %}

#### Signals to watch <a href="#signals-to-watch" id="signals-to-watch"></a>

* **Observed headers:** Use a [command probe](/resilience-testing/chaos-testing/probes/command-probe.md) that runs `curl -I http://localhost:<port>/` via SSM and asserts on the headers.
* **Auth failures:** Use a Prometheus probe on the application's auth-failure counter when stripping `Authorization`.
* **Client-side cache hit rate:** When changing `Cache-Control`, watch downstream cache hit rate metrics for a drop.

***

### Verify the fault execution effect <a href="#verify-the-fault-execution-effect" id="verify-the-fault-execution-effect"></a>

While the experiment is running:

1. **Inspect response headers via SSM.**

   ```bash
   aws ssm send-command \
     --region <region> \
     --instance-ids <id> \
     --document-name AWS-RunShellScript \
     --parameters 'commands=["curl -I http://localhost:<TARGET_SERVICE_PORT>/"]'
   ```

   The output should reflect the headers in `HEADERS_MAP`.

***

### Recovery and cleanup <a href="#recovery-and-cleanup" id="recovery-and-cleanup"></a>

* **End of duration:** The chaos pod stops the HTTP proxy and removes the redirection rules.
* **Abort the experiment:** Stopping the experiment from Chaos Studio cancels the in-flight SSM command and runs cleanup.
* **Manual cleanup:** If the proxy is left running, kill it via SSM Session Manager and remove any installed iptables rules.

***

### Limitations <a href="#limitations" id="limitations"></a>

* **HTTP only (no HTTPS):** The proxy operates on plaintext HTTP. Move TLS termination upstream before using this fault.
* **Linux-only payload:** This fault runs on Linux instances.
* **SSM Agent required:** Instances without the SSM Agent online cannot be targeted.
* **All-or-nothing header rewrite:** Every request or response on the port is modified. There is no per-URL or per-status-code filter.
* **HEADERS\_MAP must be valid JSON:** Quote keys and values correctly. Malformed JSON fails the SSM Run Command.

***

### Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

<details>

<summary>EC2 HTTP modify header experiment fails with InvalidInstanceId in Harness Chaos Engineering</summary>

The SSM Agent is not online for the target instance. Confirm with aws ssm describe-instance-information --filters 'Key=InstanceIds,Values='. If missing, install the SSM Agent and attach an instance profile that includes AmazonSSMManagedInstanceCore.

</details>

<details>

<summary>EC2 HTTP modify header runs but headers are unchanged in client responses</summary>

The most common causes are: HEADERS\_MODE is set to the wrong side (use 'request' to mutate inbound or 'response' to mutate outbound); HEADERS\_MAP is malformed JSON; TLS terminates at the port (HTTPS is not supported); or a downstream proxy strips or re-adds the header after the chaos proxy modifies it. Verify with 'curl -I <http://localhost:/>' via SSM during the fault.

</details>

<details>

<summary>EC2 HTTP modify header rejected my HEADERS_MAP</summary>

HEADERS\_MAP must be a single-line JSON object with quoted keys and string values, for example: a key "X-Auth" with value "fake". Escape quotes if you are pasting into a YAML field. Empty values remove the header (set the value of "Authorization" to an empty string to strip it).

</details>

***

### Related faults <a href="#related-faults" id="related-faults"></a>

* [EC2 HTTP latency](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/aws/ec2-http-latency.md): Delay HTTP responses instead of changing headers.
* [EC2 HTTP modify body](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/aws/ec2-http-modify-body.md): Rewrite the response body instead of headers.
* [EC2 HTTP status code](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/aws/ec2-http-status-code.md): Rewrite the response status code.
* [EC2 HTTP reset peer](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/aws/ec2-http-reset-peer.md): Reset TCP connections mid-flight.
* [Common AWS fault tunables](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/aws/aws-fault-tunables.md): Shared environment variables for AWS faults.