> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/resilience-testing/chaos-engineering/faults/chaos-fault-categories/vmware/linux/vmware-network-latency.md).

# VMware network latency

VMware network latency is a VMware chaos fault that adds `NETWORK_LATENCY` milliseconds (with optional `JITTER`) of latency to egress traffic on the network interface `NETWORK_INTERFACE` of the Linux VM `VM_NAME` for `TOTAL_CHAOS_DURATION` seconds, then removes the latency rule. You can scope the impact to specific destinations via `DESTINATION_IPS` or `DESTINATION_HOSTS` and to specific ports via `SOURCE_PORTS`/`DESTINATION_PORTS`. The fault uses VMware Tools (Guest Operations API) to apply the rule inside the guest as `VM_USER_NAME`.

Use this fault to test how a workload on a VMware-hosted VM behaves when a downstream dependency becomes slow: whether retries and timeouts work, whether circuit breakers open correctly, and whether monitoring detects the regression within the alerting SLA.

{% hint style="info" %}
**RUN YOUR FIRST EXPERIMENT**

If you have not configured the chaos infrastructure yet, go to [Quickstart](/resilience-testing/chaos-engineering/new-to-chaos-engineering/quickstart.md) to install the chaos infrastructure and run an experiment end to end.
{% endhint %}

***

### Use cases <a href="#use-cases" id="use-cases"></a>

Run this fault when you want to answer concrete questions like:

* **Slow downstream:** When latency to a specific dependency spikes, does the caller honor its timeout and circuit-breaker policy?
* **End-to-end SLO:** Does the end-user SLO degrade gracefully or breach when the network slows?
* **Retry storms:** Does retry logic amplify load when the dependency is slow?

***

### Prerequisites <a href="#prerequisites" id="prerequisites"></a>

* **Kubernetes version:** 1.21 or later for the chaos infrastructure cluster.
* **VMware Tools running on the guest:** Verify with `vmware-toolbox-cmd -v`.
* **Linux `tc` (iproute2) installed inside the guest:** Go to [VMware Linux binary installation](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/vmware/linux/binary-installation.md).
* **Sudo or capability for `tc`:** `VM_USER_NAME` can run `tc qdisc` (typically requires `sudo` or `CAP_NET_ADMIN`).
* **vCenter chaos role:** `GOVC_USERNAME` is mapped to the chaos role per [VMware permissions](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/vmware/permissions.md).

***

### Supported environments <a href="#supported-environments" id="supported-environments"></a>

| Platform                                                                      | Support status                                                                                                                                               |
| ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Linux VMs hosted on vSphere / vCenter (any distro with VMware Tools and `tc`) | Supported                                                                                                                                                    |
| Windows VMs                                                                   | Not supported (use [VMware Windows network latency](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/windows/windows-network-latency.md)) |

***

### Permissions required <a href="#permissions-required" id="permissions-required"></a>

**On vCenter.** Map `GOVC_USERNAME` to the chaos role described in [VMware permissions](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/vmware/permissions.md). The role needs Guest Operations (Program execution, Modifications, Queries).

**On the guest OS.** `VM_USER_NAME` must be able to run `tc qdisc` on `NETWORK_INTERFACE`.

***

### Authentication <a href="#authentication" id="authentication"></a>

| Layer    | Tunables                                                      |
| -------- | ------------------------------------------------------------- |
| vCenter  | `GOVC_URL`, `GOVC_USERNAME`, `GOVC_PASSWORD`, `GOVC_INSECURE` |
| Guest OS | `VM_USER_NAME`, `VM_PASSWORD`                                 |

Store each credential as a text secret in [Harness Secret Manager](/harness-ai/use-harness-platform/secrets/add-use-text-secrets.md) and reference the secret identifier when configuring the experiment.

***

### Fault tunables <a href="#fault-tunables" id="fault-tunables"></a>

Configure the following fault parameters when you add VMware network latency to an experiment in Chaos Studio. Defaults are shown for reference.

**Required parameters**

| Tunable        | Description                                     | Default    |
| -------------- | ----------------------------------------------- | ---------- |
| `VM_NAME`      | Name of the target VM as it appears in vCenter. | (required) |
| `VM_USER_NAME` | OS user account on the target VM.               | (required) |
| `VM_PASSWORD`  | Password for `VM_USER_NAME`.                    | (required) |

**Network chaos parameters**

| Tunable             | Description                                                                                        | Default |
| ------------------- | -------------------------------------------------------------------------------------------------- | ------- |
| `NETWORK_INTERFACE` | Name of the interface to apply the latency rule to (for example `eth0`).                           | `eth0`  |
| `NETWORK_LATENCY`   | Network latency to inject in milliseconds.                                                         | `2000`  |
| `JITTER`            | Jitter to add to the latency, in milliseconds.                                                     | `0`     |
| `DESTINATION_IPS`   | Comma-separated list of destination IPv4/IPv6/CIDR ranges to affect. Empty means all destinations. | `""`    |
| `DESTINATION_HOSTS` | Comma-separated list of destination DNS names to affect. Resolved at fault start.                  | `""`    |
| `SOURCE_PORTS`      | Comma-separated list of source ports to filter on.                                                 | `""`    |
| `DESTINATION_PORTS` | Comma-separated list of destination ports to filter on.                                            | `""`    |

**Chaos parameters**

| Tunable                | Description                                                                                                                                                                                                      | Default    |
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- |
| `TOTAL_CHAOS_DURATION` | Total duration of the fault in seconds.                                                                                                                                                                          | `30`       |
| `CHAOS_INTERVAL`       | Delay in seconds between iterations.                                                                                                                                                                             | `10`       |
| `SEQUENCE`             | Order in which multiple targets are stressed: `parallel` or `serial`.                                                                                                                                            | `parallel` |
| `RAMP_TIME`            | Wait period in seconds before and after the fault. Go to [ramp time](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/common-tunables-for-all-faults.md#ramp-time) to read how it is applied. | `0`        |

**vCenter authentication**

| Tunable         | Description                                           | Default |
| --------------- | ----------------------------------------------------- | ------- |
| `GOVC_URL`      | vCenter server URL.                                   | `""`    |
| `GOVC_USERNAME` | vCenter user mapped to the chaos role.                | `""`    |
| `GOVC_PASSWORD` | Password for `GOVC_USERNAME`.                         | `""`    |
| `GOVC_INSECURE` | Skip SSL certificate verification when set to `true`. | `true`  |

Tunables that apply to every fault are documented in [common tunables for all faults](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/common-tunables-for-all-faults.md).

***

### Fault execution in brief <a href="#fault-execution-in-brief" id="fault-execution-in-brief"></a>

Authenticates to vCenter, opens a Guest Operations session on `VM_NAME` as `VM_USER_NAME`, installs a queueing discipline on `NETWORK_INTERFACE` that delays egress packets matching `DESTINATION_IPS` / `DESTINATION_HOSTS` / `SOURCE_PORTS` / `DESTINATION_PORTS` by `NETWORK_LATENCY` ms (+/- `JITTER` ms) for `TOTAL_CHAOS_DURATION` seconds, then removes the rule.

***

### Expected behavior during fault execution <a href="#expected-behavior-during-fault-execution" id="expected-behavior-during-fault-execution"></a>

* Egress latency from `VM_NAME` to the matched destinations rises by `NETWORK_LATENCY` ms.
* Upstream callers see higher round-trip latency; SLO-sensitive paths may breach.
* After the duration ends, the `tc` rule is removed and latency returns to baseline.

{% hint style="info" %}
**WHEN THE FAULT ENDS**

The chaos pod removes the `tc qdisc` rule from `NETWORK_INTERFACE`. Latency returns to baseline within seconds.
{% endhint %}

#### Signals to watch <a href="#signals-to-watch" id="signals-to-watch"></a>

* **End-to-end latency:** Use an [HTTP probe](/resilience-testing/chaos-engineering/use-chaos-engineering/probes/http-probe.md) from outside the VM.
* **Caller behavior:** Use a Prometheus probe on caller-side timeout and circuit-breaker metrics.

***

### Verify the fault execution effect <a href="#verify-the-fault-execution-effect" id="verify-the-fault-execution-effect"></a>

1. **SSH into the VM and inspect the qdisc.**

   ```bash
   sudo tc qdisc show dev eth0
   ```

   You should see the `netem` rule with the configured delay during the chaos window, and it should be removed afterwards.
2. **Ping the target from outside the VM.**

   Round-trip should rise by `NETWORK_LATENCY` during the chaos window.

***

### Recovery and cleanup <a href="#recovery-and-cleanup" id="recovery-and-cleanup"></a>

* **End of duration:** The chaos pod removes the `tc qdisc` rule via Guest Operations.
* **Abort the experiment:** Stopping the experiment also removes the rule.
* **Manual recovery:** SSH into the VM and run `sudo tc qdisc del dev <NETWORK_INTERFACE> root` to clear lingering rules.

***

### Limitations <a href="#limitations" id="limitations"></a>

* **Egress only:** The rule affects egress traffic from the VM. Ingress is not slowed (peer egress is unaffected).
* **Single interface per run:** Each fault run scopes one `NETWORK_INTERFACE`. Repeat the fault for additional interfaces.
* **`tc` required:** Without `tc` (iproute2) and the netem kernel module, the fault cannot run.
* **DNS at start time:** `DESTINATION_HOSTS` is resolved once at fault start. DNS changes during the chaos window are not picked up.

***

### Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

<details>

<summary>VMware network latency has no effect in Harness Chaos Engineering</summary>

Verify NETWORK\_INTERFACE matches the active interface (run ip a inside the VM). Verify VM\_USER\_NAME has sudo for tc. Verify DESTINATION\_IPS / DESTINATION\_HOSTS actually match the traffic you are measuring.

</details>

<details>

<summary>tc qdisc rule left behind after experiment in HCE</summary>

Run sudo tc qdisc del dev root inside the guest to remove lingering rules. This can happen if VMware Tools or the chaos pod was killed mid-run.

</details>

***

### Related faults <a href="#related-faults" id="related-faults"></a>

* [VMware network loss](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/vmware/linux/vmware-network-loss.md): Drop packets instead of delaying them.
* [VMware network rate limit](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/vmware/linux/vmware-network-rate-limit.md): Cap bandwidth instead of injecting latency.