> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/resilience-testing/chaos-engineering/faults/chaos-fault-categories/linux/linux-network-latency.md).
# Linux network latency
Linux network latency is a chaos fault that adds `NETWORK_LATENCY` milliseconds of delay (plus optional `JITTER`) to packets leaving the target Linux machine on `NETWORK_INTERFACES` for `DURATION`, then restores normal connectivity. Delay is restricted to traffic destined for `DESTINATION_HOSTS`/`DESTINATION_IPS` and the configured port filters; SSH ports stay reachable when `WHITELIST_SSH` is `true`. The fault runs through the Linux Chaos Infrastructure (LCI) systemd service installed on the target VM.
Use this fault to test how a workload behaves when the network is slow: whether application timeouts fire cleanly, whether retries amplify the delay, whether tail latency exceeds the SLA, and whether monitoring detects the degradation within the alerting SLA.
{% hint style="info" %}
**RUN YOUR FIRST EXPERIMENT**
If you have not installed the Linux Chaos Infrastructure yet, go to [Linux Chaos Infrastructure](/resilience-testing/chaos-engineering/use-chaos-engineering/infrastructure/types/legacy-infra/linux.md) to install the agent and connect the VM to the control plane.
{% endhint %}
***
### Use cases
Run this fault when you want to answer concrete questions like:
* **Latency tolerance:** When traffic to `DESTINATION_HOSTS` is delayed by `NETWORK_LATENCY` milliseconds, does the application stay inside its p95/p99 SLA?
* **Timeout behavior:** Do client timeouts fire cleanly, or do connections pile up and exhaust thread pools?
* **Retry storms:** Do retries amplify the slow traffic, or do exponential backoff and jitter contain it?
* **Monitoring fidelity:** Do alerts on end-to-end latency, queue depth, and timeout counters fire within the alerting SLA?
***
### Prerequisites
* **Linux Chaos Infrastructure installed:** The `linux-chaos-infrastructure` systemd service is `active` on the target VM and the infrastructure is in `CONNECTED` state. Go to [Linux Chaos Infrastructure](/resilience-testing/chaos-engineering/use-chaos-engineering/infrastructure/types/legacy-infra/linux.md) to install it.
* **Target interface exists:** Each entry in `NETWORK_INTERFACES` exists on the target VM. Confirm with `ip -br link`.
* **`tc` available:** The fault uses Linux Traffic Control (`tc`) with the `netem` qdisc. Provided by iproute2.
***
### Supported environments
The fault has been tested on the following Linux distributions. Go to [Linux fault requirements](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/linux/permissions.md) to see the full compatibility matrix.
| Platform | Support status |
| ----------------------------------------------- | -------------- |
| Ubuntu 16+, Debian 10+ | Supported |
| CentOS 7+, RHEL 7+, Fedora 30+ | Supported |
| openSUSE LEAP 15.4+ / SUSE Linux Enterprise 15+ | Supported |
***
### Permissions required
This fault is classified as an **Advanced** Linux fault. It requires the Linux Chaos Infrastructure systemd service to run with the root user and root user group on the target VM so it can manage the `tc` qdisc. No cloud credentials are needed.
***
### Fault tunables
Configure the following fault parameters when you add Linux network latency to an experiment in Chaos Studio. Defaults are shown for reference.
**Chaos parameters**
| Tunable | Description | Default |
| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
| `DURATION` | Total duration of the fault. Accepts `[hours]h[minutes]m[seconds]s` format. | `30s` |
| `NETWORK_LATENCY` | Latency to inject in milliseconds. | `2000` |
| `JITTER` | Additional jitter (variance) to apply with `NETWORK_LATENCY`, in milliseconds. | `0` |
| `NETWORK_INTERFACES` | Comma-separated network interfaces to apply chaos on. | `eth0` |
| `RAMP_TIME` | Wait period in seconds before and after the fault. Go to [ramp time](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/common-tunables-for-all-faults.md#ramp-time) to read how it is applied. | `0` |
**Target filters (provide at least one host or IP to limit blast radius)**
| Tunable | Description | Default |
| ------------------- | ----------------------------------------------------------------------------------------------------- | ------- |
| `DESTINATION_HOSTS` | Comma-separated destination host names to target. | `""` |
| `DESTINATION_IPS` | Comma-separated destination IPs to target. Per-IP ports can be specified using the `ip\|port` format. | `""` |
| `SOURCE_PORTS` | Comma-separated source ports to target. Prefix with `!` to whitelist instead of target. | `""` |
| `DESTINATION_PORTS` | Comma-separated destination ports to target. Prefix with `!` to whitelist instead of target. | `""` |
| `WHITELIST_SSH` | Keep SSH ports (`22,2222`) excluded from chaos to preserve management access. | `true` |
When neither `DESTINATION_HOSTS` nor `DESTINATION_IPS` is set, the fault applies to all destinations on the interface.
Tunables that apply to every fault are documented in [common tunables for all faults](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/common-tunables-for-all-faults.md).
***
### Fault execution in brief
Adds a `netem` qdisc on `NETWORK_INTERFACES` that delays egress packets matching the configured destination and port filters by `NETWORK_LATENCY` (with optional `JITTER`) for `DURATION`, then removes the qdisc.
***
### Expected behavior during fault execution
* Round-trip time to matched destinations rises by approximately `NETWORK_LATENCY` (with `+/- JITTER` variance).
* Application calls take longer; tail latency (`p99`) is most affected.
* Time-sensitive protocols (synchronous RPC, database keep-alives) may exceed their timeouts.
* After the duration ends, the qdisc is removed and packet delay returns to baseline.
{% hint style="info" %}
**WHEN THE FAULT ENDS**
The chaos pod removes the `netem` qdisc. Packets flow normally on the next send; queued packets drain within the configured delay.
{% endhint %}
#### Signals to watch
Attach [resilience probes](/resilience-testing/chaos-engineering/use-chaos-engineering/probes.md) to assert each layer:
* **Network RTT:** Use a [command probe](/resilience-testing/chaos-testing/probes/command-probe.md) running `ping -c 5 ` and assert the RTT rose by `NETWORK_LATENCY`.
* **Application latency:** Use a [Prometheus probe](/resilience-testing/chaos-engineering/use-chaos-engineering/probes/apm-probes.md) on application p95/p99 metrics.
* **End-to-end availability:** Use an [HTTP probe](/resilience-testing/chaos-engineering/use-chaos-engineering/probes/http-probe.md) on a user-visible endpoint.
***
### Verify the fault execution effect
While the experiment is running, confirm latency was added and then removed:
1. **Inspect the active qdisc.**
```bash
tc -s qdisc show dev
```
A `netem` qdisc with the configured `delay` should be present during the chaos window and removed afterwards.
2. **Run ping against a matched destination.**
```bash
ping -c 10
```
RTT should rise by approximately `NETWORK_LATENCY` (with `+/- JITTER` variance) during the chaos window.
3. **Inspect Linux Chaos Infrastructure logs.**
```bash
sudo journalctl -u linux-chaos-infrastructure -n 100 --no-pager
```
***
### Recovery and cleanup
* **End of duration:** The chaos pod removes the `netem` qdisc when `DURATION` elapses.
* **Abort the experiment:** Stopping the experiment from Chaos Studio also removes the qdisc.
* **Manual recovery:** If the qdisc survives an abort, remove it with `sudo tc qdisc del dev root`.
* **Workload recovery:** Applications resume normal latency on the next send.
***
### Limitations
* **Egress only:** The fault delays packets leaving the VM on the configured interfaces; inbound packets are not delayed.
* **Single VM scope:** Each fault run targets one VM.
* **Delay variance:** With `JITTER` set, delay is randomized within `+/- JITTER` of `NETWORK_LATENCY`; the actual distribution approximates a uniform random over many packets.
* **SSH whitelisting:** When `WHITELIST_SSH` is `false`, your SSH session may stall if SSH ports match the filter.
* **Existing qdisc:** If a custom qdisc is already attached, the fault may fail; review with `tc qdisc show` before running.
***
### Troubleshooting
Linux network latency fault shows no measurable RTT increase in Harness Chaos Engineering
Confirm the destination filter matches your test traffic. Run tc -s qdisc show dev to verify the netem qdisc is installed. NETWORK\_LATENCY values below the baseline RTT may not be observable.
SSH session became slow during the experiment
WHITELIST\_SSH defaults to true to exclude ports 22 and 2222. If your SSH session became slow, your SSH may be on a different port; add it with the ! prefix in SOURCE\_PORTS to whitelist it.
Latency persists after the experiment ends
If the netem qdisc was not removed, remove it manually with sudo tc qdisc del dev root. Inspect with tc qdisc show dev first.
***
### Related faults
* [Linux network loss](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/linux/linux-network-loss.md): Drop packets instead of delaying them.
* [Linux network corruption](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/linux/linux-network-corruption.md): Corrupt packets instead of delaying them.
* [Linux network rate limit](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/linux/linux-network-rate-limit.md): Throttle bandwidth instead of adding delay.