> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/resilience-testing/chaos-engineering/faults/chaos-fault-categories/linux/linux-disk-fill.md).

# Linux disk fill

Linux disk fill is a chaos fault that writes a file under `FILL_PATH` until it occupies `FILL_STORAGE` of space (or `FILL_STORAGE` percent of the volume) on the target Linux machine for `DURATION`, then removes the file and frees the space. The fault runs through the Linux Chaos Infrastructure (LCI) systemd service installed on the target VM.

Use this fault to test how a workload behaves when its writable storage fills up: whether the application surfaces a clean `ENOSPC` error or wedges, whether log rotation kicks in, whether monitoring detects low disk space within the alerting SLA, and whether the system recovers automatically once space is reclaimed.

{% hint style="info" %}
**RUN YOUR FIRST EXPERIMENT**

If you have not installed the Linux Chaos Infrastructure yet, go to [Linux Chaos Infrastructure](/resilience-testing/chaos-engineering/use-chaos-engineering/infrastructure/types/legacy-infra/linux.md) to install the agent and connect the VM to the control plane.
{% endhint %}

***

### Use cases <a href="#use-cases" id="use-cases"></a>

Run this fault when you want to answer concrete questions like:

* **Write paths under ENOSPC:** When `FILL_PATH` runs out of space, do application writes fail cleanly or wedge?
* **Log rotation:** Does logrotate (or equivalent) kick in and free space before the volume fills?
* **Monitoring fidelity:** Do disk-space alerts (`node_filesystem_avail_bytes`, `df -h`) fire within the alerting SLA?
* **Recovery:** After the fill file is removed, do the application and monitoring return to baseline without a restart?

***

### Prerequisites <a href="#prerequisites" id="prerequisites"></a>

* **Linux Chaos Infrastructure installed:** The `linux-chaos-infrastructure` systemd service is `active` on the target VM and the infrastructure is in `CONNECTED` state. Go to [Linux Chaos Infrastructure](/resilience-testing/chaos-engineering/use-chaos-engineering/infrastructure/types/legacy-infra/linux.md) to install it.
* **Writable target path:** `FILL_PATH` exists, is writable by the LCI service, and resides on a filesystem with enough headroom for `FILL_STORAGE`.
* **`dd` available:** The fault uses `dd` from coreutils to write the fill file.

***

### Supported environments <a href="#supported-environments" id="supported-environments"></a>

The fault has been tested on the following Linux distributions. Go to [Linux fault requirements](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/linux/permissions.md) to see the full compatibility matrix.

| Platform                                        | Support status |
| ----------------------------------------------- | -------------- |
| Ubuntu 16+, Debian 10+                          | Supported      |
| CentOS 7+, RHEL 7+, Fedora 30+                  | Supported      |
| openSUSE LEAP 15.4+ / SUSE Linux Enterprise 15+ | Supported      |

***

### Permissions required <a href="#permissions-required" id="permissions-required"></a>

This fault is classified as a **Basic** Linux fault. It runs with the privileges of the Linux Chaos Infrastructure systemd service (root user and root user group) on the target VM. No cloud credentials are needed.

***

### Fault tunables <a href="#fault-tunables" id="fault-tunables"></a>

Configure the following fault parameters when you add Linux disk fill to an experiment in Chaos Studio. Defaults are shown for reference.

**Required parameters**

| Tunable     | Description                                                                                  | Default    |
| ----------- | -------------------------------------------------------------------------------------------- | ---------- |
| `FILL_PATH` | Absolute path where the fill file is created. Must exist and be writable by the LCI service. | (required) |

**Chaos parameters**

| Tunable           | Description                                                                                                                                                                                                      | Default |
| ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
| `DURATION`        | Total duration of the fault. Accepts `[hours]h[minutes]m[seconds]s` format (for example, `30s`, `1m25s`, `1h3m2s`).                                                                                              | `30s`   |
| `FILL_STORAGE`    | Amount of storage to fill. Accepts absolute values (for example, `256m`, `1g`) or a percentage of the volume (for example, `50%`).                                                                               | `50%`   |
| `DATA_BLOCK_SIZE` | Block size used by `dd` when writing the fill file. Larger blocks fill faster; smaller blocks complete on volumes with small free space.                                                                         | `256k`  |
| `RAMP_TIME`       | Wait period in seconds before and after the fault. Go to [ramp time](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/common-tunables-for-all-faults.md#ramp-time) to read how it is applied. | `0`     |

Tunables that apply to every fault are documented in [common tunables for all faults](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/common-tunables-for-all-faults.md).

***

### Fault execution in brief <a href="#fault-execution-in-brief" id="fault-execution-in-brief"></a>

Writes a fill file under `FILL_PATH` using `dd` until it occupies `FILL_STORAGE`, holds for `DURATION`, then deletes the file and frees the space.

***

### Expected behavior during fault execution <a href="#expected-behavior-during-fault-execution" id="expected-behavior-during-fault-execution"></a>

* Free space on the volume hosting `FILL_PATH` drops by approximately `FILL_STORAGE` for the duration of the fault.
* New writes by other processes on the same volume may fail with `ENOSPC` ("No space left on device") if the volume reaches 100%.
* Disk-space monitoring (`node_filesystem_avail_bytes`, `df -h`) reports the reduced free space.
* After the duration ends, the fill file is removed and free space returns to baseline.

{% hint style="info" %}
**WHEN THE FAULT ENDS**

The chaos pod deletes the fill file. Free space returns immediately; the kernel may keep page-cache entries around until reclaim.
{% endhint %}

#### Signals to watch <a href="#signals-to-watch" id="signals-to-watch"></a>

Attach [resilience probes](/resilience-testing/chaos-engineering/use-chaos-engineering/probes.md) to assert each layer:

* **Disk space:** Use a [Prometheus probe](/resilience-testing/chaos-engineering/use-chaos-engineering/probes/apm-probes.md) on `node_filesystem_avail_bytes{mountpoint="<mount>"}` and assert it dropped during the chaos window.
* **Write errors:** Use a [command probe](/resilience-testing/chaos-testing/probes/command-probe.md) to scan application logs for `ENOSPC` or "No space left on device" entries.
* **End-to-end availability:** Use an [HTTP probe](/resilience-testing/chaos-engineering/use-chaos-engineering/probes/http-probe.md) on a user-visible endpoint that exercises a write path.

***

### Verify the fault execution effect <a href="#verify-the-fault-execution-effect" id="verify-the-fault-execution-effect"></a>

While the experiment is running, confirm the volume filled and was reclaimed:

1. **Observe free space on the target volume.**

   ```bash
   df -h <mount-of-FILL_PATH>
   ls -lh <FILL_PATH>
   ```

   Free space should drop by approximately `FILL_STORAGE` during the chaos window and return to baseline afterwards.
2. **Inspect Linux Chaos Infrastructure logs.**

   ```bash
   sudo journalctl -u linux-chaos-infrastructure -n 100 --no-pager
   ```

   Look for the fault start, the fill file path, the dd progress, and the fault end markers.

***

### Recovery and cleanup <a href="#recovery-and-cleanup" id="recovery-and-cleanup"></a>

* **End of duration:** The chaos pod deletes the fill file when `DURATION` elapses; free space returns to baseline.
* **Abort the experiment:** Stopping the experiment from Chaos Studio also triggers the cleanup.
* **Manual recovery:** If the fill file survives an abort, remove it with `sudo rm <FILL_PATH>/<fill-file>` (the LCI logs record the exact filename).
* **Workload recovery:** Applications that experienced `ENOSPC` may need to be restarted if they cached the failure state.

***

### Limitations <a href="#limitations" id="limitations"></a>

* **Filesystem-level only:** The fault fills space at the filesystem level. It does not exhaust inodes; for inode exhaustion use a custom fault.
* **One path per run:** A single experiment fills exactly one path on the target VM. Use multiple experiments to fill multiple volumes.
* **Disk-cache effects:** Filling a volume close to 100% can degrade other I/O on the same disk due to fragmentation and cache pressure.
* **No mid-flight resize:** `FILL_STORAGE` cannot be adjusted during the chaos window; abort and re-run the experiment to change it.

***

### Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

<details>

<summary>Linux disk fill fault fails with no such file or directory in Harness Chaos Engineering</summary>

FILL\_PATH must exist on the target VM and be writable by the linux-chaos-infrastructure service (which runs as root). Verify with sudo test -w FILL\_PATH and create the directory if missing with sudo mkdir -p FILL\_PATH.

</details>

<details>

<summary>Volume did not fill to the configured FILL_STORAGE</summary>

If the volume has less free space than FILL\_STORAGE, the fault fills to whatever is available. Reduce FILL\_STORAGE or run df -h to confirm headroom before the experiment.

</details>

<details>

<summary>Disk space stays low after the experiment ends</summary>

If the fill file was not removed (for example, the experiment aborted ungracefully), delete it manually with sudo rm FILL\_PATH/. The exact filename is recorded in sudo journalctl -u linux-chaos-infrastructure logs.

</details>

***

### Related faults <a href="#related-faults" id="related-faults"></a>

* [Linux fs fill](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/linux/linux-fs-fill.md): Fill a filesystem path with a smaller tunable surface.
* [Linux disk I/O stress](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/linux/linux-disk-io-stress.md): Stress disk I/O bandwidth instead of capacity.
* [Linux memory stress](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/linux/linux-memory-stress.md): Consume memory instead of disk space.
