> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/resilience-testing/chaos-engineering/faults/chaos-fault-categories/cloud-foundry/cf-app-stop.md).

# CF app stop

CF app stop is a Cloud Foundry chaos fault that stops the target app (`app` in `organization`/`space`) for `duration` seconds, then starts it back up. It exercises how the Cloud Foundry platform, upstream routers, and downstream consumers behave when an app is gracefully taken offline and brought back.

Use this fault to validate disaster-recovery behaviors such as the platform restarting the app, the router rejecting traffic during the outage, blue/green peers absorbing requests, and alerting firing inside SLA.

{% hint style="info" %}
**RUN YOUR FIRST EXPERIMENT**

If you have not configured the chaos infrastructure yet, go to [Quickstart](/resilience-testing/chaos-engineering/new-to-chaos-engineering/quickstart.md) to install the Linux chaos infrastructure and run an experiment end to end.
{% endhint %}

***

### Use cases <a href="#use-cases" id="use-cases"></a>

* **Outage absorption:** Validate that consumers (gateways, other apps) handle the stopped app cleanly: retries, fallbacks, circuit breakers.
* **Platform recovery:** Confirm Cloud Foundry brings the app back to a healthy state when the fault re-starts it.
* **Alert fidelity:** Verify health-check based alerts (`/health`, route probes) fire inside the alerting SLA.
* **Runbook validation:** Practice the on-call response to a CF app outage in a controlled window.

***

### Before you begin <a href="#before-you-begin" id="before-you-begin"></a>

* **Chaos infrastructure:** A Linux chaos infrastructure (LCI) installed on a machine that can reach the Cloud Foundry API and UAA endpoints. Go to [Cloud Foundry chaos deployment](https://github.com/iKettles/harness-gitbook/tree/main/docs/chaos-engineering/faults/chaos-faults/cloud-foundry/cf-chaos-components-and-their-deployment-architecture/README.md) to read the supported deployment models.
* **CF credentials:** `CF_API_ENDPOINT`, `CF_USERNAME`, `CF_PASSWORD`, and `UAA_SERVER_ENDPOINT` available to the chaos infrastructure (see [Authentication](#authentication)).
* **Target identifiers:** You know the `organization`, `space`, and `app` name as registered in CF.
* **Permissions:** The CF user holds the roles described in [Permissions required](#permissions-required).

***

### Supported environments <a href="#supported-environments" id="supported-environments"></a>

| Platform                                                     | Support status                                                                                                                                |
| ------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- |
| Cloud Foundry on TAS, PCF, or open-source CF deployments     | Supported                                                                                                                                     |
| Kubernetes-only deployments (no Cloud Foundry control plane) | Not supported (use the [Kubernetes pod faults](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/kubernetes/kubernetes.md)) |

***

### Permissions required <a href="#permissions-required" id="permissions-required"></a>

**On Cloud Foundry.** The CF user authenticating as `CF_USERNAME` needs both read and write scopes on the target app:

| Action                                        | Required role (any)                                             | Required OAuth scope (any)                                                           |
| --------------------------------------------- | --------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
| List apps the user can access                 | `SpaceDeveloper`, `SpaceAuditor`, `OrgManager`, or `OrgAuditor` | `cloud_controller.read`, `cloud_controller.global_auditor`, `cloud_controller.admin` |
| Stop the target app (set state to `STOPPED`)  | `SpaceDeveloper` in the app's space                             | `cloud_controller.write`, `cloud_controller.admin`                                   |
| Start the target app (set state to `STARTED`) | `SpaceDeveloper` in the app's space                             | `cloud_controller.write`, `cloud_controller.admin`                                   |

***

### Authentication <a href="#authentication" id="authentication"></a>

| Layer                                                | Where to provide                                                                        | Tunables                                                                                               |
| ---------------------------------------------------- | --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ |
| Cloud Foundry API                                    | `/etc/linux-chaos-infrastructure/cf.env` on the LCI host (or environment / config file) | `CF_API_ENDPOINT`, `CF_USERNAME`, `CF_PASSWORD`, `UAA_SERVER_ENDPOINT`                                 |
| vSphere (only when `faultInjectorLocation: vSphere`) | `/etc/linux-chaos-infrastructure/vsphere.env`                                           | `GOVC_URL`, `GOVC_USERNAME`, `GOVC_PASSWORD`, `GOVC_INSECURE`, `VM_NAME`, `VM_USERNAME`, `VM_PASSWORD` |

Full key/format reference is in [CF secrets](#cf-secrets) and [vSphere secrets](#vsphere-secrets) below.

***

### Fault tunables <a href="#fault-tunables" id="fault-tunables"></a>

**Required parameters**

| Tunable        | Description                                          | Default    |
| -------------- | ---------------------------------------------------- | ---------- |
| `organization` | Cloud Foundry organization that owns the target app. | (required) |
| `space`        | CF space within the organization.                    | (required) |
| `app`          | Name of the app to stop.                             | (required) |

**Chaos parameters**

| Tunable                 | Description                                                                                                                                                                            | Default |
| ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
| `duration`              | Total chaos duration. The app stays stopped for this period before it is restarted.                                                                                                    | `30s`   |
| `faultInjectorLocation` | Where the fault-injector runs relative to the LCI. Supports `local` and `vSphere`. Go to [Fault injector location](#fault-injector-location).                                          | `local` |
| `faultInjectorPort`     | Local port used by the fault-injector. If unavailable, a random port in `50320-51320` is picked. Go to [Fault injector port](#fault-injector-port).                                    | `50320` |
| `skipSSLValidation`     | Skip SSL validation when calling CF APIs. Go to [Skip SSL validation](#skip-ssl-validation).                                                                                           | `false` |
| `rampTime`              | Wait period in seconds before and after the fault. Go to [ramp time](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/common-tunables-for-all-faults.md#ramp-time). | `0`     |

Tunables that apply to every fault are documented in [common tunables for all faults](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/common-tunables-for-all-faults.md).

***

### Fault execution in brief <a href="#fault-execution-in-brief" id="fault-execution-in-brief"></a>

Authenticates to the Cloud Foundry API, locates the app in `organization`/`space`, sets its state to `STOPPED`, waits `duration` seconds, and then sets the state back to `STARTED`. Recovery completes when CF reports the app `STARTED` and healthy.

***

### Expected behavior during fault execution <a href="#expected-behavior-during-fault-execution" id="expected-behavior-during-fault-execution"></a>

* The app's CF status transitions to `STOPPED`; its instances are drained and removed.
* Requests routed to the app fail (typically 502/503 from the CF router) until the fault re-starts the app.
* After the fault ends, CF starts the app and health checks return successful before traffic is restored.

#### Signals to watch <a href="#signals-to-watch" id="signals-to-watch"></a>

* **App availability:** Use an [HTTP probe](/resilience-testing/chaos-engineering/use-chaos-engineering/probes/http-probe.md) against a public route on the app.
* **Downstream errors:** Probe a consumer that depends on the app; assert it returns its expected fallback response, not a 5xx.

***

### Recovery and cleanup <a href="#recovery-and-cleanup" id="recovery-and-cleanup"></a>

* The fault re-starts the app automatically when `duration` elapses.
* If the experiment is aborted, the fault still attempts to start the app before exiting.

***

### Limitations <a href="#limitations" id="limitations"></a>

* Stops the entire app, not individual instances. To affect a subset, use [CF app container kill](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/cloud-foundry/cf-app-container-kill.md) with `instanceAffectedPercentage`.
* The app must already be in `STARTED` state at experiment start.

***

### Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

<details>

<summary>CF app stop fails with 'app not found' in Harness Chaos Engineering</summary>

Verify the organization, space, and app values match what is registered in Cloud Foundry (case-sensitive). Run cf app from the LCI host to confirm visibility under the same CF user.

</details>

<details>

<summary>CF API call fails with TLS certificate error</summary>

If your CF API uses a self-signed or internal CA certificate, set skipSSLValidation: true on the fault, or install the trusted CA on the chaos infrastructure host.

</details>

<details>

<summary>App is still STOPPED after the experiment ends</summary>

The fault attempts to restart the app on cleanup. If the start failed (for example, due to insufficient memory in the org/space quota), start the app manually with cf start and investigate the platform logs.

</details>

***

### Common configurations <a href="#common-configurations" id="common-configurations"></a>

#### Fault injector location <a href="#fault-injector-location" id="fault-injector-location"></a>

The `faultInjectorLocation` input determines where the fault-injector utility runs:

* `local`: Runs on the same host as the LCI. Use this when the LCI host can reach the CF API directly.
* `vSphere`: Runs on a remote vSphere-managed VM. Use this when the chaos infrastructure is deployed separately from the network path that reaches CF.

```yaml
apiVersion: litmuchaos.io/v1alpha1
kind: LinuxFault
metadata:
  name: cf-app-stop
  labels:
    name: app-stop
spec:
  cfAppStop/inputs:
    duration: 30s
    faultInjectorLocation: vSphere
    app: cf-app
    organization: dev-org
    space: dev-space
```

#### Skip SSL validation <a href="#skip-ssl-validation" id="skip-ssl-validation"></a>

Set `skipSSLValidation: true` when calling a CF API that uses a self-signed or internal CA certificate.

```yaml
apiVersion: litmuchaos.io/v1alpha1
kind: LinuxFault
metadata:
  name: cf-app-stop
  labels:
    name: app-stop
spec:
  cfAppStop/inputs:
    duration: 30s
    faultInjectorLocation: vSphere
    app: cf-app
    organization: dev-org
    space: dev-space
    skipSSLValidation: true
```

#### Fault injector port <a href="#fault-injector-port" id="fault-injector-port"></a>

Set `faultInjectorPort` to override the default `50320`. If the chosen port is already in use, the fault-injector picks a random port in `50320-51320`.

```yaml
apiVersion: litmuchaos.io/v1alpha1
kind: LinuxFault
metadata:
  name: cf-app-stop
  labels:
    name: app-stop
spec:
  cfAppStop/inputs:
    duration: 30s
    faultInjectorLocation: local
    app: cf-app
    organization: dev-org
    space: dev-space
    faultInjectorPort: 50331
```

***

The following Cloud Foundry secrets reside on the same machine where the chaos infrastructure is executed. These secrets are provided in the `/etc/linux-chaos-infrastructure/cf.env` file in the following format:

```env
CF_API_ENDPOINT=XXXXXXXXXXXXXXXXXXX
CF_USERNAME=XXXXXXXXXXXXXXXXXXXXXXX
CF_PASSWORD=XXXXXXXXXXXXXXXXXXXXXXX
UAA_SERVER_ENDPOINT=XXXXXXXXXXXXXXX
```

{% hint style="info" %}
If the secrets file is not provided, the secrets are attempted to be derived from environment variables and the config file by the fault-injector.
{% endhint %}

| ENV name              | Description                                      | Example                           |
| --------------------- | ------------------------------------------------ | --------------------------------- |
| CF\_API\_ENDPOINT     | API endpoint for the CF setup                    | `https://api.system.cf-setup.com` |
| CF\_USERNAME          | Username for the CF user                         | `username`                        |
| CF\_PASSWORD          | Password for the CF user                         | `password`                        |
| UAA\_SERVER\_ENDPOINT | API endpoint for the UAA server for the CF setup | `https://uaa.system.cf-setup.com` |

## Fault injector ENVs and config file <a href="#fault-injector-envs-and-config-file" id="fault-injector-envs-and-config-file"></a>

If `/etc/linux-chaos-infrastructure/cf.env` file is not provided, fault-injector attempts to derive the secrets from environment variables or a configuration file. Any secret that is re-declared will be overridden in the following order of decreasing precedence:

1. `/etc/linux-chaos-infrastructure/cf.env` file
2. Environment variables
3. Configuration file

The configuration file should be provided at `/etc/linux-chaos-infrastructure/cf-fault-injector.yaml`:

```yaml
cf-api-endpoint: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
username: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
password: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
uaa-server-endpoint: XXXXXXXXXXXXXXXXXXXXXXXXXX
```

A mapping between all the three formats for providing the secrets is as follows:

| cf.env                | ENV                   | cf-fault-injector.yaml |
| --------------------- | --------------------- | ---------------------- |
| CF\_API\_ENDPOINT     | CF\_API\_ENDPOINT     | cf-api-endpoint        |
| CF\_USERNAME          | USERNAME              | username               |
| CF\_PASSWORD          | PASSWORD              | password               |
| UAA\_SERVER\_ENDPOINT | UAA\_SERVER\_ENDPOINT | uaa-server-endpoint    |

These secrets are provided only if vSphere is used as the deployment platform for CF.

The following vSphere secrets reside on the same machine where the chaos infrastructure is executed. These secrets are provided in the `/etc/linux-chaos-infrastructure/vsphere.env` file in the following format:

```env
GOVC_URL=XXXXXXXXXXXXXXXXXXXXXX
GOVC_USERNAME=XXXXXXXXXXXXXXXXX
GOVC_PASSWORD=XXXXXXXXXXXXXXXXX
GOVC_INSECURE=XXXXXXXXXXXXXXXXX
VM_NAME=XXXXXXXXXXXXXXXXXXXXXXX
VM_USERNAME=XXXXXXXXXXXXXXXXXXX
VM_PASSWORD=XXXXXXXXXXXXXXXXXXX
```

| ENV Name       | Description                                                          | Notes                          |
| -------------- | -------------------------------------------------------------------- | ------------------------------ |
| GOVC\_URL      | Endpoint for vSphere                                                 | For example, `192.168.214.244` |
| GOVC\_USERNAME | Username for the vSphere user                                        | For example, `username`        |
| GOVC\_PASSWORD | Password for the vSphere user                                        | For example, `password`        |
| GOVC\_INSECURE | Skip SSL validation for govc commands                                | For example, `true`            |
| VM\_NAME       | Name of the vSphere VM where the fault-injector utility is installed | For example, `cf-vm`           |
| VM\_USERNAME   | Username for the VM guest user                                       | For example, `root`            |
| VM\_PASSWORD   | Password for the VM guest user                                       | For example, `password`        |

***

### Related faults <a href="#related-faults" id="related-faults"></a>

* [CF app container kill](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/cloud-foundry/cf-app-container-kill.md): Kill a single app container instance instead of stopping the whole app.
* [CF app route unmap](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/cloud-foundry/cf-app-route-unmap.md): Disconnect the app from its route without stopping the app itself.