> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/resilience-testing/chaos-engineering/faults/chaos-fault-categories/azure/azure-instance-stop.md).

# Azure instance stop

Azure instance stop is an Azure chaos fault that stops (deallocates) one or more Virtual Machines listed in `AZURE_INSTANCE_NAMES` (in `RESOURCE_GROUP`, subscription `AZURE_SUBSCRIPTION_ID`) for `TOTAL_CHAOS_DURATION` seconds, then starts them again. When `SCALE_SET=enable`, the fault deallocates VMSS instances; the scale set's auto-recovery decides whether to bring them back.

Use this fault to test how a workload behaves when a VM disappears: whether load balancers shift traffic, whether VMSS auto-healing recreates the instance inside the alerting SLA, whether AKS node-down handling reschedules pods, and whether monitoring detects the outage within the alerting SLA.

{% hint style="info" %}
**RUN YOUR FIRST EXPERIMENT**

If you have not configured the chaos infrastructure yet, go to [Quickstart](/resilience-testing/chaos-engineering/new-to-chaos-engineering/quickstart.md) to install the chaos infrastructure and run an experiment end to end.
{% endhint %}

***

### Use cases <a href="#use-cases" id="use-cases"></a>

Run this fault when you want to answer concrete questions like:

* **VM disappears:** When the target VM deallocates, do load balancers fail traffic over inside the SLA?
* **VMSS recovery:** Does the scale set recreate the deallocated instance with the expected boot time?
* **AKS node-down handling:** If the VM is an AKS worker, does the cluster drain pods and reschedule them on healthy nodes?
* **Monitoring fidelity:** Do alerts on `Microsoft.Compute/virtualMachines/Availability`, instance count, and end-to-end availability fire within the alerting SLA?

***

### Prerequisites <a href="#prerequisites" id="prerequisites"></a>

* **Kubernetes version:** 1.21 or later for the chaos infrastructure cluster.
* **Target VMs reachable:** Each entry in `AZURE_INSTANCE_NAMES` exists in `RESOURCE_GROUP` inside `AZURE_SUBSCRIPTION_ID`.
* **VM in `running` state:** The fault refuses to deallocate a VM that is already `stopped` or `deallocated`.
* **Azure credentials available:** A service principal JSON delivered as a File Secret in Harness Secret Manager, workload identity bound to the chaos infra service account, or managed identity on the AKS node pool.
* **RBAC granted:** The principal includes the role listed below.

***

### Supported environments <a href="#supported-environments" id="supported-environments"></a>

| Platform                            | Support status                                                |
| ----------------------------------- | ------------------------------------------------------------- |
| Standalone Virtual Machines         | Supported                                                     |
| Virtual Machine Scale Set instances | Supported (set `SCALE_SET=enable`)                            |
| AKS worker nodes (VMSS-backed)      | Supported with `SCALE_SET=enable`                             |
| Spot VMs                            | Supported (note: Azure may not start them back automatically) |

***

### Permissions required <a href="#permissions-required" id="permissions-required"></a>

The Azure principal used by the chaos pod (service principal, workload identity, or managed identity) needs the following role on the target resource group or subscription.

**Recommended built-in role:** `Virtual Machine Contributor`

**Custom role (minimum actions):**

```json
{
  "Name": "Harness Chaos VM Stop",
  "Actions": [
    "Microsoft.Compute/virtualMachines/read",
    "Microsoft.Compute/virtualMachines/start/action",
    "Microsoft.Compute/virtualMachines/deallocate/action",
    "Microsoft.Compute/virtualMachineScaleSets/virtualMachines/read",
    "Microsoft.Compute/virtualMachineScaleSets/virtualMachines/start/action",
    "Microsoft.Compute/virtualMachineScaleSets/virtualMachines/deallocate/action"
  ],
  "AssignableScopes": ["/subscriptions/<SUBSCRIPTION_ID>"]
}
```

Go to [Azure fault permissions](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/azure/security-configurations/fault-permissions.md) to read the full permission catalog.

***

### Authentication <a href="#authentication" id="authentication"></a>

Pick one of the following methods. Go to [Azure authentication methods](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/azure/security-configurations/azure-authentication-methods.md) to read the full setup.

| Method            | When to use it                                                                                             | How to configure                                                                                                                         |
| ----------------- | ---------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| Service principal | Chaos infrastructure runs outside AKS, or you want explicit static credentials                             | Upload the service principal JSON file as a **File Secret in Harness Secret Manager** and reference it via `AZURE_AUTHENTICATION_SECRET` |
| Workload identity | Chaos infrastructure runs on AKS with OIDC issuer + workload identity enabled                              | Annotate the chaos infra service account with `azure.workload.identity/client-id`; the pod authenticates without static credentials      |
| Managed identity  | Chaos infrastructure runs on AKS with a system-assigned or user-assigned managed identity on the node pool | No tunable changes; the pod inherits the identity from IMDS                                                                              |

***

### Fault tunables <a href="#fault-tunables" id="fault-tunables"></a>

Configure the following fault parameters when you add Azure instance stop to an experiment in Chaos Studio. Defaults are shown for reference.

**Required parameters**

| Tunable                | Description                                                 | Default    |
| ---------------------- | ----------------------------------------------------------- | ---------- |
| `AZURE_INSTANCE_NAMES` | Comma-separated list of VM names (for example `vm-1,vm-2`). | (required) |
| `RESOURCE_GROUP`       | Resource group that contains the VMs.                       | (required) |

**Chaos parameters**

| Tunable                | Description                                                                                                                                                                                                      | Default    |
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- |
| `TOTAL_CHAOS_DURATION` | Total duration of the fault in seconds. The VMs stay deallocated for this period.                                                                                                                                | `30`       |
| `CHAOS_INTERVAL`       | Delay in seconds between successive iterations when running for more than one cycle.                                                                                                                             | `30`       |
| `SCALE_SET`            | Set to `enable` when the VMs belong to a Virtual Machine Scale Set. Otherwise leave empty.                                                                                                                       | `""`       |
| `SEQUENCE`             | Order in which multiple instances are stopped: `parallel` or `serial`.                                                                                                                                           | `parallel` |
| `RAMP_TIME`            | Wait period in seconds before and after the fault. Go to [ramp time](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/common-tunables-for-all-faults.md#ramp-time) to read how it is applied. | `0`        |

**Authentication**

| Tunable                       | Description                                                                                           | Default |
| ----------------------------- | ----------------------------------------------------------------------------------------------------- | ------- |
| `AZURE_SUBSCRIPTION_ID`       | Target Azure subscription ID. Required when using workload identity or managed identity.              | `""`    |
| `AZURE_CLIENT_ID`             | Client ID of a user-assigned managed identity (only if you have multiple identities attached).        | `""`    |
| `AZURE_AUTHENTICATION_SECRET` | Identifier of the **File Secret in Harness Secret Manager** that contains the service principal JSON. | `""`    |

Tunables that apply to every fault are documented in [common tunables for all faults](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/common-tunables-for-all-faults.md).

***

### Fault execution in brief <a href="#fault-execution-in-brief" id="fault-execution-in-brief"></a>

Calls the Azure Resource Manager API to deallocate each VM in `AZURE_INSTANCE_NAMES` (in `RESOURCE_GROUP`), waits for `TOTAL_CHAOS_DURATION` seconds, then starts the VMs again.

***

### Expected behavior during fault execution <a href="#expected-behavior-during-fault-execution" id="expected-behavior-during-fault-execution"></a>

* The target VMs transition `running` → `stopping` → `deallocated` and stay there for `TOTAL_CHAOS_DURATION`.
* For AKS worker nodes: pods on the node go to `NotReady`/`Unknown`, then the scheduler reschedules them.
* For VMs behind a load balancer: backend health probes fail; traffic shifts to healthy backends.
* After the duration ends, the VMs transition back to `running`.

{% hint style="info" %}
**WHEN THE FAULT ENDS**

The chaos pod calls `start` on every targeted VM. Boot time depends on the OS image and post-boot init scripts.
{% endhint %}

#### Signals to watch <a href="#signals-to-watch" id="signals-to-watch"></a>

Attach [resilience probes](/resilience-testing/chaos-engineering/use-chaos-engineering/probes.md) to assert each layer:

* **Instance state:** Use a [command probe](/resilience-testing/chaos-testing/probes/command-probe.md) running `az vm get-instance-view -g <rg> -n <vm> --query 'instanceView.statuses[?starts_with(code, ''PowerState/'')].code'` and assert the state changed.
* **Application availability:** Use an [HTTP probe](/resilience-testing/chaos-engineering/use-chaos-engineering/probes/http-probe.md) on the user-visible endpoint behind the load balancer.

***

### Verify the fault execution effect <a href="#verify-the-fault-execution-effect" id="verify-the-fault-execution-effect"></a>

1. **Inspect VM power state with `az`.**

   ```bash
   az vm get-instance-view --resource-group <rg> --name <vm> \
     --query "instanceView.statuses[?starts_with(code, 'PowerState/')].code"
   ```

   The state should be `PowerState/deallocated` during the chaos window and `PowerState/running` afterwards.
2. **Inspect Azure activity log.**

   ```bash
   az monitor activity-log list --resource-group <rg> --max-events 20 \
     --query "[?contains(operationName.value,'deallocate')]"
   ```

***

### Recovery and cleanup <a href="#recovery-and-cleanup" id="recovery-and-cleanup"></a>

* **End of duration:** The chaos pod calls `start` on every targeted VM.
* **Abort the experiment:** Stopping the experiment from Chaos Studio also calls `start`.
* **Manual recovery:** If the chaos pod exited before restart, run `az vm start --resource-group <rg> --name <vm>` manually.
* **Workload recovery:** Boot time depends on the OS image and init scripts; AKS node `Ready` transitions usually complete within 2-3 minutes.

***

### Limitations <a href="#limitations" id="limitations"></a>

* **Same-subscription targeting:** A single experiment targets one `AZURE_SUBSCRIPTION_ID`.
* **Resource group scope:** All entries in `AZURE_INSTANCE_NAMES` must be in `RESOURCE_GROUP`.
* **Spot VMs:** Azure may not start back evicted Spot VMs automatically.
* **VMSS instance IDs:** When `SCALE_SET=enable`, entries in `AZURE_INSTANCE_NAMES` are VMSS instance IDs (`0`, `1`, ...), not VM names.

***

### Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

<details>

<summary>Azure instance stop fails with AuthorizationFailed in Harness Chaos Engineering</summary>

The Azure principal used by the chaos pod is missing Microsoft.Compute/virtualMachines/deallocate or start. Assign the Virtual Machine Contributor role (or a custom role with the required actions) on the target resource group or subscription.

</details>

<details>

<summary>Azure instance stop fails with ResourceNotFound</summary>

Confirm each VM name in AZURE\_INSTANCE\_NAMES exists in RESOURCE\_GROUP with az vm list -g --query '\[].name'. Confirm AZURE\_SUBSCRIPTION\_ID matches the subscription that owns the resource group.

</details>

<details>

<summary>VMs stayed deallocated after the experiment ended</summary>

If the chaos pod exited before start, run az vm start --resource-group --name manually. For VMSS instances, run az vmss start --resource-group --name --instance-ids .

</details>

***

### Related faults <a href="#related-faults" id="related-faults"></a>

* [Azure AKS node down](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/azure/azure-aks-node-down.md): Deallocate AKS VMSS nodes selected by node pool or zone.
* [Azure disk loss](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/azure/azure-disk-loss.md): Detach disks instead of stopping VMs.
* [Azure web app stop](/resilience-testing/chaos-engineering/faults/chaos-fault-categories/azure/azure-web-app-stop.md): Stop an App Service web app instead of a VM.