Skip to main content

CLB AZ down

CLB (Classic Load Balancer) AZ (Availability Zones) down takes down the AZ on a target CLB for a specific duration. This fault restricts access to certain availability zones for a specific duration.

CLB AZ Down

Use cases​

CLB AZ down:

  • Breaks the connectivity of a CLB with the given zones and impacts their delivery.
  • Detaches the AZ from the classic load balancer thereby disrupting the dependent application's performance.
  • Tests the application sanity, availability, and recovery workflows of the application pod attached to the load balancer.

Prerequisites​

  • Kubernetes >= 1.17
  • Appropriate AWS access to attach or detach an AZ from CLB.
  • A minimum of one AZ attached to the CLB after injecting chaos; otherwise, the fault fails to detach the given AZ.
  • The CLB should be attached to the given availability zones.
  • The Kubernetes secret should have AWS access configuration (key) in the CHAOS_NAMESPACE. Below is a sample secret file. 
    apiVersion: v1
    kind: Secret
    metadata:
      name: cloud-secret
    type: Opaque
    stringData:
      cloud_config.yml: |-
        # Add the cloud AWS credentials respectively
        [default]
        aws_access_key_id = XXXXXXXXXXXXXXXXXXX
        aws_secret_access_key = XXXXXXXXXXXXXXX
tip

HCE recommends that you use the same secret name, that is, cloud-secret. Otherwise, you will need to update the AWS_SHARED_CREDENTIALS_FILE environment variable in the fault template with the new secret name and you won't be able to use the default health check probes.

Below is an example AWS policy to execute the fault.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeInstanceStatus",
                "ec2:DescribeInstances",
                "ec2:DescribeSubnets",
                "elasticloadbalancing:DetachLoadBalancerFromSubnets",
                "elasticloadbalancing:AttachLoadBalancerToSubnets",
                "elasticloadbalancing:DescribeLoadBalancers"
            ],
            "Resource": "*"
        }
    ]
}
note

Mandatory tunables​

Tunable Description Notes
LOAD_BALANCER_NAME Name of the load balancer whose AZ has to be detached. For example, clb-name.
ZONES Target zones that have to be detached from the CLB. For example, us-east-1a. For more information, go to zones.
REGION Region name for the target volumes. For example, us-east-1.

Optional tunables​

Tunable Description Notes
TOTAL_CHAOS_DURATION Duration to insert chaos (in seconds). Default: 30 s. For more information, go to duration of the chaos.
AWS_SHARED_CREDENTIALS_FILE Path to the AWS secret credentials. Default: /tmp/cloud_config.yml.
CHAOS_INTERVAL Duration between the attachment and detachment of the volumes (in seconds). Default: 30 s. For more information, go to chaos interval.
SEQUENCE Sequence of chaos execution for multiple volumes. Default: parallel. Supports serial and parallel. For more information, go to sequence of chaos execution.
RAMP_TIME Period to wait before and after injecting chaos (in seconds). For example, 30 s. For more information, go to ramp time.

Target zones​

Comma-separated list of the target zones. Tune it by using the ZONES environment variable.

The following YAML snippet illustrates the use of this environment variable:

# contains clb az down for given zones
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: engine-nginx
spec:
  engineState: "active"
  chaosServiceAccount: litmus-admin
  experiments:
  - name: clb-az-down
    spec:
      components:
        env:
        # load balancer name for chaos
        - name: LOAD_BALANCER_NAME
          value: 'tes-clb'
        # target zones for the chaos
        - name: ZONES
          value: 'us-east-1a,us-east-1b'
        # region for chaos
        - name: REGION
          value: 'us-east-1'