Skip to main content

Run chaos experiments in sandbox

This topic guides you through an interactive tutorial that uses a sample application (Podtato head) to execute a chaos experiment and improve the application's resilience.

Prerequisites​

Before you begin the tutorial, you need to have a Harness account. To create one, follow these steps. Go to onboarding flow diagram to understand the flow of obtaining the Harness developer certification.

Create a Harness account​

  1. Navigate to app.harness.io and Sign up.

    navigate

  2. Select Sign up with Email and enter your work email and password.

    enter details

  3. You will receive a verification email that leads you to the page with various modules.

    verify

High-level flow of control in a sandbox​

Below is a diagram that shows a high-level view of the flow of control in a sandbox environment:

control flow

Execute pod delete on the Podtato head app​

Once you have created an account with Harness, select Get started of the Chaos Engineering module.

landing

Create a sandbox​

  1. This leads you to the chaos module landing page. Select Create sandbox. Notice how it is a Zero risk operation!

    select module

  2. This creates a new environment where chaos experiments can be executed.

    env

  3. This takes you to the Interactive learning tab, where you click Start.

    start

note
  • Once you create a sandbox, you can't delete it.
  • All users within a sandbox can share the same sandbox.
  • The first time you create a sandbox, the session starts automatically. After that, you need to explicitly Start the session.
  • You will have a total of 20 sessions (each session is 60 minutes) valid for 30 days from the day of creation of the sandbox.
  • You can Stop anytime and restart the session. The remaining part of the time will be added to the unused quota.

Execute pod delete experiment​

  1. Experiment execution begins by creating the necessary clusters in the environment you created earlier. Once the clusters are ready, the sandbox creates the pod delete experiment. Select Next.

    exp created

  2. Select Run to execute the pod delete chaos experiment that was created earlier. In the background, the experiment creates resilience probes that check the availability of the Podtato app's left arm that is deleted during the pod delete experiment.

    exp runs

  3. You can see that the experiment deletes a pod (specifically the left arm pod) and displays the resilience of the Podtato head application.

    exp runs

note

You will see that the resilience score is 50. The reason behind the value is that when you execute the experiment, two probes of equal weight are configured in the background. When one of them fails, it results in a resilience score of 50.

  1. This shows that the Podtato head app is not resilient, and you can take steps to enhance the application's resilience. Select Next to configure the application to build resilience.

    click next

tip
  • You can re-run the experiment.
  • You can scale up the Podtato head app only once.
  • You can delete the Podtato head left arm only once.
  • You can execute pod network loss only after completing the pod delete experiment.
  • You can re-run the experiment.
  • You can't roll back to the previous step/experiment at any point.

Re-run to build resilience​

  1. You will see two commands on the screen, select Run to execute the commands. This shows that 2 replicas of the same pod are created, thereby improving the availability of the pod and building resilience. This way, even if one of the pods is deleted during the pod delete experiment, the replica pod can take over and the application can function without any glitch.

    improve resilience

  2. Select Next. Once the application is configured, select Run to re-run the experiment.

    improve resilience

  3. This runs the pod delete experiment again. This time, you will observe that the resilience score of the application is 100 percent. This means that even when the pod delete experiment randomly deleted a pod, another pod took its place and ensured that the app ran smoothly.

    improve score

note

The resilience score is 100 because both the probes that were configured with equal weight passed. probe passed

  1. Click Next to execute another experiment (pod network loss).

    execute next

tip

Click Run to execute the sandbox again if you run out of time.

finish

Perform the same steps for the next experiment​

  1. The above steps complete the execution of the pod delete experiment. There is one more experiment to execute, that is, pod network loss. You can follow the same steps as you did for the pod delete experiment to execute this experiment as well. In this experiment, a probe is configured in the background to check for the availability of the Podtato app's left arm, which is targeted.

    pod network loss

  2. Select Next to enter the last step of the hands-on lab.

    pod network done

  3. You have completed the Chaos hands-on lab, by executing two experiments in the sandbox and improving the resilience of the Podtato head application. Select Go to Harness University or follow this tutorial to obtain the developer certification.

    pod network done

Create experiments in sandbox​

Apart from executing pod delete and pod network loss experiments on the Podtato head application, you can also create other experiments in the sandbox.

  • Navigate to your sandbox, and select Chaos experiments. You will see the experiments that you executed earlier (or are currently being executed) in the sandbox.

    exp exec

  • Select New experiment and follow the steps to create an experiment.

Next steps​