Pod memory hog

Pod memory hog is a Kubernetes pod-level chaos fault that consumes memory resources in excess, resulting in a significant spike in the memory usage of a pod.

• Simulates a condition where the memory usage of an application spikes up unexpectedly.

Usage

View fault usage

Memory usage within containers is subject to various constraints in Kubernetes. If the limits are specified in their spec, exceeding them results in termination of the container (due to OOMKill of the primary process, often pid 1). This restarts container dependng on policy specified. For containers with no limits on memory, node can be killed based on their oom_score. This results in a bigger blast radius. It simulates the situation of memory leaks in the deployment of microservices, application slowness due to memory starvation, and noisy neighbour problems due to hogging. It verifies pod priority and QoS setting for eviction purposes. It also verifies application restarts on OOM kills. This fault causes stress within the target container, which may result in the primary process in the container to be constrained or eat up the available system memory on the node.

Prerequisites

• Kubernetes> 1.16.

Default validations

The application pods should be in running state before and after chaos injection.

Fault tunables

Fault tunables

Optional fields

Variables	Description	s
MEMORY_CONSUMPTION	The amount of memory used of hogging a Kubernetes pod (megabytes)	Defaults to 500MB
NUMBER_OF_WORKERS	The number of workers used to run the stress process	Defaults to 1
TOTAL_CHAOS_DURATION	The time duration for chaos insertion (seconds)	Defaults to 60s
LIB_IMAGE	Image used to run the helper pod.	Defaults to litmuschaos/go-runner:1.13.8
STRESS_IMAGE	Container run on the node at runtime by the pumba lib to inject stressors. Only used in LIB pumba	Default to alexeiled/stress-ng:latest-ubuntu
TARGET_PODS	Comma separated list of application pod name subjected to pod memory hog chaos	If not provided, it will select target pods randomly based on provided appLabels
TARGET_CONTAINER	Name of the target container under chaos.	If not provided, it will select the first container of the target pod
CONTAINER_RUNTIME	container runtime interface for the cluster	Defaults to docker, supported values: docker, containerd and crio for litmus and only docker for pumba LIB
SOCKET_PATH	Path of the containerd/crio/docker socket file	Defaults to /var/run/docker.sock
PODS_AFFECTED_PERC	The Percentage of total pods to target	Defaults to 0 (corresponds to 1 replica), provide numeric value only
RAMP_TIME	Period to wait before injection of chaos in sec	For example, 30
SEQUENCE	It defines sequence of chaos execution for multiple target pods	Default value: parallel. Supported: serial, parallel

Fault examples

Common and pod-specific tunables

Refer to the common attributes and pod-specific tunables to tune the common tunables for all fault and pod specific tunables.

Memory consumption

It specifies the amount of memory consumed by the target pod for a duration specified by TOTAL_CHAOS_DURATION environment variable. You can tune it using the MEMORY_CONSUMPTION environment variable.

Use the following example to tune it:

# define the memory consumption in MB
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: engine-nginx
spec:
  engineState: "active"
  annotationCheck: "false"
  appinfo:
    appns: "default"
    applabel: "app=nginx"
    appkind: "deployment"
  chaosServiceAccount: litmus-admin
  experiments:
    - name: pod-memory-hog
      spec:
        components:
          env:
            # memory consumption value
            - name: MEMORY_CONSUMPTION
              value: "500" #in MB
            - name: TOTAL_CHAOS_DURATION
              value: "60"

Workers for stress

It specifies the number of workers used to stress the resources. You can tune it using the NUMBER_OF_WORKERS environment variable.

Use the following example to tune it:

# number of workers used for the stress
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: engine-nginx
spec:
  engineState: "active"
  annotationCheck: "false"
  appinfo:
    appns: "default"
    applabel: "app=nginx"
    appkind: "deployment"
  chaosServiceAccount: litmus-admin
  experiments:
    - name: pod-memory-hog
      spec:
        components:
          env:
            # number of workers for stress
            - name: NUMBER_OF_WORKERS
              value: "1"
            - name: TOTAL_CHAOS_DURATION
              value: "60"

Container runtime and socket path

It defines the CONTAINER_RUNTIME and SOCKET_PATH envrionment variables to set the container runtime and socket file path, respectively.

• CONTAINER_RUNTIME: It supports docker, containerd, and crio runtimes. The default value is docker.
• SOCKET_PATH: It contains path of docker socket file by default(/var/run/docker.sock). For containerd, specify path as /var/containerd/containerd.sock. For crio, speecify path as /var/run/crio/crio.sock.