Skip to main content

CF app network latency

CF app network loss injects network latency into a Cloud Foundry app instance, causing it to delay network connectivity.

CF App Network Latency

Use cases

CF app network latency:

  • Checks resilience upon app network latency.
  • Validates the effectiveness of disaster recovery and high availability of the app.

Mandatory tunables

Tunable Description Notes
cfDeploymentPlatform Deployment platform used for cloud foundry with respect to where the infrastructure is hosted. Supports local and vSphere. For more information, go to CF deployment platform.
organization Organization where the target app resides. For example, dev-org
space Space where the target app resides. The space must reside within the given organization. For example, dev-space
app The app to be stopped. The app must reside within the given organization and space. For example, cf-app
boshDeployment The bosh deployment under which the CF components are being managed. It can be obtained using the BOSH CLI command bosh deployments. For more information, go to BOSH deployment.

Optional tunables

Tunable Description Notes
instanceAffectedPercentage Percentage of total number of app instances that will be targeted. Default: 0 (1 instance). For more information, go to instance affected percentage.
networkInterface The network interface used by the container. Default: eth0
destinationHosts List of the target host names or keywords. For example, google.com,litmuschaos.io. If neither destinationHosts nor destinationIPs is present, the fault injects chaos for all host names or domains.
destinationIPs List of the target IPs. For example, 1.1.1.1,8.8.8.8 If neither destinationHosts nor destinationIPs is provided, all host names or domains are targeted.
networkLatency Amount of latency to added to connection in ms. For example: 2000 Defaults to 2000
jitter Amount of jitter to be added in ms. Jitter will define the max randomised deviation from the provided latency value. For example: 100 Defaults to 0
sourcePorts Source ports to be filtered for chaos. For example: 5000,8080 . Alternatively, the ports can be whitelisted, that is, filtered to be exempt from chaos. Prepend a ! to the list of ports to be exempted. For example, !5000,8080 .
destinationPorts Destination ports to be filtered for chaos. For example: 5000,8080 Alternatively, the ports can be whitelisted, that is, filtered to be exempt from chaos. Prepend a ! to the list of ports to be exempted. For example, !5000,8080 .
faultInjectorPort Local server port used by the fault-injector utility. Default: 50320. If the default port is unavailable, a random port in the range of 50320-51320 is selected. For more information, go to fault injector port.
duration Duration through which chaos is injected into the target resource (in seconds). Default: 30s. For more information, go to chaos duration.
skipSSLValidation Skip SSL validation while invoking CF APIs. Supports true and false. Default: false. For more information, go to skip SSL validation.
rampTime Period to wait before and after injecting chaos (in seconds). Defaults to 0.

CF secrets

The following Cloud Foundry secrets reside on the same machine where the chaos infrastructure is executed. These secrets are provided in the /etc/linux-chaos-infrastructure/cf.env file in the following format:

CF_API_ENDPOINT=XXXXXXXXXXXXXXXXXXX
CF_USERNAME=XXXXXXXXXXXXXXXXXXXXXXX
CF_PASSWORD=XXXXXXXXXXXXXXXXXXXXXXX
UAA_SERVER_ENDPOINT=XXXXXXXXXXXXXXX
BOSH_CLIENT=XXXXXXXXXXXXXXXXXXXXXXX
BOSH_CLIENT_SECRET=XXXXXXXXXXXXXXXX
BOSH_CA_CERT=XXXXXXXXXXXXXXXXXXXXXX
BOSH_ENVIRONMENT=XXXXXXXXXXXXXXXXXX
info

If the secrets file is not provided, the secrets are attempted to be derived from environment variables and the config file by the fault-injector.

ENV nameDescriptionExample
CF_API_ENDPOINTAPI endpoint for the CF setuphttps://api.system.cf-setup.com
CF_USERNAMEUsername for the CF userusername
CF_PASSWORDPassword for the CF userpassword
UAA_SERVER_ENDPOINTAPI endpoint for the UAA server for the CF setuphttps://uaa.system.cf-setup.com
BOSH_CLIENTUsed by the bosh CLI, the BOSH clientadmin
BOSH_CLIENT_SECRETUsed by the bosh CLI, the BOSH client secretUBu9Fu3oW35sO6fw12auPH76gsRTy7
BOSH_CA_CERTUsed by the bosh CLI, the file path for BOSH CA certificate/root/root_ca_certificate
BOSH_ENVIRONMENTUsed by the bosh CLI, the BOSH environmentbosh.corp.local

Fault injector ENVs and config file

If /etc/linux-chaos-infrastructure/cf.env file is not provided, fault-injector attempts to derive the secrets from environment variables or a configuration file. Any secret that is re-declared will be overridden in the following order of decreasing precedence:

  1. /etc/linux-chaos-infrastructure/cf.env file
  2. Environment variables
  3. Configuration file

The configuration file should be provided at /etc/linux-chaos-infrastructure/cf-fault-injector.yaml:

cf-api-endpoint: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
username: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
password: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
uaa-server-endpoint: XXXXXXXXXXXXXXXXXXXXXXXXXX
bosh-client: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
bosh-client-secret: XXXXXXXXXXXXXXXXXXXXXXXXXXX
bosh-ca-cert: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
bosh-environment: XXXXXXXXXXXXXXXXXXXXXXXXXXXXX

A mapping between all the three formats for providing the secrets is as follows:

cf.envENVcf-fault-injector.yaml
CF_API_ENDPOINTCF_API_ENDPOINTcf-api-endpoint
CF_USERNAMEUSERNAMEusername
CF_PASSWORDPASSWORDpassword
UAA_SERVER_ENDPOINTUAA_SERVER_ENDPOINTuaa-server-endpoint
BOSH_CLIENTBOSH_CLIENTbosh-client
BOSH_CLIENT_SECRETBOSH_CLIENT_SECRETbosh-client-secret
BOSH_CA_CERTBOSH_CA_CERTbosh-ca-cert
BOSH_ENVIRONMENTBOSH_ENVIRONMENTbosh-environment

vSphere secrets

These secrets are provided only if vSphere is used as the deployment platform for CF.

The following vSphere secrets reside on the same machine where the chaos infrastructure is executed. These secrets are provided in the /etc/linux-chaos-infrastructure/vsphere.env file in the following format:

GOVC_URL=XXXXXXXXXXXXXXXXXXXXXX
GOVC_USERNAME=XXXXXXXXXXXXXXXXX
GOVC_PASSWORD=XXXXXXXXXXXXXXXXX
GOVC_INSECURE=XXXXXXXXXXXXXXXXX
VM_NAME=XXXXXXXXXXXXXXXXXXXXXXX
VM_USERNAME=XXXXXXXXXXXXXXXXXXX
VM_PASSWORD=XXXXXXXXXXXXXXXXXXX
ENV Name Description Notes
GOVC_URL Endpoint for vSphere For example, 192.168.214.244
GOVC_USERNAME Username for the vSphere user For example, username
GOVC_PASSWORD Password for the vSphere user For example, password
GOVC_INSECURE Skip SSL validation for govc commands For example, true
VM_NAME Name of the vSphere VM where the fault-injector utility is installed For example, cf-vm
VM_USERNAME Username for the VM guest user For example, root
VM_PASSWORD Password for the VM guest user For example, password

Destination hosts

The destinationHosts input variable subjects the comma-separated names of the target hosts to chaos.

The following YAML snippet illustrates the use of this environment variable:

apiVersion: litmuchaos.io/v1alpha1
kind: LinuxFault
metadata:
name: cf-app-network-latency
labels:
name: app-network-latency
spec:
cfAppNetworkChaos/inputs:
duration: 30s
cfDeploymentPlatform: vSphere
app: cf-app
organization: dev-org
space: dev-space
destinationHosts: 'google.com'

Destination IPs

The destinationIPs input variable subjects the comma-separated names of the target IPs to chaos.

The following YAML snippet illustrates the use of this environment variable:

apiVersion: litmuchaos.io/v1alpha1
kind: LinuxFault
metadata:
name: cf-app-network-latency
labels:
name: app-network-latency
spec:
cfAppNetworkChaos/inputs:
duration: 30s
cfDeploymentPlatform: vSphere
app: cf-app
organization: dev-org
space: dev-space
destinationIPs: '1.1.1.1'

Source and destination ports

By default, the network experiments disrupt traffic for all the source and destination ports. Tune the interruption of specific port(s) using sourcePorts and destinationPorts inputs, respectively.

  • sourcePorts: Ports of the target application whose accessibility is impacted.
  • destinationPorts: Ports of the destination services or pods or the CIDR blocks(range of IPs) whose accessibility is impacted.

The following YAML snippet illustrates the use of this environment variable:

apiVersion: litmuchaos.io/v1alpha1
kind: LinuxFault
metadata:
name: cf-app-network-latency
labels:
name: app-network-latency
spec:
cfAppNetworkChaos/inputs:
duration: 30s
cfDeploymentPlatform: vSphere
app: cf-app
organization: dev-org
space: dev-space
destinationIPs: '1.1.1.1'
sourcePorts: '8080,3000'
destinationPorts: '5000,3000'

Ignore Source and Destination Ports

By default, the network experiments disrupt traffic for all the source and destination ports. Ignore the specific ports using sourcePorts and destinationPorts inputs, respectively.

  • sourcePorts: Provide source ports that are not subject to chaos as comma-separated values preceded by !.
  • destinationPorts: Provide destination ports that are not subject to chaos as comma-separated values preceded by !.

The following YAML snippet illustrates the use of this environment variable:

apiVersion: litmuchaos.io/v1alpha1
kind: LinuxFault
metadata:
name: cf-app-network-latency
labels:
name: app-network-latency
spec:
cfAppNetworkChaos/inputs:
duration: 30s
cfDeploymentPlatform: vSphere
app: cf-app
organization: dev-org
space: dev-space
destinationIPs: '1.1.1.1'
sourcePorts: '!8080,3000'
destinationPorts: '!5000,3000'

Latency and jitter

The latency and jitter input variables add delay and a small deviation to the delay, respectively, with respect to the connection.

The following YAML snippet illustrates the use of this environment variable:

apiVersion: litmuchaos.io/v1alpha1
kind: LinuxFault
metadata:
name: cf-app-network-latency
labels:
name: app-network-latency
spec:
networkChaos/inputs:
duration: 30s
cfDeploymentPlatform: vSphere
app: cf-app
organization: dev-org
space: dev-space
networkLatency: '1000'
jitter: '200'

BOSH deployment

The boshDeployment input determines the BOSH deployment name under which all the CF resources are being managed. It can be obtained using the BOSH CLI command bosh deployments.

The following YAML snippet illustrates the use of this environment variable:

# bosh deployment
apiVersion: litmuchaos.io/v1alpha1
kind: LinuxFault
metadata:
name: cf-app-network-latency
labels:
name: app-network-latency
spec:
cfAppNetworkChaos/inputs:
duration: 30s
cfDeploymentPlatform: vSphere
app: cf-app
organization: dev-org
space: dev-space
boshDeployment: cf

Instance affected percentage

The instanceAffectedPercentage input specifies the percentage of total number of app instances that will be targeted. It defaults to 0 (1 instance).

The following YAML snippet illustrates the use of this environment variable:

# instance affected percentage
apiVersion: litmuchaos.io/v1alpha1
kind: LinuxFault
metadata:
name: cf-app-network-latency
labels:
name: app-network-latency
spec:
cfAppNetworkChaos/inputs:
duration: 30s
cfDeploymentPlatform: vSphere
app: cf-app
organization: dev-org
space: dev-space
boshDeployment: cf
instanceAffectedPercentage: 50

CF deployment platform

The cfDeploymentPlatform input variable determines the deployment platform used for CF with respect to the infrastructure.

  • The deployment platform can be local, that is, the same environment used by the infrastructure, or a remote machine.
  • The deployment platform is where the fault-injector utility executes.

The following YAML snippet illustrates the use of this environment variable:

# cf deployment platform
apiVersion: litmuchaos.io/v1alpha1
kind: LinuxFault
metadata:
name: cf-app-network-latency
labels:
name: app-network-latency
spec:
cfAppNetworkChaos/inputs:
duration: 30s
cfDeploymentPlatform: vSphere
app: cf-app
organization: dev-org
space: dev-space

Skip SSL validation

The skipSSLValidation input variable determines whether to skip SSL validation for calling the CF APIs.

The following YAML snippet illustrates the use of this environment variable:

# skip ssl validation for cf
apiVersion: litmuchaos.io/v1alpha1
kind: LinuxFault
metadata:
name: cf-app-network-latency
labels:
name: app-network-latency
spec:
cfAppNetworkChaos/inputs:
duration: 30s
cfDeploymentPlatform: vSphere
app: cf-app
organization: dev-org
space: dev-space
skipSSLValidation: true

Fault injector port

The faultInjectorPort input variable determines the port used for the fault-injector local server.

The following YAML snippet illustrates the use of this environment variable:

# fault injector port
apiVersion: litmuchaos.io/v1alpha1
kind: LinuxFault
metadata:
name: cf-app-network-latency
labels:
name: app-network-latency
spec:
cfAppNetworkChaos/inputs:
duration: 30s
cfDeploymentPlatform: local
app: cf-app
organization: dev-org
space: dev-space
faultInjectorPort: 50331