Skip to main content

Kyverno

Kyverno policies block configurations that don't match a policy (enforce mode) or can generate policy violations (audit mode). It scans existing configurations and reports violations in the cluster. HCE recommends using the provided policy configuration to enable the execution of all supported (out-of-the-box) experiments listed in the Enterprise chaos hub. This is only a recommendation and left to user discretion depending on the experiments desired.

The details listed here are expected to aid users of Kyverno. If you are using alternate means to enforce runtime security, such as native Kubernetes PSPs (pod security policies), go here.

Policies in HCE

HCE recommends using the following policies:

  1. Add capabilities: It restricts add capabilities except the NET_ADMIN and SYS_ADMIN for the pods that use runtime API.
  2. Host namespaces: It validates following host namespaces for the pods that use runtime API.
    1. HostPID: It allows hostPID. It should be set to true.
    2. HostIPC: It restricts the host IPC. It should be set to false.
    3. HostNetwork: It restricts the hostNetwork. It should be set to false.
  3. Host paths: It restricts hostPath except the socket-path and container-path host paths for the pods that uses runtime API. It allows hostPaths for service-kill experiments.
  4. Privilege escalation: It restricts privilege escalation except for the pods that use runtime API.
  5. Privilege container: It restricts privileged containers except for the pods that use runtime API.
  6. User groups: It allows users groups for all the experiment pods.

Install policies

Kyverno policies are based on the Kubernetes pod security standards definitions. It is recommended that you apply all the pod security policies. To apply all the pod security policies, install Kyverno and kustomize, and run:

kustomize build https://github.com/litmuschaos/chaos-charts/security/kyverno-policies | kubectl apply -f -

Pod security policies in restricted setup

If a setup contains restricted policies, it will not allow HCE experiments to be executed by default. For example, deny-privilege-escalation policy doesn't allow privileged escalation. It denies all the pods of privileged escalation.

To allow HCE pods to use the privileged escalation, add the HCE service acccount or ClusterRole or Role inside the exclude block as:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: deny-privilege-escalation
annotations:
policies.kyverno.io/category: Pod Security Standards (Restricted)
policies.kyverno.io/severity: medium
policies.kyverno.io/subject: Pod
policies.kyverno.io/description: >-
Privilege escalation, such as via set-user-ID or set-group-ID file mode, should not be allowed.
This policy ensures the `allowPrivilegeEscalation` fields are either undefined
or set to `false`.
spec:
background: true
validationFailureAction: enforce
rules:
- name: deny-privilege-escalation
match:
resources:
kinds:
- Pod
exclude:
clusterRoles:
# add litmus cluster roles here
- litmus-admin
roles:
# add litmus roles here
- litmus-roles
subjects:
# add serviceAccount name here
- kind: ServiceAccount
name: pod-network-loss-sa
validate:
message: >-
Privilege escalation is disallowed. The fields
spec.containers[*].securityContext.allowPrivilegeEscalation, and
spec.initContainers[*].securityContext.allowPrivilegeEscalation must
be undefined or set to `false`.
pattern:
spec:
=(initContainers):
- =(securityContext):
=(allowPrivilegeEscalation): "false"
containers:
- =(securityContext):
=(allowPrivilegeEscalation): "false"