Build custom delegate images with third-party tools
Harness Manager installs and configures delegates with the binaries that most CI/CD pipelines require. In some cases, however, a preconfigured image isn't the right fit. For example, preconfigured images can:
- Introduce the vulnerabilities of the binaries they include.
- Restrict you to the use of the included third-party tools and versions.
This document explains how you can:
- Build and host a custom delegate image that includes the tools you select.
- Use your custom delegate in CI/CD pipelines.
Delegates with an immutable image type (image tag yy.mm.xxxxx
) include non-root user privileges and are compatible with OpenShift. For information on delegate types, go to Delegate image types.
About the Harness Delegate minimal image
Harness recommends that you use the Harness Delegate minimal image (yy.mm.xxxxx.minimal
) when you set up the Harness Platform for production use. This image has been thoroughly scanned and is free of any high or critical vulnerabilities. Users focused on security tend to prefer this option.
However, the minimal delegate image lacks some binaries that are required for Continuous Delivery (CD) steps to function properly and remain vulnerability-free from third-party tools. Consequently, using the minimal delegate image requires you to configure your delegates and install necessary binaries. For information on delegate types, go to Delegate image types.
The Harness Delegate minimal image (yy.mm.xxxxx.minimal
) is a lighter, more secure version of the default Harness Delegate image. Its main purpose is to provide an enhanced security profile for users, especially those who prioritize their systems' security. The Harness Delegate minimal images includes the following features.
-
Security Scanned: The image undergoes rigorous scanning processes to ensure that it is devoid of any high-risk or critical vulnerabilities. This makes it an optimal choice for organizations or users who have stringent security requirements. Harness aims to minimize critical/high vulnerabilities within this image. Achieving complete mitigation isn't always possible due to the continual discovery of vulnerabilities in third-party libraries/tools without immediate remediation.
-
Limited Binaries: Unlike the standard delegate, the minimal image does not include all of the default binaries. While this contributes to its lightweight nature and security, it also means that users have additional responsibilities. They must manually configure and add any necessary binaries to make their setup functional.
-
User Responsibilities: Because the minimal delegate image is devoid of the default binaries, users are in charge of tailoring it to their needs. This includes installing specific binaries essential for their CD steps. This level of control also allows users to maintain an updated environment. By installing the latest versions of necessary binaries, they can ensure that the delegate remains free from potential vulnerabilities found in outdated third-party tools.
-
Preferred by Security-Conscious Users: Due to its clean security slate, many users who prioritize system security gravitate towards the minimal delegate image. By starting with a minimal setup and adding only what is necessary, they can maintain a tighter control over the software and tools present, thus minimizing potential security risks.
Use the delegate minimal image to create a custom delegate image
Select the delegate minimal image
You can build on either of the following Harness-provided images.
Image | Description |
---|---|
Harness Delegate Docker image | A publicly available Docker image providing Harness Delegate. |
Harness Minimal Delegate Docker image | A minimal delegate image is available in Docker Hub at https://hub.docker.com/r/harness/delegate/tags. |
Use the last published yy.mm.xxxxx
version of the minimal image from the Docker repository.
Build the delegate image
When you build a custom delegate image, you modify the image you select with user privileges and binaries. This section explains the build script used for the process. In this example, the script builds a custom image for deployment by Kubernetes and by Terraform.
The first lines of the script provide information about the base image and user privileges. This example uses the minimal image with delegate minor version 77029.
FROM harness/delegate:24.04.82804.minimal
USER root
The delegate container is granted root user privileges.
The first RUN
block installs or updates the unzip
and yum-utils
tools. The --nodocs
option prevents the installation of documentation on the image.
RUN microdnf update \
&& microdnf install --nodocs \
unzip \
yum-utils
The second RUN
block uses the yum
utility to create a configuration file for the HashiCorp repository, and then uses the microdnf
package manager to install the required Terraform components:
RUN yum-config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo \
&& microdnf install -y terraform
The final RUN
block retrieves the Kubernetes kubectl
command-line tool that is required to manipulate clusters. The Linux chmod +x
instruction makes the utility executable:
RUN mkdir /opt/harness-delegate/tools && cd /opt/harness-delegate/tools \
&& curl -LO "https://dl.k8s.io/release/$(curl> -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" && chmod +x kubectl
The ENV
instruction defines the Linux $PATH
environment variable that provides the location of the tools to be installed:
ENV PATH=/opt/harness-delegate/tools/:$PATH
The final instruction switches the user back to harness
to ensure the custom image does not run as root:
USER harness
The complete script is as follows:
FROM harness/delegate:24.04.82804.minimal
USER root
RUN microdnf update \
&& microdnf install --nodocs \
unzip \
yum-utils
RUN yum-config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo \
&& microdnf install -y terraform
RUN mkdir /opt/harness-delegate/tools && cd /opt/harness-delegate/tools \
&& curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" && chmod +x kubectl
ENV PATH=/opt/harness-delegate/tools/:$PATH
USER harness
The following example Dockerfile adds all the tools necessary for the Harness platform that are not part of the base image to the minimal delegate. You can remove tools for features you don't use or update versions for your requirements.
Example Dockerfile with all tools
FROM harness/delegate:yy.mm.xxxxx.minimal
USER 0
ENV TARGETARCH=amd64
RUN microdnf install --nodocs git \
&& microdnf clean all \
&& rm -rf /var/cache/yum
RUN mkdir -m 777 -p client-tools/kubectl/v1.24.3 \
&& curl -s -L -o client-tools/kubectl/v1.24.3/kubectl https://app.harness.io/public/shared/tools/kubectl/release/v1.24.3/bin/linux/$TARGETARCH/kubectl \
&& mkdir -m 777 -p client-tools/helm/v2.13.1 \
&& curl -s -L -o client-tools/helm/v2.13.1/helm https://app.harness.io/public/shared/tools/helm/release/v2.13.1/bin/linux/$TARGETARCH/helm \
&& mkdir -m 777 -p client-tools/helm/v3.1.2 \
&& curl -s -L -o client-tools/helm/v3.1.2/helm https://app.harness.io/public/shared/tools/helm/release/v3.1.2/bin/linux/$TARGETARCH/helm \
&& mkdir -m 777 -p client-tools/helm/v3.8.0 \
&& curl -s -L -o client-tools/helm/v3.8.0/helm https://app.harness.io/public/shared/tools/helm/release/v3.8.0/bin/linux/$TARGETARCH/helm \
&& mkdir -m 777 -p client-tools/go-template/v0.4.2 \
&& curl -s -L -o client-tools/go-template/v0.4.2/go-template https://app.harness.io/public/shared/tools/go-template/release/v0.4.2/bin/linux/$TARGETARCH/go-template \
&& mkdir -m 777 -p client-tools/harness-pywinrm/v0.4-dev \
&& curl -s -L -o client-tools/harness-pywinrm/v0.4-dev/harness-pywinrm https://app.harness.io/public/shared/tools/harness-pywinrm/release/v0.4-dev/bin/linux/$TARGETARCH/harness-pywinrm \
&& mkdir -m 777 -p client-tools/chartmuseum/v0.15.0 \
&& curl -s -L -o client-tools/chartmuseum/v0.15.0/chartmuseum https://app.harness.io/public/shared/tools/chartmuseum/release/v0.15.0/bin/linux/$TARGETARCH/chartmuseum \
&& mkdir -m 777 -p client-tools/tf-config-inspect/v1.2 \
&& curl -s -L -o client-tools/tf-config-inspect/v1.2/terraform-config-inspect https://app.harness.io/public/shared/tools/terraform-config-inspect/v1.2/linux/$TARGETARCH/terraform-config-inspect \
&& mkdir -m 777 -p client-tools/oc/v4.2.16 \
&& curl -s -L -o client-tools/oc/v4.2.16/oc https://app.harness.io/public/shared/tools/oc/release/v4.2.16/bin/linux/$TARGETARCH/oc \
&& mkdir -m 777 -p client-tools/kustomize/v4.5.4 \
&& curl -s -L -o client-tools/kustomize/v4.5.4/kustomize https://app.harness.io/public/shared/tools/kustomize/release/v4.5.4/bin/linux/$TARGETARCH/kustomize \
&& mkdir -m 777 -p client-tools/scm/f1024c6b \
&& curl -s -L -o client-tools/scm/f1024c6b/scm https://app.harness.io/public/shared/tools/scm/release/f1024c6b/bin/linux/$TARGETARCH/scm \
&& chmod -R 775 /opt/harness-delegate \
&& chgrp -R 0 /opt/harness-delegate \
&& chown -R 1001 /opt/harness-delegate
ENV PATH=/opt/harness-delegate/client-tools/kubectl/v1.24.3/:$PATH
ENV PATH=/opt/harness-delegate/client-tools/go-template/v0.4.2/:$PATH
ENV PATH=/opt/harness-delegate/client-tools/chartmuseum/v0.15.0/:$PATH
ENV PATH=/opt/harness-delegate/client-tools/tf-config-inspect/v1.2/:$PATH
ENV PATH=/opt/harness-delegate/client-tools/kustomize/v4.5.4/:$PATH
USER 1001
Upload the image to Docker Hub
The next step is to upload your custom image to Docker Hub. For information on working with Docker repositories, go to Manage repositories in the Docker documentation.
Modify the delegate manifest
Before you can deploy a delegate, you must:
- Update the image path to the repository location of the custom image.
- Suspend delegate auto-upgrade functionality.
Delegate auto-upgrade is not compatible with custom images.
Example manifest file
apiVersion: v1
kind: Namespace
metadata:
name: harness-delegate-ng
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: harness-delegate-ng-cluster-admin
subjects:
- kind: ServiceAccount
name: default
namespace: harness-delegate-ng
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
---
apiVersion: v1
kind: Secret
metadata:
name: custom-del-account-token
namespace: harness-delegate-ng
type: Opaque
data:
DELEGATE_TOKEN: ""
---
# If delegate needs to use a proxy, please follow instructions available in the documentation
# /docs/first-gen/firstgen-platform/account/manage-delegates/configure-delegate-proxy-settings/
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
harness.io/name: custom-del
name: custom-del
namespace: harness-delegate-ng
spec:
replicas: 1
selector:
matchLabels:
harness.io/name: custom-del
template:
metadata:
labels:
harness.io/name: custom-del
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "3460"
prometheus.io/path: "/api/metrics"
spec:
terminationGracePeriodSeconds: 600
restartPolicy: Always
containers:
- image: foobar/org:custom-delegate
imagePullPolicy: Always
name: delegate
securityContext:
allowPrivilegeEscalation: false
runAsUser: 0
ports:
- containerPort: 8080
resources:
limits:
cpu: "0.5"
memory: "2048Mi"
requests:
cpu: "0.5"
memory: "2048Mi"
livenessProbe:
httpGet:
path: /api/health
port: 3460
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
failureThreshold: 2
startupProbe:
httpGet:
path: /api/health
port: 3460
scheme: HTTP
initialDelaySeconds: 30
periodSeconds: 10
failureThreshold: 15
envFrom:
- secretRef:
name: custom-del-account-token
env:
- name: JAVA_OPTS
value: "-Xms64M"
- name: ACCOUNT_ID
value:
- name: MANAGER_HOST_AND_PORT
value: https://app.harness.io/gratis
- name: DEPLOY_MODE
value: KUBERNETES
- name: DELEGATE_NAME
value: custom-del
- name: DELEGATE_TYPE
value: "KUBERNETES"
- name: DELEGATE_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: INIT_SCRIPT
value: ""
- name: DELEGATE_DESCRIPTION
value: ""
- name: DELEGATE_TAGS
value: ""
- name: NEXT_GEN
value: "true"
- name: CLIENT_TOOLS_DOWNLOAD_DISABLED
value: "true"
- name: LOG_STREAMING_SERVICE_URL
value: "https://app.harness.io/gratis/log-service/"
---
apiVersion: v1
kind: Service
metadata:
name: delegate-service
namespace: harness-delegate-ng
spec:
type: ClusterIP
selector:
harness.io/name: custom-del
ports:
- port: 8080
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: upgrader-cronjob
namespace: harness-delegate-ng
rules:
- apiGroups: ["batch", "apps", "extensions"]
resources: ["cronjobs"]
verbs: ["get", "list", "watch", "update", "patch"]
- apiGroups: ["extensions", "apps"]
resources: ["deployments"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: custom-del-upgrader-cronjob
namespace: harness-delegate-ng
subjects:
- kind: ServiceAccount
name: upgrader-cronjob-sa
namespace: harness-delegate-ng
roleRef:
kind: Role
name: upgrader-cronjob
apiGroup: ""
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: upgrader-cronjob-sa
namespace: harness-delegate-ng
---
apiVersion: v1
kind: Secret
metadata:
name: custom-del-upgrader-token
namespace: harness-delegate-ng
type: Opaque
data:
UPGRADER_TOKEN: "YOUR_DELEGATE_TOKEN"
---
apiVersion: v1
kind: ConfigMap
metadata:
name: custom-del-upgrader-config
namespace: harness-delegate-ng
data:
config.yaml: |
mode: Delegate
dryRun: false
workloadName: custom-del
namespace: harness-delegate-ng
containerName: delegate
delegateConfig:
accountId: YOUR_ACCOUNT_ID
managerHost: https://app.harness.io/gratis
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
labels:
harness.io/name: custom-del-upgrader-job
name: custom-del-upgrader-job
namespace: harness-delegate-ng
spec:
suspend: true
schedule: "0 */1 * * *"
concurrencyPolicy: Forbid
startingDeadlineSeconds: 20
jobTemplate:
spec:
template:
spec:
serviceAccountName: upgrader-cronjob-sa
restartPolicy: Never
containers:
- image: harness/upgrader:latest
name: upgrader
imagePullPolicy: Always
envFrom:
- secretRef:
name: custom-del-upgrader-token
volumeMounts:
- name: config-volume
mountPath: /etc/config
volumes:
- name: config-volume
configMap:
name: custom-del-upgrader-config
Upgrade the image path
Open the delegate manifest file and locate the container spec
(spec.containers
). Change the image path to reflect the repository location of your uploaded image as shown in the following YAML.
spec:
terminationGracePeriodSeconds: 600
restartPolicy: Always
containers:
- image: example/org:custom-delegate
imagePullPolicy: Always
name: delegate
securityContext:
allowPrivilegeEscalation: false
runAsUser: 0
For purposes of this example, the image was uploaded to example/org:custom-delegate
.
Suspend delegate auto-upgrade
Before you deploy a custom delegate, you must suspend its auto-upgrade functionality. This step prevents your image from being automatically upgraded and the installed binaries removed.
To suspend auto-upgrade, in the delegate manifest, locate the CronJob
resource. In the resource spec
, set the suspend
field to true
as shown in the following YAML:
apiVersion: batch/v1beta1
kind: CronJob
metadata:
labels:
harness.io/name: custom-del-upgrader-job
name: custom-del-upgrader-job
namespace: harness-delegate-ng
spec:
suspend: true
schedule: "0 */1 * * *"
concurrencyPolicy: Forbid
startingDeadlineSeconds: 20
Deploy the delegate
You can deploy the delegate from Harness Manager or by applying the modified delegate manifest file to your cluster.
You can confirm the successful deployment and registration of the delegate in Harness Manager. Check the delegate information to ensure that auto-upgrade is not enabled.
Use your custom delegate image in pipelines
You can use your registered delegate to run Kubernetes and Terraform pipelines. It is a good idea to run a pipeline to validate the delegate image. Harness steps in your pipelines use the installed tooling on the delegate to perform builds or deployments.
For information about creating a Kubernetes pipeline, go to Kubernetes deployment tutorial.
For information about creating a Terraform Plan, go to Provision with the Terraform Apply Step.