Skip to main content

Build and Push to GAR

This topic explains how to configure the Build and Push to GAR step in a Harness CI pipeline. This step is used to build and push to Google Artifact Registry (GAR).

You need:

Kubernetes cluster build infrastructures require root access

With Kubernetes cluster build infrastructures, Build and Push steps use kaniko. Other build infrastructures use drone-docker. Kaniko requires root access to build the Docker image. It doesn't support non-root users.

If your build runs as non-root (runAsNonRoot: true), and you want to run the Build and Push step as root, you can set Run as User to 0 on the Build and Push step to use the root user for that individual step only.

If your security policy doesn't allow running as root, go to Build and push with non-root users.

Add a Build and Push to GAR step

In your pipeline's Build stage, add a Build and Push to GAR step and configure the settings accordingly.

Here is a YAML example of a Build and Push to GAR step.

              - step:
type: BuildAndPushGAR
name: BuildAndPushGAR_1
identifier: BuildAndPushGAR_1
spec:
connectorRef: YOUR_GCP_CONNECTOR_ID
host: LOCATION-docker.pkg.dev
projectID: GOOGLE_CLOUD_CONSOLE_PROJECT_ID
imageName: REPO_NAME/IMAGE_NAME
tags:
- <+pipeline.sequenceId>

When you run a pipeline, you can observe the step logs on the build details page. If the Build and Push to GAR step succeeds, you can find the uploaded image on GAR.

Build and Push to GAR step settings

The Build and Push to GAR step has the following settings. Depending on the stage's build infrastructure, some settings might be unavailable or optional. Settings specific to containers, such as Set Container Resources, are not applicable when using a VM or Harness Cloud build infrastructure.

Name

Enter a name summarizing the step's purpose. Harness automatically assigns an Id (Entity Identifier) based on the Name. You can change the Id.

GCP Connector

The Harness GCP connector to use to connect to GAR. The GCP account associated with the GCP connector needs specific roles. For more information, go to the Google Cloud Platform (GCP) connector settings reference.

This step supports GCP connectors that use access key authentication. It doesn't support GCP connectors that inherit delegate credentials.

tip

If you are using this step with Harness Cloud build infrastructure, you can also leverage the OIDC connectivity mode in your GCP connector.

Host

The Google Artifact Registry hostname, for example LOCATION-docker.pkg.dev. For more information, go to the GAR documentation on Repository and image names.

Project ID

The Google Cloud Console Project ID. For more information, go to the GAR documentation on Repository and image names.

Image Name

The name of the repository where you want to push the artifact and the name you want to give the image, such as REPO_NAME/IMAGE_NAME. For more information, go to the GAR documentation on Repository and image names.

The target repository must be a standard repository.

Tags

Add Docker build tags. This is equivalent to the -t flag. For more information, go to the GAR documentation on Tagging images.

Add each tag separately.

tip

When you push an image to a repo, you tag the image so you can identify it later. For example, in one pipeline stage, you push the image, and, in a later stage, you use the image name and tag to pull it and run integration tests on it.

Harness expressions are a useful way to define tags. For example, you can use the expression <+pipeline.sequenceId> as a tag. This expression represents the incremental build identifier, such as 9. By using a variable expression, rather than a fixed value, you don't have to use the same image name every time.

For example, if you use <+pipeline.sequenceId> as a tag, after the pipeline runs, you can see the Build Id in the output.

And this same number is applied as the tag for your image in GAR.

Later in the pipeline, you can use the same expression to pull the tagged image, such as REPO_NAME/IMAGE_NAME:<+pipeline.sequenceId>.

Optimize

With Kubernetes cluster build infrastructures, select this option to enable --snapshotMode=redo. This setting causes file metadata to be considered when creating snapshots, and it can reduce the time it takes to create snapshots. For more information, go to the kaniko documentation for the snapshotMode flag.

For information about setting other kaniko runtime flags, go to Environment variables.

Dockerfile

The name of the Dockerfile. If you don't provide a name, Harness assumes that the Dockerfile is in the root folder of the codebase.

Context

Enter a path to a directory containing files that make up the build's context. When the pipeline runs, the build process can refer to any files found in the context. For example, a Dockerfile can use a COPY instruction to reference a file in the context.

Labels

Specify Docker object labels to add metadata to the Docker image.

Build Arguments

The Docker build-time variables. This is equivalent to the --build-arg flag.

Target

The Docker target build stage, equivalent to the --target flag, such as build-env.

Remote Cache Image

Use this setting to enable remote Docker layer caching where each Docker layer is uploaded as an image to a Docker repo you identify. If the same layer is used in later builds, Harness downloads the layer from the Docker repo. You can also specify the same Docker repo for multiple Build and Push steps, enabling these steps to share the same remote cache. This can dramatically improve build time by sharing layers across pipelines, stages, and steps.

For Remote Cache Image, enter the name of the remote cache registry and image, such as LOCATION-docker.pkg.dev/PROJECT_ID/REPO_NAME/IMAGE_NAME.

The remote cache repository must be in the same account and organization as the build image. For caching to work, the specified image name must exist.

Environment Variables (plugin runtime flags)

Build and Push steps use plugins to complete build and push operations. With Kubernetes cluster build infrastructures, these steps use kaniko, and, with other build infrastructures, these steps use drone-docker.

These plugins have a number of additional runtime flags that you might need for certain use cases. For information about the flags, go to the kaniko plugin documentation and the drone-docker plugin documentation.

How you configure plugin runtime flags depends on your build infrastructure.

Set plugin runtime flags with Kubernetes cluster build infrastructure

When using the built-in Build and Push steps with a Kubernetes cluster build infrastructure, you can use the Environment Variables setting to set kaniko plugin runtime flags.

warning

Unlike in other Harness CI steps, the Environment Variables setting in Build and Push steps only accepts the known kaniko plugin runtime flags. You must set other types of environment variables in your Dockerfile, build arguments, or as stage variables, depending on their usage and purpose in your build.

In Environment Variables, you must input a Name and Value for each variable. Format the name as PLUGIN_FLAG_NAME.

For example, to set --skip-tls-verify, add an environment variable named PLUGIN_SKIP_TLS_VERIFY and set the variable value to true.

              - step:
identifier: buildandpush
name: buildandpush
type: BuildAndPush---
spec:
...
envVariables:
PLUGIN_SKIP_TLS_VERIFY: true

To build without pushing, use the no-push kaniko flag.

YAML example: Build and Push step with multiple environment variables

This YAML example shows a Build and Push to GAR step with several PLUGIN environment variables.

              - step:
identifier: pushGCR
name: push GCR
type: BuildAndPushGAR ## Type depends the selected Build and Push step, such as Docker, GAR, ACR, and so on.
spec: ## Some parts of 'step.spec' vary by Build and Push step type (Docker, GAR, ACR, etc).
connectorRef: GCR_CONNECTOR
host: "us.gcr.io"
projectID: "some-gcp-project"
imageName: "some-image-name"
tags:
- "1.0"
- "1.2"
buildArgs:
foo: bar
hello: world
labels:
foo: bar
hello: world
target: dev-env
context: "."
dockerfile: "harnessDockerfile"
remoteCacheImage: "test/cache"
envVariables: ## Specify plugin runtime flags as environment variables under 'step.spec'.
PLUGIN_TAR_PATH: ./harnesstarpath
PLUGIN_IMAGE_DOWNLOAD_RETRY: "2"
PLUGIN_COMPRESSED_CACHING: "false"
PLUGIN_USE_NEW_RUN: "true"
PLUGIN_GARBAGE: yoyo
Stage variables

Previously, you could set some kaniko runtime flags as stage variables. If you had done this and you are using Kubernetes cluster build infrastructure, then Harness recommends moving these kaniko plugin stage variables to the Environment Variables in your Build and Push step. Don't change non-kaniko plugin variables, such as PLUGIN_USER_ROLE_ARN.

For other types of environment variables (that aren't Build and Push plugin runtime flags), stage variables are still inherently available to steps as environment variables. However, where you declare environment variables depends on their usage and purpose in your build. You might need to set them in your Dockerfile, build args, or otherwise.

Set plugin runtime flags with other build infrastructures

With Harness Cloud, self-managed VM, or local runner build infrastructures, you can set some drone-docker plugin runtime flags as stage variable.

Currently, Harness supports the following drone-docker flags:

  • auto_tag: Enable auto-generated build tags.
  • auto_tag_suffix: Auto-generated build tag suffix.
  • custom_labels: Additional arbitrary key-value labels.
  • artifact_file: Harness uses this to show links to uploaded artifacts on the Artifacts tab.
  • dry_run: Disables pushing to the registry. Used to build without pushing.
  • custom_dns: Provide your custom CNS address.

To set these flags in your Build and Push steps, add stage variables formatted as PLUGIN_FLAG_NAME.

For example, to set custom_dns, add a stage variable named PLUGIN_CUSTOM_DNS and set the variable value to your custom DNS address.

        variables:
- name: PLUGIN_CUSTOM_DNS
type: String
description: ""
required: false
value: "vvv.xxx.yyy.zzz"

Run as User

With Kubernetes cluster build infrastructures, you can specify the user ID to use to run all processes in the pod if running in containers. For more information, go to Set the security context for a pod.

This step requires root access. You can use the Run as User setting if your build runs as non-root (runAsNonRoot: true), and you can run the Build and Push step as root. To do this, set Run as User to 0 to use the root user for this individual step only.

If your security policy doesn't allow running as root, go to Build and push with non-root users.

Set container resources

Set maximum resource limits for the resources used by the container at runtime:

  • Limit Memory: The maximum memory that the container can use. You can express memory as a plain integer or as a fixed-point number using the suffixes G or M. You can also use the power-of-two equivalents Gi and Mi. The default is 500Mi.
  • Limit CPU: The maximum number of cores that the container can use. CPU limits are measured in CPU units. Fractional requests are allowed; for example, you can specify one hundred millicpu as 0.1 or 100m. The default is 400m. For more information, go to Resource units in Kubernetes.

Timeout

Set the timeout limit for the step. Once the timeout limit is reached, the step fails and pipeline execution continues. To set skip conditions or failure handling for steps, go to:

Conditions, looping, and failure strategies

You can find the following settings on the Advanced tab in the step settings pane:

Troubleshoot Build and Push steps

Go to the CI Knowledge Base for questions and issues related to building and pushing images, such as: