Ctrlk
Sign UpGet a DemoContact Us
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Unified CI/CD GAR GKE pipeline

Build a unified CI/CD pipeline using Google Artifact Registry and Google Kubernetes Engine.

What you will learn from this topic

In this tutorial, you explore how to build a streamlined CI/CD pipeline using the Harness Platform, integrating the robust services of Google Artifact Registry (GAR) and Google Kubernetes Engine (GKE). GAR excels in managing and storing container images securely, while GKE offers a scalable environment for container deployment. The Harness Platform serves as a powerful orchestrator, simplifying the build and push process to GAR and managing complex deployments in GKE. The tutorial also covers implementing approval steps for enhanced security and setting up Slack notifications for real-time updates, showing how these tools together support a robust, streamlined CI/CD process.

Architectural Diagram
Pipeline Editor

Before you begin

Before you begin this tutorial, make sure you have the following:

  • A Harness free plan. If you do not have one, sign up for free.

  • A GitHub account.

  • A Docker Hub account.

  • A GCP account with permissions for Google Artifact Registry and Kubernetes Engine.

  • Access to a Slack workspace and permissions to create a Slack app.

This tutorial includes a bonus section where you run security tests during the build process and create a policy for the deployment process. To follow this section, you need a Harness enterprise account.

Required setup and configurations

Demo application

You can either fork Captain Canary Adventure (CCA) App or bring your own application, as long as it has a Dockerfile.

This tutorial assumes the use of a fork of the CCA App. If you are using your own app, make the necessary changes.

GitHub and Docker authentication

Create a GitHub personal access token (PAT) that has read access to the demo application repository. Create a Docker access token.

Image registry setup on Google Cloud Platform (GCP)

To set up the image registry on GCP, complete the following steps:

  1. Enable the Artifact Registry API.

  2. From Artifact Registry, select + CREATE REPOSITORY.

  3. Give this repository a name, cca-registry, choose Docker as the format, Standard as the mode, location type Region (choose a region near you), Google-managed encryption key for encryption, have Dry Run selected, and select CREATE.

Kubernetes cluster setup with GKE

To set up the Kubernetes cluster, complete the following steps:

  1. Enable Kubernetes Engine API.

  2. Create a GKE (autopilot) cluster by selecting a region near you.

GCP IAM and service account setup

To set up IAM and the service account, complete the following steps:

  1. Create a GCP Service Account. Copy the email address generated for this service account. It is in this format: SERVICE_ACCOUNT_NAME@GCP_PROJECT_NAME.iam.gserviceaccount.com.

  2. Navigate to the artifact registry repository you created earlier, select it, and select the Permissions tab. You need to create two types of access to the artifact registry repository: a public read access and a fine-grained write access. Select + ADD PRINCIPAL from the Permissions tab and paste the email address of the service account previously copied. Assign the Artifact Registry Writer role to this principal. Next, select + ADD PRINCIPAL from the Permissions tab and enter allUsers for the principal and Artifact Registry Reader for the role. You might see a warning like this: This resource is public and can be accessed by anyone on the internet.

  3. Navigate to IAM & Admin, locate the service account, select it, and then select ADD KEY -> Create new key. Choose the JSON format, and a key for your service account downloads to your computer. Exercise caution and do not share this key with anyone; treat it as you would a password.

Slack workspace and app setup

To create a Slack app and incoming webhook, you need elevated privilege in that Slack workspace. Create a new Slack workspace for this tutorial and an incoming webhook for a specific channel.

Your newly created Slack webhook looks like this: https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX.

Treat this as sensitive information.

Harness entity setup

To set up the required Harness entities, complete the following steps:

  1. Create secrets:

    a. GitHub Secret: Navigate to the Harness console. From Project Setup -> Secrets, select + New Secret -> Text, give the secret a name (for example, cca-git-pat) and paste in the previously created GitHub PAT.

    b. Docker Secret: Similarly, create a Docker secret (you can name it docker-secret) and use the previously created Docker PAT as the secret value.

    c. Slack Webhook: Similarly, create a Slack webhook secret (you can name it slack-webhook) and paste in the previously created Slack webhook value.

  2. Connectors in Harness help you pull in artifacts, sync with repos, integrate verification and analytics tools, and use collaboration channels. From Project Setup -> Connectors -> + New Connector, create the following connectors:

    a. GitHub Connector: The Harness platform connects to the source code repository using this connector. Give this connector a name (for example, cca-git-connector), choose URL type as Repository, connection type as HTTP, and paste in the forked GitHub repository URL of the demo app. Use your GitHub username and the previously created GitHub secret for authentication. Select the connectivity mode as Connect through Harness Platform. The connection test should be successful.

    b. Docker Connector: The Harness platform pulls in the Docker image for the Slack notification using this connector. Give this connector a name (for example, docker-connector), choose provider type as DockerHub, Docker Registry URL as https://index.docker.io/v2/, enter your Docker username, and select the previously created Docker secret. Select the connectivity mode as Connect through Harness Platform. The connection test should be successful.

    c. Kubernetes Connector: The Harness platform creates and manages resources on your GKE cluster using this connector. Give this connector a name (for example, gke-connector). Choose Use the credentials of a specific Harness Delegate... and select + Install new Delegate. The Harness Delegate is a service you run in your local network or VPC to connect all of your providers with your Harness account. Follow the instructions to install a delegate on your Kubernetes cluster, and once the installation is complete, select the newly created delegate from the dropdown. The connection test should be successful.

    d. GCP Connector: The GCP connector allows you to connect to your Google Cloud Platform resource and perform actions through the Harness platform. Give this connector a name and select Specify credentials here under the Details section. Add a new secret name and upload the GCP service account key JSON file you previously downloaded. Select the connectivity mode as Connect through Harness Platform. The connection test should be successful.

    GCP Connector Configuration

Build and push image to GAR

First, create the build and push image part of the pipeline. Select Pipelines -> + Create a Pipeline and give it a name (for example, gar-gke-cicd-pipeline). Select the Inline option to store the pipeline definition in Harness, and then select Start.

Select Add Stage and choose Build as the stage type. A Harness pipeline can consist of one or more stages. Give this stage a name (for example, Push to GAR), select the Clone Codebase option (this should be enabled by default), and choose the GitHub connector you previously created from the dropdown. The repository name should auto-populate. Select Set Up Stage.

Under Infrastructure, specify where you want to deploy your application. Select Cloud for Harness hosted builds and choose Linux/AMD64 for the Platform option.

Under Execution, select Add Step -> Add Step and find the Build and Push to GAR step from the Step Library. Name this step (for example, BuildAndPushToGAR) and select the GCP connector you created earlier from the dropdown. When choosing the host, use the region selected when creating the image registry repository (for example, northamerica-northeast1-docker.pkg.dev). For more information, see Build and push to GAR or the GCP Artifact Registry docs for more details on selecting the region for GAR. Enter your GCP project ID under Project Id. For Image Name, use the image registry repository name followed by the application name in this format: cca-registry/cca-app. Use latest for now as the Tags. Select Apply Changes.

Now, select Run to execute the pipeline. Enter Master as the git branch name for the build (or main if you are not using the forked CCA app). A successful execution of the pipeline should resemble the following:

Successful Build and Push Pipeline Execution

Deploy to GKE

If you are using a fork of the Captain Canary Adventure App, update the deployment.yaml before deploying the application to Kubernetes. The current YAML uses Harness variables, but because you are not there yet, you need to hardcode some values for now.

Assuming your GKE Artifact Registry repository name is cca-registry and the image name is cca-app, replace the current values in values.yaml with the following:

Select + Add Stage in the pipeline and choose Deploy as the stage type. Give the stage a name (for example, GKE Deploy), select Kubernetes as the deployment type, and select Set Up Stage.

Next, create a service to deploy. Choose Kubernetes as the deployment type, select the GitHub connector, and specify the paths for the manifests. For example, for the Captain Canary Adventure App, here are the manifest details:

Captain Canary K8s Manifest Details

Environments represent your deployment targets, such as QA or Prod. Each environment contains one or more Infrastructure Definitions that list your target clusters, hosts, and namespaces. Select + New Environment, give this environment a name (for example, cca-env), select Pre-Production as the environment type, and select Save.

Next, create an infrastructure definition. Select + New Infrastructure, under cluster details, select the GKE connector, provide a Kubernetes namespace where your application is deployed (for example, cca-ns), and select Save. For execution strategies, choose Rolling Deployment and select Use Strategy. Under optional configuration, select Enable Kubernetes Pruning. With this setting, Harness uses pruning to remove any resources present in an old manifest but no longer in the manifest used for the current deployment. For more information on this configuration, go to Prune Kubernetes resources.

You are all set. Select Save and then Run. Use master for the git branch. A successful pipeline execution overrides the cca-app:latest image on your GAR repository and deploys this image to your GKE cluster.

Add approval and Slack notifications

In practical DevOps pipelines, gates are implemented to control artifact promotion to the production environment. Harness supports various types of approvals in Continuous Delivery (CD) pipelines. In this tutorial, you use the manual approval step.

Within the gke-deploy stage, select + Add Step before the Rollout Deployment step and find Harness Approval under Approval in the Step Library. Keep all default options, and select the approver from the User Groups. Choose Project -> All Project Users under user group selection. If you are the only member of this project, you are the sole approver. Select Apply Selected. Select Apply Changes for the manual approval step, and then select Save.

Now, add a notification stage so that whenever a deployment is approved in the CI/CD pipeline, a notification is sent to a Slack channel, indicating who approved it.

From the pipeline, select Add Stage after the gke-deploy stage, select Build as the stage type, give this stage a name (for example, Notifications Stage), disable the Clone Codebase option, and select Set Up Stage. Under Infrastructure, choose Use a New Infrastructure -> Cloud and Linux -> AMD64 for the Operating System. Under Execution, select Add Step -> Add Step and find Plugin in the Build section of the Step Library.

Name this step (for example, Slack Notification), choose the Docker connector you previously created under Container Registry, for the image, use plugins/slack, and add the following key-values under Optional Configuration -> Settings (assuming the ID for your Slack webhook secret is slackwebhook).

Key
Value

webhook

<+secrets.getValue("slackwebhook")>

template

The deployment is moved to prod by <+approval.approvalActivities[0].user.name>

The template uses a Harness variable expression to retrieve the name of the approver from the previous stage.

Before running the pipeline, one more update is needed. Currently, every image built, pushed, and deployed has the same image tag, making it challenging to track based on the build number. Harness provides powerful built-in and custom variable expressions for various practical use cases.

Select Variables for your pipeline and select + Add Variable at the pipeline level. Add the following two variables:

Variable Name
Variable Value

imageName

cca-registry/cca-app

imageTag

<+pipeline.sequenceId>

For the imageTag, select the thumbtack icon and select Expression. Every time you run the pipeline, the pipeline sequence ID changes, and subsequently, the image that is built and deployed also changes.

Now that you have updated the pipeline to pass in the imageName and imageTag as variables, update the codebase to replace the hardcoded values. Revert the changes to deployment.yaml and values.yaml you previously made. The values.yaml file receives the imageName and imageTag during pipeline runtime, and then the deployment.yaml file uses those values from the values.yaml file.

Now, select Save and then Run. After a successful gar-build-and-push stage, you should see an image in the GAR repository with a numeric tag that matches the pipeline sequence ID. Right after, you should see the following prompt for approval:

Harness Approval

You can optionally add a comment and select Approve. The pipeline continues as before and you see a deployment on your Kubernetes cluster. However, this time, you see a Slack notification resulting from your approval. To modify the text that appears on the notification, you can modify the template value in the Slack plugin step settings.

Security tests and policy enforcement (bonus section)

These features are only available on Harness paid plans.

Run OWASP tests

You can scan your code repositories using OWASP Dependency-Check within a Harness pipeline. Within the gar-build-and-push stage, select + Add Step -> Add Step before the BuildAndPushGAR step. From the step library, find Owasp under the Security Tests section.

Use the following settings to configure the OWASP Dependency Check and select Apply Changes:

Setting Name
Value

Name

Owasp Tests

Scan Mode

Orchestration

Target.Name

cca-owasp-tests

Variant

master (this is the branch name for the repo)

Log Level

Info

Fail On Severity

Critical

You can have any string values for the step name and target.name. For the Variant, use the branch name for your codebase (for example, master or main). Selecting Critical for Fail On Severity means that if there is any critical error, this test fails and the pipeline execution halts. For more information on these configurations, go to OWASP scanner reference.

Select Save and then Run. If your codebase does not have an OWASP critical bug, the pipeline should execute successfully. To enforce a fail on this OWASP scan, use a codebase with known vulnerabilities like WebGoat to see the OWASP scanner in action.

Add a policy to mandate approval step on deployment stages

Harness Policy as Code uses Open Policy Agent (OPA) as the central service to store and enforce policies for the different entities and processes across the Harness platform. In this section, you define a policy that denies a pipeline execution if there is no approval step defined in a deployment stage.

From Project Setup -> Policies, follow the wizard to create a policy from the policy library. Use the Pipeline - Approval policy.

Pipeline Approval Policy

In the next screen, choose Project scope, trigger event On Run, and for the severity, choose Error & Exit. Next, select Yes to apply the policy.

Now, remove the approval step from the gke-deploy stage. Select Edit on the pipeline and select the cross button on the Harness Approval step. Select Save and then Run. You should see the following error:

Policy Enforcement In Action

Add the Harness Approval back, save and run the pipeline, and this time the pipeline should execute successfully. An end-to-end successful pipeline execution looks like this:

End to end pipeline execution

View the running application

While connected to your GKE cluster, execute the following command:

This assumes that you deployed the application to the cca-ns namespace.

The output is something like this:

Navigate to the IP address listed under the EXTERNAL-IP column for your case, and you should see a running Captain Canary Adventure application. Because the application runs on port 80, you can omit the port number from the URL.

Captain Canary Application Running

Homework task

If you want to take this pipeline one step further, you can use caching to share data across stages because each stage in a Harness CI pipeline has its own build infrastructure. See how to save and restore cache from Google Cloud Storage (GCS).

Next steps

PreviousSecure Container Image Signing with Cosign and OPA
NextAWS Deployments

Last updated

Was this helpful?