> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/continuous-delivery/troubleshooting-and-resources/tutorials/cloud-native-cicd-pipelines/gar-gke-pipeline.md).
# Unified CI/CD GAR GKE pipeline
Build a unified CI/CD pipeline using Google Artifact Registry and Google Kubernetes Engine.
## What you will learn from this topic
* How to [build and push an image to Google Artifact Registry (GAR)](#build-and-push-image-to-gar)
* How to [deploy the image to Google Kubernetes Engine (GKE)](#deploy-to-gke)
* How to [add an approval step and Slack notifications](#add-approval-and-slack-notifications) to the pipeline
* How to follow the bonus section to [run security tests and enforce a policy](#security-tests-and-policy-enforcement-bonus-section) on the pipeline
In this tutorial, you explore how to build a streamlined CI/CD pipeline using the Harness Platform, integrating the robust services of Google Artifact Registry (GAR) and Google Kubernetes Engine (GKE). GAR excels in managing and storing container images securely, while GKE offers a scalable environment for container deployment. The Harness Platform serves as a powerful orchestrator, simplifying the build and push process to GAR and managing complex deployments in GKE. The tutorial also covers implementing approval steps for enhanced security and setting up Slack notifications for real-time updates, showing how these tools together support a robust, streamlined CI/CD process.
## Before you begin
Before you begin this tutorial, make sure you have the following:
* A Harness free plan. If you do not have one, [sign up for free](https://app.harness.io/auth/#/signup/?\&utm_campaign=cd-devrel).
* A GitHub account.
* A Docker Hub account.
* A GCP account with permissions for Google Artifact Registry and Kubernetes Engine.
* Access to a Slack workspace and permissions to create a Slack app.
This tutorial includes a bonus section where you run security tests during the build process and create a policy for the deployment process. To follow this section, you need a Harness enterprise account.
## Required setup and configurations
### Demo application
You can either [fork Captain Canary Adventure (CCA) App](https://github.com/harness-community/captain-canary-adventure-app/fork) or bring your own application, as long as it has a Dockerfile.
{% hint style="info" %}
This tutorial assumes the use of a fork of the CCA App. If you are using your own app, make the necessary changes.
{% endhint %}
### GitHub and Docker authentication
Create a [GitHub personal access token (PAT)](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) that has read access to the demo application repository. Create a [Docker access token](https://docs.docker.com/security/for-developers/access-tokens/).
### Image registry setup on Google Cloud Platform (GCP)
To set up the image registry on GCP, complete the following steps:
1. [Enable the Artifact Registry API](https://console.cloud.google.com/apis/api/artifactregistry.googleapis.com).
2. From [Artifact Registry](https://console.cloud.google.com/artifacts), select **+ CREATE REPOSITORY**.
3. Give this repository a name, `cca-registry`, choose `Docker` as the format, `Standard` as the mode, location type `Region` (choose a region near you), `Google-managed encryption` key for encryption, have `Dry Run` selected, and select **CREATE**.
### Kubernetes cluster setup with GKE
To set up the Kubernetes cluster, complete the following steps:
1. [Enable Kubernetes Engine API](https://console.cloud.google.com/apis/api/container.googleapis.com).
2. [Create a GKE (autopilot) cluster](https://console.cloud.google.com/kubernetes/auto/add) by selecting a region near you.
### GCP IAM and service account setup
To set up IAM and the service account, complete the following steps:
1. [Create a GCP Service Account](https://console.cloud.google.com/iam-admin/serviceaccounts). Copy the email address generated for this service account. It is in this format: `SERVICE_ACCOUNT_NAME@GCP_PROJECT_NAME.iam.gserviceaccount.com`.
2. Navigate to the artifact registry repository you created earlier, select it, and select the **Permissions** tab. You need to create two types of access to the artifact registry repository: a public read access and a fine-grained write access. Select **+ ADD PRINCIPAL** from the **Permissions** tab and paste the email address of the service account previously copied. Assign the `Artifact Registry Writer` role to this principal. Next, select **+ ADD PRINCIPAL** from the **Permissions** tab and enter `allUsers` for the principal and `Artifact Registry Reader` for the role. You might see a warning like this: `This resource is public and can be accessed by anyone on the internet.`
3. Navigate to **IAM & Admin**, locate the service account, select it, and then select **ADD KEY** -> **Create new key**. Choose the JSON format, and a key for your service account downloads to your computer. Exercise caution and do not share this key with anyone; treat it as you would a password.
### Slack workspace and app setup
To create a Slack app and incoming webhook, you need elevated privilege in that Slack workspace. [Create a new Slack workspace](https://slack.com/help/articles/206845317-Create-a-Slack-workspace) for this tutorial and [an incoming webhook](https://api.slack.com/messaging/webhooks) for a specific channel.
Your newly created Slack webhook looks like this: `https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX`.
*Treat this as sensitive information*.
### Harness entity setup
To set up the required Harness entities, complete the following steps:
1. Create secrets:
a. GitHub Secret: Navigate to the [Harness console](https://app.harness.io/). From **Project Setup** -> **Secrets**, select **+ New Secret** -> **Text**, give the secret a name (for example, `cca-git-pat`) and paste in the previously created GitHub PAT.
b. Docker Secret: Similarly, create a Docker secret (you can name it `docker-secret`) and use the previously created Docker PAT as the secret value.
c. Slack Webhook: Similarly, create a Slack webhook secret (you can name it `slack-webhook`) and paste in the previously created Slack webhook value.
2. [Connectors](https://developer.harness.io/harness-platform/use-harness-platform/connectors) in Harness help you pull in artifacts, sync with repos, integrate verification and analytics tools, and use collaboration channels. From **Project Setup** -> **Connectors** -> **+ New Connector**, create the following connectors:
a. GitHub Connector: The Harness platform connects to the source code repository using this connector. Give this connector a name (for example, `cca-git-connector`), choose URL type as **Repository**, connection type as **HTTP**, and paste in the forked GitHub repository URL of the demo app. Use your GitHub username and the previously created GitHub secret for authentication. Select the connectivity mode as **Connect through Harness Platform**. The connection test should be successful.
b. Docker Connector: The Harness platform pulls in the Docker image for the Slack notification using this connector. Give this connector a name (for example, `docker-connector`), choose provider type as **DockerHub**, Docker Registry URL as `https://index.docker.io/v2/`, enter your Docker username, and select the previously created Docker secret. Select the connectivity mode as **Connect through Harness Platform**. The connection test should be successful.
c. Kubernetes Connector: The Harness platform creates and manages resources on your GKE cluster using this connector. Give this connector a name (for example, `gke-connector`). Choose **Use the credentials of a specific Harness Delegate...** and select **+ Install new Delegate**. The Harness Delegate is a service you run in your local network or VPC to connect all of your providers with your Harness account. Follow the instructions to install a delegate on your Kubernetes cluster, and once the installation is complete, select the newly created delegate from the dropdown. The connection test should be successful.
d. GCP Connector: The GCP connector allows you to connect to your Google Cloud Platform resource and perform actions through the Harness platform. Give this connector a name and select **Specify credentials here** under the **Details** section. Add a new secret name and upload the GCP service account key JSON file you previously downloaded. Select the connectivity mode as **Connect through Harness Platform**. The connection test should be successful.
## Build and push image to GAR
First, create the build and push image part of the pipeline. Select **Pipelines** -> **+ Create a Pipeline** and give it a name (for example, `gar-gke-cicd-pipeline`). Select the **Inline** option to store the pipeline definition in Harness, and then select **Start**.
Select **Add Stage** and choose **Build** as the stage type. A Harness pipeline can consist of one or more stages. Give this stage a name (for example, `Push to GAR`), select the **Clone Codebase** option (this should be enabled by default), and choose the GitHub connector you previously created from the dropdown. The repository name should auto-populate. Select **Set Up Stage**.
Under **Infrastructure**, specify where you want to deploy your application. Select **Cloud** for Harness hosted builds and choose **Linux/AMD64** for the Platform option.
Under **Execution**, select **Add Step** -> **Add Step** and find the **Build and Push to GAR** step from the Step Library. Name this step (for example, `BuildAndPushToGAR`) and select the GCP connector you created earlier from the dropdown. When choosing the host, use the region selected when creating the image registry repository (for example, `northamerica-northeast1-docker.pkg.dev`). For more information, see [Build and push to GAR](https://developer.harness.io/continuous-integration/use-harness-ci/use-harness-ci/build-and-upload-artifacts/build-and-push/build-and-push-to-gar#host) or the [GCP Artifact Registry docs](https://cloud.google.com/artifact-registry/docs/repositories/repo-locations) for more details on selecting the region for GAR. Enter your GCP project ID under **Project Id**. For **Image Name**, use the image registry repository name followed by the application name in this format: `cca-registry/cca-app`. Use `latest` for now as the **Tags**. Select **Apply Changes**.
Now, select **Run** to execute the pipeline. Enter `Master` as the git branch name for the build (or `main` if you are not using the forked CCA app). A successful execution of the pipeline should resemble the following:
## Deploy to GKE
If you are using a fork of the Captain Canary Adventure App, update the `deployment.yaml` before deploying the application to Kubernetes. The current YAML uses Harness variables, but because you are not there yet, you need to hardcode some values for now.
Assuming your GKE Artifact Registry repository name is `cca-registry` and the image name is `cca-app`, replace the current values in `values.yaml` with the following:
```yaml
image:
name: cca-registry/cca-app
tag: latest
```
Select **+ Add Stage** in the pipeline and choose **Deploy** as the stage type. Give the stage a name (for example, `GKE Deploy`), select **Kubernetes** as the deployment type, and select **Set Up Stage**.
Next, create a service to deploy. Choose **Kubernetes** as the deployment type, select the GitHub connector, and specify the paths for the manifests. For example, for the Captain Canary Adventure App, here are the manifest details:
Environments represent your deployment targets, such as QA or Prod. Each environment contains one or more Infrastructure Definitions that list your target clusters, hosts, and namespaces. Select **+ New Environment**, give this environment a name (for example, `cca-env`), select **Pre-Production** as the environment type, and select **Save**.
Next, create an infrastructure definition. Select **+ New Infrastructure**, under cluster details, select the GKE connector, provide a Kubernetes namespace where your application is deployed (for example, `cca-ns`), and select **Save**. For execution strategies, choose **Rolling Deployment** and select **Use Strategy**. Under optional configuration, select **Enable Kubernetes Pruning**. With this setting, Harness uses pruning to remove any resources present in an old manifest but no longer in the manifest used for the current deployment. For more information on this configuration, go to [Prune Kubernetes resources](/continuous-delivery/use-continuous-delivery/deploy-services-on-different-platforms/kubernetes/cd-kubernetes-category/prune-kubernetes-resources.md).
You are all set. Select **Save** and then **Run**. Use `master` for the git branch. A successful pipeline execution overrides the `cca-app:latest` image on your GAR repository and deploys this image to your GKE cluster.
## Add approval and Slack notifications
In practical DevOps pipelines, gates are implemented to control artifact promotion to the production environment. Harness supports [various types of approvals](https://developer.harness.io/harness-platform/use-harness-platform/approvals/approvals-tutorial) in Continuous Delivery (CD) pipelines. In this tutorial, you use the manual approval step.
Within the **gke-deploy** stage, select **+ Add Step** before the **Rollout Deployment** step and find **Harness Approval** under Approval in the Step Library. Keep all default options, and select the approver from the User Groups. Choose **Project** -> **All Project Users** under user group selection. If you are the only member of this project, you are the sole approver. Select **Apply Selected**. Select **Apply Changes** for the manual approval step, and then select **Save**.
Now, add a notification stage so that whenever a deployment is approved in the CI/CD pipeline, a notification is sent to a Slack channel, indicating who approved it.
From the pipeline, select **Add Stage** after the gke-deploy stage, select **Build** as the stage type, give this stage a name (for example, `Notifications Stage`), disable the Clone Codebase option, and select **Set Up Stage**. Under Infrastructure, choose **Use a New Infrastructure** -> **Cloud** and **Linux -> AMD64** for the Operating System. Under Execution, select **Add Step** -> **Add Step** and find **Plugin** in the Build section of the Step Library.
Name this step (for example, `Slack Notification`), choose the Docker connector you previously created under Container Registry, for the image, use `plugins/slack`, and add the following key-values under **Optional Configuration** -> **Settings** (assuming the ID for your Slack webhook secret is `slackwebhook`).
| Key | Value |
| -------- | -------------------------------------------------------------------------------- |
| webhook | `<+secrets.getValue("slackwebhook")>` |
| template | The deployment is moved to prod by `<+approval.approvalActivities[0].user.name>` |
The template uses a Harness variable expression to retrieve the name of the approver from the previous stage.
Before running the pipeline, one more update is needed. Currently, every image built, pushed, and deployed has the same image tag, making it challenging to track based on the build number. Harness provides powerful [built-in and custom variable expressions](https://developer.harness.io/harness-platform/use-harness-platform/variables-and-expressions/harness-variables) for various practical use cases.
Select **Variables** for your pipeline and select **+ Add Variable** at the pipeline level. Add the following two variables:
| Variable Name | Variable Value |
| ------------- | ------------------------ |
| imageName | cca-registry/cca-app |
| imageTag | `<+pipeline.sequenceId>` |
For the **imageTag**, select the thumbtack icon and select **Expression**. Every time you run the pipeline, the pipeline sequence ID changes, and subsequently, the image that is built and deployed also changes.
Now that you have updated the pipeline to pass in the **imageName** and **imageTag** as variables, update the codebase to replace the hardcoded values. Revert the changes to `deployment.yaml` and `values.yaml` you previously made. The `values.yaml` file receives the **imageName** and **imageTag** during pipeline runtime, and then the `deployment.yaml` file uses those values from the `values.yaml` file.
Now, select **Save** and then **Run**. After a successful **gar-build-and-push** stage, you should see an image in the GAR repository with a numeric tag that matches the pipeline sequence ID. Right after, you should see the following prompt for approval:
You can optionally add a comment and select **Approve**. The pipeline continues as before and you see a deployment on your Kubernetes cluster. However, this time, you see a Slack notification resulting from your approval. To modify the text that appears on the notification, you can modify the **template** value in the Slack plugin step settings.
## Security tests and policy enforcement (bonus section)
{% hint style="info" %}
These features are only available on Harness paid plans.
{% endhint %}
### Run OWASP tests
You can scan your code repositories using [OWASP Dependency-Check](https://owasp.org/www-project-dependency-check/) within a Harness pipeline. Within the `gar-build-and-push` stage, select **+ Add Step** -> **Add Step** before the `BuildAndPushGAR` step. From the step library, find **Owasp** under the Security Tests section.
Use the following settings to configure the OWASP Dependency Check and select **Apply Changes**:
| Setting Name | Value |
| ---------------- | --------------------------------------------- |
| Name | Owasp Tests |
| Scan Mode | Orchestration |
| Target.Name | cca-owasp-tests |
| Variant | master (this is the branch name for the repo) |
| Log Level | Info |
| Fail On Severity | Critical |
You can have any string values for the step name and target.name. For the **Variant**, use the branch name for your codebase (for example, `master` or `main`). Selecting **Critical** for **Fail On Severity** means that if there is any critical error, this test fails and the pipeline execution halts. For more information on these configurations, go to [OWASP scanner reference](https://developer.harness.io/security-testing-orchestration/use-sto/sto-scanner-configuration/owasp-scanner-reference).
Select **Save** and then **Run**. If your codebase does not have an OWASP critical bug, the pipeline should execute successfully. To enforce a fail on this OWASP scan, use a codebase with known vulnerabilities like [WebGoat](https://github.com/WebGoat/WebGoat) to see the OWASP scanner in action.
### Add a policy to mandate approval step on deployment stages
Harness Policy as Code uses [Open Policy Agent (OPA)](https://www.openpolicyagent.org/) as the central service to store and enforce policies for the different entities and processes across the Harness platform. In this section, you define a policy that denies a pipeline execution if there is no approval step defined in a deployment stage.
From **Project Setup** -> **Policies**, follow the wizard to create a policy from the policy library. Use the **Pipeline - Approval** policy.
In the next screen, choose **Project** scope, trigger event **On Run**, and for the severity, choose **Error & Exit**. Next, select **Yes** to apply the policy.
Now, remove the approval step from the gke-deploy stage. Select **Edit** on the pipeline and select the cross button on the Harness Approval step. Select **Save** and then **Run**. You should see the following error:
Add the Harness Approval back, save and run the pipeline, and this time the pipeline should execute successfully. An end-to-end successful pipeline execution looks like this:
## View the running application
While connected to your GKE cluster, execute the following command:
```shell
kubectl get svc -n cca-ns
```
This assumes that you deployed the application to the `cca-ns` namespace.
The output is something like this:
```shell
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
cca-app-service LoadBalancer 34.118.227.33 34.152.47.53 80:30008/TCP 6d19h
```
Navigate to the IP address listed under the EXTERNAL-IP column for your case, and you should see a running Captain Canary Adventure application. Because the application runs on port 80, you can omit the port number from the URL.
## Homework task
If you want to take this pipeline one step further, you can use caching to share data across stages because each stage in a Harness CI pipeline has its own build infrastructure. See how to [save and restore cache from Google Cloud Storage (GCS)](https://developer.harness.io/continuous-integration/use-harness-ci/use-harness-ci/caching-ci-data/save-cache-in-gcs).
***
## Next steps
* [Build and push to GAR](https://developer.harness.io/continuous-integration/use-harness-ci/use-harness-ci/build-and-upload-artifacts/build-and-push/build-and-push-to-gar#host): Learn more about the Build and Push to GAR step.
* [Approvals tutorial](https://developer.harness.io/harness-platform/use-harness-platform/approvals/approvals-tutorial): Review the available approval types for CD pipelines.
* [Prune Kubernetes resources](/continuous-delivery/use-continuous-delivery/deploy-services-on-different-platforms/kubernetes/cd-kubernetes-category/prune-kubernetes-resources.md): Learn how Harness prunes stale Kubernetes resources during deployment.
* [OWASP scanner reference](https://developer.harness.io/security-testing-orchestration/use-sto/sto-scanner-configuration/owasp-scanner-reference): Learn about the OWASP Dependency-Check step configuration.
{% @harness-feedback/feedback %}