> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/continuous-integration/3.0/use-harness-ci/use-harness-ci/set-up-build-infrastructure/k8s-build-infrastructure/tutorial-ci-kubernetes-build-infra.md).
# Tutorial - Build and test on a Kubernetes cluster
Learn how to create a pipeline that uses a Kubernetes cluster build infrastructure.
This tutorial demonstrates how you can use Harness CI to build and test on a [Kubernetes cluster build infrastructure](/continuous-integration/3.0/use-harness-ci/use-harness-ci/set-up-build-infrastructure/k8s-build-infrastructure/set-up-a-kubernetes-cluster-build-infrastructure.md). In this tutorial, you'll create a Harness CI pipeline with two stages that does the following:
1. Clones a sample app code repo.
2. Builds the app code and runs unit tests.
3. Packages the app as a Docker image, and uploads it to a Docker Hub repo.
4. Pulls the uploaded image into the build infrastructure as a dependency.
5. Runs an integration test against the app.
This tutorial uses publicly-available code and images. You need a Harness account, Github account, and a Docker Hub account.
{% hint style="info" %}
All build infrastructure options, including Kubernetes cluster builds, are available across Free, Team, and Enterprise plans.
{% endhint %}
## Prerequisites
This tutorial assumes you have experience with Kubernetes, such as setting up service accounts and clusters.
In addition to a Harness account, you need the following accounts and tools:
* A **GitHub account** where you can fork the tutorial repo.
* A **Docker Hub account and repo** where the pipeline can push and pull app images.
* A **Kubernetes cluster** where you'll install the [Harness Delegate](/harness-platform/3.0/harness-platform-resources/delegates/delegate-closed-beta/delegate-overview.md) and run the build farm. The cluster needs the following minimum specifications:
* Pods: 3 (two for the Delegate and one for the build farm)
* Machine type: 4vCPU
* Memory: 16GB RAM
* Networking: Outbound HTTPS for the Harness and Docker Hub connections. Allow port 22 for SSH.
* Namespace: During the tutorial, when you install the Harness Delegate, the `harness-delegate-ng` namespace is created. You'll use the same namespace for the build infrastructure.
* A **Kubernetes service account** with permission to create entities in the target namespace. The set of permissions should include `list`, `get`, `create`, and `delete` permissions. Usually, the `cluster-admin` permission or `namespace admin` permission is sufficient. For more information, go to the Kubernetes documentation on [User-Facing Roles](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles).
{% hint style="warning" %}
Google Kubernetes Engine (GKE) [Autopilot](https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview) is not recommended. For more information, go to [Set up a Kubernetes cluster build infrastructure](/continuous-integration/3.0/use-harness-ci/use-harness-ci/set-up-build-infrastructure/k8s-build-infrastructure/set-up-a-kubernetes-cluster-build-infrastructure.md).
{% endhint %}
## Prepare the codebase
1. Fork the tutorial repo [harness-community/goHelloWorldServer](https://github.com/harness-community/goHelloWorldServer) to your GitHub account. Alternately, you can use your own code repo. This tutorial works for any Git repo that you can access.
2. Create a GitHub personal access token with the `repo`, `admin:repo_hook`, and `user` scopes. For instructions, go to the GitHub documentation on [creating a personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token). For information about the token's purpose in Harness, go to the **Authentication** section of the [GitHub connector settings reference](/harness-platform/3.0/harness-platform-resources/connectors/code-repositories/ref-source-repo-provider/git-hub-connector-settings-reference.md#authentication).
3. Store the token somewhere secure; you'll need it later in the tutorial.
4. In Harness, switch to the **Project** you want to use for this tutorial, or create a project.
Create a project
Use these steps to create a project in your Harness account.
1. Select **Project**, select **All Projects** or **Manage Projects**, and then select **New Project**.
2. Enter a **Name**, such as `CI kubernetes tutorial`.
3. Leave the **Organization** as **default**.
4. Select **Save and Continue**.
5. On **Invite Collaborators**, you can add others to your project, if desired. You don't need to add yourself.
6. Select **Save and Continue**.
7. On the Modules page, select **Continuous Integration**, and then select **Go to Module**.
If this is your first project with CI, the CI pipeline wizard starts after you select **Go to Module**. You'll need to exit the wizard to create the GitHub connector if you don't already have a GitHub connector for the account where you forked the tutorial repo.
### Create a GitHub connector
Next, you'll create a *connector* that allows Harness to connect to your Git codebase, and you'll install a Harness Delegate in your Kubernetes cluster. A connector is a configurable object that connects to an external resource automatically while the pipeline runs. For more information, go to the [GitHub connector settings reference](/harness-platform/3.0/harness-platform-resources/connectors/code-repositories/ref-source-repo-provider/git-hub-connector-settings-reference.md).
1. Under **Project Settings**, select **Connectors**.
2. Select **New Connector**, and then select **GitHub** under **Code Repositories**.
3. Enter a recognizable name, such as `CI kubernetes tutorial connector`, and select **Continue**.
4. Configure the **Details** as follows, and then select **Continue**:
* **URL Type:** Select **Repository**.
* **Connection Type:** Select **HTTP**.
* **GitHub Repository URL:** Enter the URL to your fork of the tutorial repo, such as `https://github.com/YOUR_ACCOUNT/goHelloWorldServer.git`.
5. Configure the **Credentials** as follows, and then select **Continue**:
* **Username:** Enter the username for the GitHub account where you forked the tutorial repo.
* **Personal Access Token:** Create a secret for the personal access token you created earlier. Harness secrets are safe; they're stored in the [Harness Secret Manager](/harness-platform/3.0/harness-platform-resources/secrets/secrets-management/harness-secret-manager-overview.md). You can also use your own Secret Manager with Harness.
* **Enable API access:** Enable this option, and then select the same personal access token secret.
6. For **Select Connectivity Mode**, select **Connect through a Harness Delegate**, and then select **Continue**.
You can choose to establish connections directly from your Harness platform or through a delegate service running within your corporate firewall. The Harness Delegate is a local service that connects your infrastructure, collaboration, verification, and other providers with your Harness platform. For this tutorial, you'll install a delegate in your Kubernetes cluster.
7. On **Delegates Setup**, select **Only use Delegates with all of the following tags**, and then select **Install new Delegate**.
8. Select **Kubernetes Manifest** and follow the instructions given on the **New Delegate** page to install the Delegate on a pod in your Kubernetes cluster.
You can use a Helm Chart, Terraform, or Kubernetes Manifest to install Kubernetes delegates. For this tutorial, select **Kubernetes Manifest**. For information about the other options and detailed instructions, go to [Delegate installation overview](/harness-platform/3.0/harness-platform-resources/delegates/delegate-closed-beta/install-a-delegate.md).
If delegate installation succeeds, the `kubectl apply` command should produce output similar to the following:
```
namespace/harness-delegate-ng created
clusterrolebinding.rbac.authorization.k8s.io/harness-delegate-ng-cluster-admin created
secret/ci-quickstart created
statefulset.apps/ci-quickstart created
service/delegate-service created
```
9. In Harness, select **Verify** to test the connection. It might take a few minutes to verify the delegate. Once it is verified, exit delegate creation and return to connector setup.
10. Back in the connector's **Delegates Setup**, select your new delegate, and then select **Save and Continue**.
11. Wait while Harness tests the connection, and then select **Finish**.
## Create a pipeline
Pipelines are comprised of one or more stages. Each stage has one or more steps that manage and automate builds, tests, deployments, and other important build and release tasks. To learn more about pipeline components, go to [CI pipeline components](/continuous-integration/3.0/new-to-harness-ci/key-concepts.md).
1. Select **Pipelines**, and then select **Create a Pipeline**.
2. Enter a **Name** for the pipeline. Harness automatically creates a pipeline ID based on the name. Once the pipeline is created, you can't change the ID. You can use the ID to reference subordinate elements of a pipeline, such as the names of variables within the pipeline.
3. Select **Start**. You're taken to the Pipeline Studio where you can configure pipeline settings and add stages and steps to your pipeline.
### Add a Build stage
For most CI pipelines, Build stages do the heavy lifting. Build stages are where you specify the end-to-end workflow for your pipeline: the codebase to build, the build infrastructure to use, where to push the finished artifact, and any additional tasks, such as automated tests or validations.
1. In the Pipeline Studio, select **Add Stage** and select **Build**.
2. For the **Stage Name**, enter `Build Test and Push`.
3. Under **Configure Codebase**, select **Third-party Git Provider**.
4. For **Connector**, select the GitHub connector you created earlier in [Prepare the codebase](#prepare-the-codebase).
5. Select **Set Up Stage**. The Build stage is added to the pipeline.
### Define the build infrastructure
Next, you need to define the build infrastructure. Harness offers several [build infrastructure options](/continuous-integration/3.0/use-harness-ci/use-harness-ci/set-up-build-infrastructure/which-build-infrastructure-is-right-for-me.md), and this tutorial uses a [Kubernetes cluster build infrastructure](/continuous-integration/3.0/use-harness-ci/use-harness-ci/set-up-build-infrastructure/k8s-build-infrastructure/set-up-a-kubernetes-cluster-build-infrastructure.md).
1. Select the **Infrastructure** tab for your Build stage.
2. Under **Infrastructure**, select **Kubernetes**.
3. Under **Platform**, select the **Kubernetes Cluster** field to open the **Create or Select an Existing Connector** window.
4. Select **New Connector**, and configure the connector as follows. For detailed instructions and information about these settings, go to [Add a Kubernetes cluster connector](/harness-platform/3.0/harness-platform-resources/connectors/cloud-providers/add-a-kubernetes-cluster-connector.md).
* **Name:** Enter `ci-delegate`
* **Details:** Select **Use the credentials of a specific Harness Delegate**.
* **Delegates Setup:** Select the Kubernetes Delegate you installed earlier in [Prepare the codebase](#prepare-the-codebase).
* **Connection Test:** Wait for the test to finish and then click **Finish**.
5. In **Namespace**, enter `harness-delegate-ng`, and then select **Continue**.
### Add steps to build, test, and push an image
Now that the pipeline has a stage with a defined codebase and build infrastructure, you are ready to add steps to build the codebase, run unit tests, and push an artifact to Docker Hub. The first step will run unit tests and compile the codebase. The second step builds a container image and pushes it to a Docker Hub repo.
To run unit tests in a CI pipeline, you can use either a [Run step](/continuous-integration/3.0/use-harness-ci/use-harness-ci/run-step-settings.md) or a [Test step](/continuous-integration/3.0/use-harness-ci/use-harness-ci/run-tests/tests-v2.md). This tutorial uses a **Run** step. In addition to unit tests, the **Run** step can run any number of commands on a container image. **Run** steps are highly versatile and you'll use them often in your CI pipelines. While not used in this tutorial, with the **Test** step you can leverage [Test Intelligence](/continuous-integration/3.0/use-harness-ci/use-harness-ci/run-tests/ti-overview.md).
1. On the **Execution** tab for your Build stage, select **Add Step**, select **Add Step** again, and then select the **Run** step from the Step Library.
2. For **Name**, enter `Run Unit Tests`.
3. Select the **Container Registry** field, select **New Connector**, and then select **Docker Registry**.
4. Create a Docker Registry connector to connect to your Docker Hub account.
* **Name:** Enter a recognizable name for the connector.
* **Provider Type:** Select **Docker Hub**.
* **Docker Registry URL:** Enter `https://index.docker.io/v2/`.
* **Username:** Enter the username for your Docker Hub account.
* **Password:** Create a secret for a Personal Access Token that Harness can use to access your Docker Hub account.
* **Select Connectivity Mode:** Select **Connect through a Harness Delegate**.
* **Delegates Setup:** Select **Only use Delegates with all of the following tags**, and then select the Delegate you installed in your Kubernetes cluster.
* Select **Save and Continue**, wait for the connectivity test to run, and then select **Finish**.
5. Back in the **Run** step settings, enter `golang:1.17` in the **Image** field.
6. Enter the following code in the **Command** field:
```
go get gotest.tools/gotestsum
gotestsum --format=standard-verbose --junitfile unit-tests.xml || true
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -tags netgo
```
The last line contains the `go build` command that compiles the package along with its dependencies.
This **Run** step intentionally fails the tests so that you can see how Harness records test failures.
7. Under **Optional Configuration**, add a **Report Path** and enter `*.xml`.
8. Select **Apply Changes** to save the step.
9. Add a [Build and Push to Docker step](/continuous-integration/3.0/use-harness-ci/use-harness-ci/build-and-upload-artifacts/build-and-push/build-and-push-to-docker-registry.md) to your Build stage, and configure it as follows:
* **Name:** Enter a name, such as `Build and push to Docker`.
* **Docker Connector:** Select the Docker Hub connector you created for the **Run** step.
* **Docker Repository:** Enter your Docker Hub username and the destination repo name formatted as `[docker_username]/[repo_name]`. For example: `mydockerhub/ci_tutorial_repo`.
* **Tags:** Add a tag and enter `<+pipeline.sequenceId>`.
10. Select **Apply Changes** to save the step, and then select **Save** to save the pipeline.
{% hint style="info" %}
The tag `<+pipeline.sequenceId>` is a built-in Harness variable that represents the Build ID number, for example `9`. The pipeline uses the Build ID to tag the image that it pushes in the first stage and pulls in the second stage of this tutorial pipeline. You will see the Build ID when you run the pipeline. You will also use this variable to identify the image location when you set up the dependency (as a **Background** step) in the next stage.
{% endhint %}
You can run the pipeline now or continue the tutorial to add integration tests to the pipeline.
### Add integration tests
The first stage in this tutorial pipeline builds, tests, containerizes, and then pushes an image to Docker Hub. Now, you'll add a stage that pulls the image, runs it in a container (using a **Background** step), and then runs integration tests on it (using a **Run** step).
1. Select **Add Stage** (in the upper portion of the Pipeline Studio), and select **Build**.
2. Enter `Run integration test` for the **Name**, disable **Clone Codebase**, and then select **Set Up Stage**.
3. In the **Infrastructure** tab, select **Propagate from an existing stage**, and select the first **Build** stage in your pipeline. This configures the pipeline to use the same Kubernetes cluster and namespace for both stages.
4. Select **Continue**.
5. Add a **Background** step to this stage and configure it as follows. For more information, go to the [Background step settings reference](/continuous-integration/3.0/use-harness-ci/use-harness-ci/manage-dependencies/background-step-settings.md).
* **Name:** Enter a recognizable name.
* **Container Registry:** Select the Docker Hub connector you used for the steps in the previous stage.
* **Image:** Enter `[docker_username]/[repo_name]:<+pipeline.sequenceId>`. Make sure the Docker Hub username and repo name are the same as you used for the **Build and Push an Image to Docker Registry** step. For example: `mydockerhub/ci_tutorial_repo:<+pipeline.sequenceId>`.
Notice that the **Image** field uses the same variable `<+pipeline.sequenceId>` that you used in the previous stage. This tells Harness to pull the image with the same tag as the image pushed previously in the pipeline.
6. Select **Apply Changes** to save the step.
7. Add a **Run** step to your `Run integration test` stage and configure it as follows:
* **Name:** Enter a recognizable name, such as `Test server connection`.
* **Container Registry:** Select the same Docker Connector as you have for the other steps.
* **Image:** Enter `curlimages/curl:7.73.0`.
* **Command:** Enter the following code, which runs a simple server connection test:
```
sleep 10
curl localhost:8080
curl localhost:8080?Hello!_I_am_a_nice_demo!
```
8. Select **Apply Changes** to save the step, and then select **Save** to save the pipeline.
## Run the pipeline
1. Select **Run**.
2. For the **Pipeline Inputs**, select **Git branch** and enter the target branch in the code repo, such as `main`.
3. Select **Run Pipeline**.
On the [Build details page](/continuous-integration/3.0/use-harness-ci/use-harness-ci/viewing-builds.md) you can observe the pipeline while it runs. Select a stage to examine the steps in that stage. Select a step to view the step's logs. Select the **Tests** tab to [view test results](/continuous-integration/3.0/use-harness-ci/use-harness-ci/run-tests/viewing-tests.md).
You can switch to **Console view** for a focused view of the logs.
For this pipeline, note the following log details:
* In the logs for the **Build and push image to Docker Hub** step, you can see that the build number, such as `11`, is used as an image tag, for example:
```
--destination=myrepo/ciquickstart:11
```
* In the **Initialize** step of the **Run Integration Test** stage, you can see the image with the same tag is pulled from your Docker Hub repo:
```
Pulling image "myrepo/ciquickstart:11"
Successfully pulled image "myrepo/ciquickstart:11" in 1.878887425s
```
* You can find the pushed image and the associated tag in your Docker Hub repo.
## Continue your Continuous Integration journey
Congratulations! You have created a CI pipeline that builds and tests code.
With CI pipelines you can consistently execute your builds at any time. For example, you can try adding a commit trigger to the pipeline that listens for commits against the codebase and automatically kicks off the pipeline. All objects you create are available to reuse in your pipelines.
You can also save your build pipelines as part of your source code. Everything that you do in Harness is represented by YAML; you can store it all alongside your project files. Here is a YAML example of the pipeline created in this tutorial.
Pipeline YAML example
```yaml
pipeline:
name: k8s tutorial example
identifier: k8s_tutorial_example
projectIdentifier: k8s_tutorial_project
orgIdentifier: default
tags: {}
properties:
ci:
codebase:
connectorRef: ghconnector
build: <+input>
stages:
- stage:
name: Build Test and Push
identifier: Build_Test_and_Push
description: ""
type: CI
spec:
cloneCodebase: true
infrastructure:
type: KubernetesDirect
spec:
connectorRef: k8stutorial
namespace: harness-delegate-ng
automountServiceAccountToken: true
nodeSelector: {}
os: Linux
execution:
steps:
- step:
type: Run
name: Run Unit Tests
identifier: Run_Unit_Tests
spec:
connectorRef: dockerhubconnector
image: golang:1.17
shell: Sh
command: |2-
go get gotest.tools/gotestsum
gotestsum --format=standard-verbose --junitfile unit-tests.xml || true
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -tags netgo
reports:
type: JUnit
spec:
paths:
- "*.xml"
- step:
type: BuildAndPushDockerRegistry
name: BuildAndPushDockerRegistry_1
identifier: BuildAndPushDockerRegistry_1
spec:
connectorRef: dockerhubconnector
repo: mydockerhub/ci-tutorial-repo
tags:
- <+pipeline.sequenceId>
- stage:
name: n test
identifier: n_test
description: ""
type: CI
spec:
cloneCodebase: false
infrastructure:
useFromStage: Build_Test_and_Push
execution:
steps:
- step:
type: Background
name: Pull and host image
identifier: Pull_and_host_image
spec:
connectorRef: dockerhubconnector
image: mydockerhub/ci-tutorial-repo:<+pipeline.sequenceId>
shell: Sh
- step:
type: Run
name: Test server connection
identifier: Test_server_connection
spec:
connectorRef: dockerhubconnector
image: curlimages/curl:7.73.0
shell: Sh
command: |-
sleep 10
curl localhost:8080
curl localhost:8080?Hello!_I_am_a_nice_demo!
```
After you build an artifact, you can use the Harness Continuous Delivery (CD) module to deploy your artifact. If you're ready to try CD, go to [Harness CD](https://developer.harness.io/continuous-delivery/3.0/) to get started.
{% @harness-feedback/feedback %}