> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/continuous-integration/use-harness-ci/use-harness-ci/codebase-configuration/clone-and-process-multiple-codebases-in-the-same-pipeline.md).

# Clone multiple code repos in one pipeline

Use Git Clone, Run, or Plugin steps to clone additional code repos into a pipeline's workspace.

In addition to the pipeline's default [codebase](/continuous-integration/use-harness-ci/use-harness-ci/codebase-configuration/create-and-configure-a-codebase.md), you can use the **Git Clone** step to clone additional code repos into the pipeline's workspace.

For example, you can use this capability to:

* Build multiple artifacts in the same pipeline. Suppose you use Packer and Ansible to build artifacts automatically, and you have separate repositories for Packer, Ansible, and code. You can clone all three repositories into the pipeline's workspace.
* Pull code from separate code and build repositories. For instance, if the Engineering team manages your code files in one repository and the Security team manages your Dockerfiles in a different repository, you can clone both repositories into the pipeline's workspace.
* Clone codebases without using the built-in clone codebase function.

## What you will learn in this topic

By the end of this topic, you will learn how to:

* Add a **Git Clone** step or a **Run** step to clone a second repo into the pipeline's workspace.
* Build an artifact from files in both repos.
* Review YAML examples for each approach.
* Use a **Plugin** step as an alternative.
* Troubleshoot common issues when you clone multiple repos.

## Before you begin

Before you clone multiple code repositories in one pipeline, make sure you have a CI pipeline with a Build stage and a [default codebase](/continuous-integration/use-harness-ci/use-harness-ci/codebase-configuration/create-and-configure-a-codebase.md#configure-the-default-codebase).

## Add a Git Clone step

Assume the [default codebase](/continuous-integration/use-harness-ci/use-harness-ci/codebase-configuration/create-and-configure-a-codebase.md#configure-the-default-codebase) is a repo that contains app code files, and the Dockerfile that builds the app image is in a different repo. You can use a [Git Clone](/continuous-integration/use-harness-ci/use-harness-ci/codebase-configuration/git-clone-step.md) step to clone the second repo into the workspace, or to fetch specific files or directories. Then, you can use a [Build and Push](/continuous-integration/use-harness-ci/use-harness-ci/build-and-upload-artifacts/build-and-push.md) step to build and push an image with files from both repos.

Add a **Git Clone** step to clone a second repo into the pipeline's workspace.

```yaml
    - step:
        type: GitClone
        name: clone second repo
        identifier: clone_second_repo
        spec:
          connectorRef: account.git
          repoName: myOrg/mySecondRepo
          build:
            type: branch
            spec:
              branch: main
```

Go to [Git Clone step](/continuous-integration/use-harness-ci/use-harness-ci/codebase-configuration/git-clone-step.md) page for more information.

## Add a Run step

Use `git` commands in [Run steps](/continuous-integration/use-harness-ci/use-harness-ci/run-step-settings.md) to clone multiple repos into a stage. You can also provide arguments to clone subdirectories, clone recursively, and so on.

For example, this step clones a GitHub repository:

```yaml
              - step:
                  type: Run
                  identifier: clone
                  name: clone
                  spec:
                    shell: Sh
                    command: |-
                      git clone https://GH_PERSONAL_ACCESS_TOKEN@github.com/ACCOUNT_NAME/REPO_NAME.git
```

To use this command, replace:

* `ACCOUNT_NAME` with your GitHub account name.
* `REPO_NAME` with the name of the GitHub repo to clone.
* `GH_PERSONAL_ACCESS_TOKEN` with a [GitHub personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) that has pull permissions to the target repository. Additional permissions might be necessary depending on the purpose of your step. Store the token as a [Harness secret](/harness-platform/use-harness-platform/secrets.md) and use a variable expression, such as `<+secrets.getValue("YOUR_TOKEN_SECRET")>`, to call it.

Go to the **Run scripts** tab on the [Run step settings](/continuous-integration/use-harness-ci/use-harness-ci/run-step-settings.md) page for details about the **Run** step settings.

{% hint style="info" %}
When you clone additional codebases that use the same credentials as your default codebase, you can [use your default codebase connector's credentials in your Run step](/continuous-integration/troubleshooting-and-resources/ci-articles-and-faqs/articles/use-git-credentials-from-codebase-connector-in-ci-pipelines-run-step.md).
{% endhint %}

## Build an artifact from both code repos

After Harness clones the files from both repos into a common workspace, add a step that builds an image with code from both repos, such as a [Build and Push to Docker step](/continuous-integration/use-harness-ci/use-harness-ci/build-and-upload-artifacts/build-and-push/build-and-push-to-docker-registry.md).

Pay attention to settings, such as the [Dockerfile setting](/continuous-integration/use-harness-ci/use-harness-ci/build-and-upload-artifacts/build-and-push/build-and-push-to-docker-registry.md#dockerfile), that assume files are in the codebase's root directory unless you specify otherwise. The pipeline's default codebase files are cloned into the root folder (`/harness`), while other codebase files are cloned into subfolders.

Depending on the default codebase, you might need to specify a non-root path for build files. You can also use commands, such as `cp`, in [Run steps](/continuous-integration/use-harness-ci/use-harness-ci/run-step-settings.md) to move cloned files around the workspace before you build the image.

## YAML examples

{% tabs %}
{% tab title="Example with Git Clone step" %}
The following YAML example describes a pipeline that clones two code repos. The pipeline clones one as the default codebase (`cloneCodebase: true`) and the second in the `GitClone` step.

{% code expandable="true" %}

```yaml
pipeline:
  name: Clone two repos
  identifier: Clone_two_repos
  projectIdentifier: my_project
  orgIdentifier: default
  tags: {}
  properties:
    ci:
      codebase:
        connectorRef: account.git1
        repoName: my-code-repo
        build: <+input>
  stages:
    - stage:
        name: build from two repos
        identifier: build_from_two_repos
        description: ""
        type: CI
        spec:
          cloneCodebase: true
          platform:
            os: Linux
            arch: Amd64
          runtime:
            type: Cloud
            spec: {}
          execution:
            steps:
              - step:
                  type: GitClone
                  name: clone second repo
                  identifier: clone_second_repo
                  spec:
                    connectorRef: account.git2
                    build:
                      type: branch
                      spec:
                        branch: main
              - step:
                  type: BuildAndPushDockerRegistry
                  name: BuildAndPushDockerRegistry_1
                  identifier: BuildAndPushDockerRegistry_1
                  spec:
                    connectorRef: mydockerhubconnector
                    repo: username/imagerepo
                    tags:
                      - <+pipeline.executionId>
                    dockerfile: /path/to/dockerfile
```

{% endcode %}
{% endtab %}

{% tab title="Example with Run step" %}
This example clones app code as the [default codebase](/continuous-integration/use-harness-ci/use-harness-ci/codebase-configuration/create-and-configure-a-codebase.md#configure-the-default-codebase) and then uses a **Run** step to clone a separate repo that has the Dockerfile that builds the app.

Because of the build infrastructure and the `image` that the **Run** step uses, you must install Git before you clone the additional repo. The **Run** step uses the following commands to install Git, verify that it works, and then clone the repo that has the Dockerfile that builds the app.

```
apk add git
git --version
git clone https://github.com/$GITHUB_USERNAME/$DOCKERFILE_REPO_NAME
```

This example also uses [stage variables](/continuous-integration/use-harness-ci/use-harness-ci/set-up-build-infrastructure/ci-stage-settings.md#advanced-stage-variables), such as `$GITHUB_USERNAME`, to reference account and repo names. All steps in the stage can access these variables.

{% code overflow="wrap" expandable="true" %}

```yaml
    stages:
        - stage:
              name: Build Test and Push
              identifier: Build_Test_and_Push
              type: CI
              spec:
                  cloneCodebase: true
                  infrastructure:
                      type: KubernetesDirect
                      spec:
                          connectorRef: YOUR_KUBERNETES_CLUSTER_CONNECTOR_ID
                          namespace: YOUR_NAMESPACE
                          automountServiceAccountToken: true
                  execution:
                      steps:
                          - step:
                                type: Run
                                name: git-clone-and-copy-dockerfile
                                identifier: git-clone-and-copy-dockerfile
                                spec:
                                    connectorRef: YOUR_IMAGE_REGISTRY_CONNECTOR
                                    image: alpine:latest
                                    shell: Sh
                                    command: |+
                                        # This stage clones the default codebase, which copies all app files and folders to the stage workspace.
                                        # To build the image, the pipeline needs to clone the repo with the Dockerfile
                                        # And then copy the Dockerfile to the current folder.

                                        apk add git
                                        git --version
                                        git clone https://github.com/$GITHUB_USERNAME/$DOCKERFILE_REPO

                                        # We now have Docker repo at the current folder:

                                        find .

                                        # Copy Dockerfile to current folder, where the Docker Build step can find it:

                                        cp $DOCKER_REPO/$APP_REPO/Dockerfile .
                                    privileged: true
                          - step:
                                type: BuildAndPushDockerRegistry
                                name: build-my-backend-service
                                identifier: buildmybackendservice
                                spec:
                                    connectorRef: YOUR_DOCKER_CONNECTOR_ID
                                    repo: <+input>
                                    tags:
                                        - <+pipeline.sequenceId>
                                        - latest
                                    optimize: true
              variables:
                  - name: GITHUB_USERNAME
                    type: Secret
                    value: myGitHubAccount
                  - name: APP_REPO
                    type: String
                    value: myCodeRepo
                  - name: DOCKERFILE_REPO
                    type: String
                    value: myGitHubRepo
              failureStrategies: []
```

{% endcode %}
{% endtab %}
{% endtabs %}

## Use a Plugin step

As an alternative to the **Git Clone** and **Run** steps, you can clone a codebase by running the [Git Drone plugin](https://plugins.drone.io/plugins/git) in a [Plugin step](/continuous-integration/use-harness-ci/use-harness-ci/use-drone-plugins/run-a-drone-plugin-in-ci.md).

## Troubleshooting

Go to the [Harness CI Knowledge Base](/continuous-integration/troubleshooting-and-resources/ci-articles-and-faqs/continuous-integration-faqs.md) for more common questions and issues related to codebases.

<details>

<summary>How do I handle authentication for git commands in a Run step?</summary>

Store authentication credentials as [secrets](https://developer.harness.io/docs/category/secrets/), and use [expressions](https://developer.harness.io/harness-platform/use-harness-platform/variables-and-expressions/runtime-inputs#expressions), such as `<+secrets.getValue("YOUR_TOKEN_SECRET")>`, to call them in your git commands.

You can also [pull credentials from a git connector that you use elsewhere in the pipeline](/continuous-integration/troubleshooting-and-resources/ci-articles-and-faqs/articles/use-git-credentials-from-codebase-connector-in-ci-pipelines-run-step.md).

</details>

<details>

<summary>Can I use codebase variables when I clone a repo in a Run step or Git Clone step?</summary>

No. Harness resolves [codebase variables](/continuous-integration/use-harness-ci/use-harness-ci/codebase-configuration/built-in-cie-codebase-variables-reference.md) only for the built-in Clone Codebase functionality. Harness does not resolve these variables for git commands in **Run** steps or for repos that **Git Clone** steps clone. Go to [Unresolved variables](/continuous-integration/use-harness-ci/use-harness-ci/codebase-configuration/built-in-cie-codebase-variables-reference.md#unresolved-variables) for more information.

</details>

<details>

<summary>Parallel container-less Git Clone steps cause intermittent build failures</summary>

Harness CI does not support container-less git steps in parallel. Running them in parallel can cause intermittent build failures. Keep your **Git Clone** steps in sequence when you use container-less build infrastructure. Go to [Parallel container-less Git Clone steps are not supported](/continuous-integration/use-harness-ci/use-harness-ci/codebase-configuration/git-clone-step.md#parallel-container-less-git-clone-steps-are-not-supported) for more information.

</details>

<details>

<summary>Git connector fails to connect to the SCM service. SCM request fails with UNKNOWN</summary>

This error can occur if your [code repo connector](/continuous-integration/use-harness-ci/use-harness-ci/codebase-configuration/git-clone-step.md#connector) uses **SSH** authentication. To resolve this error, make sure HTTPS is enabled on port 443. The Harness connection test for Git connectors uses this protocol and port.

SCM service connection failures can also occur when you use self-signed certificates. Go to the [Harness CI Knowledge Base](/continuous-integration/troubleshooting-and-resources/ci-articles-and-faqs/continuous-integration-faqs.md#git-connector-scm-connection-errors-when-using-self-signed-certificates) for more information.

</details>

## Next steps

* [Git Clone step in CI](/continuous-integration/use-harness-ci/use-harness-ci/codebase-configuration/git-clone-step.md): Review all **Git Clone** step settings.
* [Clone a subdirectory](/continuous-integration/use-harness-ci/use-harness-ci/codebase-configuration/clone-subdirectory.md): Clone a subdirectory instead of an entire repo.
* [CI codebase variables reference](/continuous-integration/use-harness-ci/use-harness-ci/codebase-configuration/built-in-cie-codebase-variables-reference.md): Reference Git codebase attributes with built-in expressions.

{% @harness-feedback/feedback %}