> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/continuous-integration/3.0/troubleshooting-and-resources/tutorials-and-code-samples/ci-python.md).

# Python

You can build and test a [Python](https://www.python.org/) application using a Linux platform on [Harness Cloud](/continuous-integration/3.0/use-harness-ci/use-harness-ci/set-up-build-infrastructure/use-harness-cloud-build-infrastructure.md) or a [self-managed Kubernetes cluster](/continuous-integration/3.0/use-harness-ci/use-harness-ci/set-up-build-infrastructure/k8s-build-infrastructure.md) build infrastructure.

This guide assumes you've [created a Harness CI pipeline](/continuous-integration/3.0/use-harness-ci/use-harness-ci/prep-ci-pipeline-components.md).

### Install dependencies <a href="#install-dependencies" id="install-dependencies"></a>

Use [Run steps](/continuous-integration/3.0/use-harness-ci/use-harness-ci/run-step-settings.md) to install dependencies in the build environment.

{% tabs %}
{% tab title="Harness Cloud" %}

```yaml
- step:
    type: Run
    identifier: dependencies
    name: Dependencies
    spec:
      shell: Sh
      command: |-
        python -m pip install --upgrade pip
        pip install -r requirements.txt
      envVariables:
        PIP_CACHE_DIR: "/root/.cache"
```

{% endtab %}

{% tab title="Self-managed" %}

```yaml
- step:
    type: Run
    identifier: dependencies
    name: Dependencies
    spec:
      connectorRef: YOUR_IMAGE_REGISTRY_CONNECTOR
      image: python:latest
      command: |-
        python -m pip install --upgrade pip
        pip install -r requirements.txt
```

{% endtab %}
{% endtabs %}

{% hint style="info" %}
[Background steps](/continuous-integration/3.0/use-harness-ci/use-harness-ci/manage-dependencies/background-step-settings.md) can be used to run dependent services that are needed by steps in the same stage.
{% endhint %}

### Cache dependencies <a href="#cache-dependencies" id="cache-dependencies"></a>

Add caching to your stage.

{% tabs %}
{% tab title="Cache Intelligence" %}
Cache your Python module dependencies with [Cache Intelligence](/continuous-integration/3.0/use-harness-ci/use-harness-ci/caching-ci-data/cache-intelligence.md).

Add caching to your `stage.spec`:

```yaml
- stage:
    spec:
      caching:
        enabled: true
        key: cache-{{ checksum "requirements.txt" }}
        paths:
          - "/root/.cache"
      sharedPaths:
        - /root/.cache
```

{% endtab %}

{% tab title="Save and Restore Cache steps" %}
You can use built-in steps to:

* [Save and Restore Cache from S3](/continuous-integration/3.0/use-harness-ci/use-harness-ci/caching-ci-data/saving-cache.md)
* [Save and Restore Cache from GCS](/continuous-integration/3.0/use-harness-ci/use-harness-ci/caching-ci-data/save-cache-in-gcs.md)

<details>

<summary>Python cache key and path requirements</summary>

Python pipelines typically reference `requirements.txt` in **Save Cache** and **Restore Cache** steps, for example:

```yaml
spec:
  key: cache-{{ checksum "requirements.txt" }}
```

Additionally, `spec.sourcePaths` must include the python cache (typically `/root/.cache`) in the **Save Cache** step, for example:

```yaml
spec:
  sourcePaths:
    - "/root/.cache"
```

</details>

<details>

<summary>YAML example: Save and restore cache steps</summary>

Here's an example of a pipeline with **Save Cache to S3** and **Restore Cache from S3** steps.

```yaml
            steps:
              - step:
                  type: RestoreCacheS3
                  name: Restore Cache From S3
                  identifier: Restore_Cache_From_S3
                  spec:
                    connectorRef: AWS_Connector
                    region: us-east-1
                    bucket: your-s3-bucket
                    key: cache-{{ checksum "requirements.txt" }}
                    archiveFormat: Tar
              - step:
                  type: Run
                    envVariables:
                      PIP_CACHE_DIR: "/root/.cache"
                  ...
              - step:
                  type: SaveCacheS3
                  name: Save Cache to S3
                  identifier: Save_Cache_to_S3
                  spec:
                    connectorRef: AWS_Connector
                    region: us-east-1
                    bucket: your-s3-bucket
                    key: cache-{{ checksum "requirements.txt" }}
                    sourcePaths:
                      - "/root/.cache"
                    archiveFormat: Tar
```

</details>
{% endtab %}
{% endtabs %}

### Run tests <a href="#run-tests" id="run-tests"></a>

You can use **Run** and **Test** steps to [run tests in Harness CI](/continuous-integration/3.0/use-harness-ci/use-harness-ci/run-tests/run-tests-in-ci.md).

These examples run tests in a **Run** step.

{% tabs %}
{% tab title="Harness Cloud" %}

```yaml
- step:
    type: Run
    name: Test
    identifier: test
    spec:
      shell: Sh
      command: |-
        pip install pytest
        pytest tests.py --junit-xml=report.xml
      envVariables:
        PIP_CACHE_DIR: /root/.cache
```

{% endtab %}

{% tab title="Self-managed" %}

```yaml
- step:
    type: Run
    name: Test
    identifier: test
    spec:
      connectorRef: YOUR_IMAGE_REGISTRY_CONNECTOR
      image: python:latest
      shell: Sh
      command: |-
        python -m pip install --upgrade pip
        pip install -r requirements.txt
        pip install pytest
        pytest tests.py --junit-xml=report.xml
```

{% endtab %}
{% endtabs %}

#### Visualize test results <a href="#visualize-test-results" id="visualize-test-results"></a>

If you want to [view test results in Harness](/continuous-integration/3.0/use-harness-ci/use-harness-ci/run-tests/viewing-tests.md), make sure your test commands produce reports in JUnit XML format.

If you run tests in a **Run** step, your **Run** step must include the `reports` specification. The `reports` specification is not required for [Test steps (Test Intelligence)](#run-tests-with-test-intelligence).

```yaml
reports:
  type: JUnit
  spec:
    paths:
      - report.xml
```

#### Run tests with Test Intelligence <a href="#run-tests-with-test-intelligence" id="run-tests-with-test-intelligence"></a>

[Test Intelligence](/continuous-integration/3.0/use-harness-ci/use-harness-ci/run-tests/ti-overview.md) is available for Python.

```yaml
              - step:
                  type: Test
                  name: runTestsWithIntelligence
                  identifier: runTestsWithIntelligence
                  spec:
                    connectorRef: YOUR_IMAGE_REGISTRY_CONNECTOR
                    image: python:latest
                    command: |-
                      python3 -m venv .venv
                      . .venv/bin/activate

                      python3 -m pip install -r requirements/test.txt
                      python3 -m pip install -e .

                      pytest
                    shell: Python
                    intelligenceMode: true
```

#### Test splitting <a href="#test-splitting" id="test-splitting"></a>

Harness CI supports [test splitting (parallelism)](/continuous-integration/3.0/use-harness-ci/use-harness-ci/run-tests/speed-up-ci-test-pipelines-using-parallelism.md) for both **Run** and **Test** steps.

### Specify version <a href="#specify-version" id="specify-version"></a>

{% tabs %}
{% tab title="Harness Cloud" %}
Python is pre-installed on Harness Cloud runners. For details about all available tools and versions, go to [Platforms and image specifications](/continuous-integration/3.0/use-harness-ci/use-harness-ci/set-up-build-infrastructure/use-harness-cloud-build-infrastructure.md#platforms-and-image-specifications).

If your application requires a specific Python version, add a **Run** step to install it.

<details>

<summary>Install one Python version</summary>

```yaml
- step:
    type: Run
    name: Install Python
    identifier: installpython
    spec:
      shell: Sh
      command: |-
        sudo apt-get update && sudo apt-get install -y software-properties-common
        sudo add-apt-repository -y ppa:deadsnakes/ppa
        sudo apt-get update
        sudo apt-get install -y python3.10
        python3.10 --version
```

</details>

<details>

<summary>Install multiple Python versions</summary>

1. Add the [matrix looping strategy](/harness-ai/use-harness-platform/pipelines/looping-strategies/looping-strategies-matrix-repeat-and-parallelism.md) configuration to your stage.

```yaml
- stage:
    strategy:
      matrix:
        pythonVersion:
          - "3.11"
          - "3.10"
```

2. Reference the matrix variable in your steps.

```yaml
- step:
    type: Run
    name: Install Python
    identifier: installpython
    spec:
      shell: Sh
      command: |-
        sudo apt-get update && sudo apt-get install -y software-properties-common
        sudo add-apt-repository -y ppa:deadsnakes/ppa
        sudo apt-get update
        sudo apt-get install -y python<+matrix.pythonVersion>
        python<+matrix.pythonVersion> --version
```

</details>
{% endtab %}

{% tab title="Self-managed" %}
Specify the desired [Python Docker image](https://hub.docker.com/_/python) tag in your steps. There is no need for a separate install step when using Docker.

<details>

<summary>Use a specific Python version</summary>

```yaml
- step:
    type: Run
    name: Python Version
    identifier: pythonversion
    spec:
      connectorRef: YOUR_IMAGE_REGISTRY_CONNECTOR
      image: python:3.10.10
      shell: Sh
      command: |-
        python --version
```

</details>

<details>

<summary>Use multiple Python versions</summary>

1. Add the [matrix looping strategy](/harness-ai/use-harness-platform/pipelines/looping-strategies/looping-strategies-matrix-repeat-and-parallelism.md) configuration to your stage.

```yaml
- stage:
    strategy:
      matrix:
        pythonVersion:
          - 3.11.2
          - 3.10.10
```

2. Reference the matrix variable in the `image` field of your steps.

```yaml
- step:
    type: Run
    name: Python Version
    identifier: pythonversion
    spec:
      connectorRef: YOUR_IMAGE_REGISTRY_CONNECTOR
      image: python:<+ stage.matrix.pythonVersion >
      shell: Sh
      command: |-
        python --version
```

</details>
{% endtab %}
{% endtabs %}

### Full pipeline examples <a href="#full-pipeline-examples" id="full-pipeline-examples"></a>

The following full pipeline examples are based on the partial examples above.

{% tabs %}
{% tab title="Harness Cloud" %}
This pipeline uses [Harness Cloud build infrastructure](/continuous-integration/3.0/use-harness-ci/use-harness-ci/set-up-build-infrastructure/use-harness-cloud-build-infrastructure.md) and [Cache Intelligence](/continuous-integration/3.0/use-harness-ci/use-harness-ci/caching-ci-data/cache-intelligence.md).

If you copy this example, replace the placeholder values with appropriate values for your [code repo connector](/continuous-integration/3.0/use-harness-ci/use-harness-ci/codebase-configuration/create-and-configure-a-codebase.md#code-repo-connectors) and repository name. Depending on your project and organization, you may also need to replace `projectIdentifier` and `orgIdentifier`.

<details>

<summary>Pipeline with default Python version</summary>

```yaml
pipeline:
  name: Test a Python app
  identifier: Test_a_Python_app
  projectIdentifier: default
  orgIdentifier: default
  stages:
    - stage:
        name: Test
        identifier: test
        description: ""
        type: CI
        spec:
          cloneCodebase: true
          caching:
            enabled: true
            key: cache-{{ checksum "requirements.txt" }}
            paths:
              - "/root/.cache"
          execution:
            steps:
              - step:
                  type: Run
                  identifier: dependencies
                  name: Dependencies
                  spec:
                    shell: Sh
                    command: |-
                      python -m pip install --upgrade pip
                      pip install -r requirements.txt
                    envVariables:
                      PIP_CACHE_DIR: "/root/.cache"
              - step:
                  type: Run
                  name: Test
                  identifier: test
                  spec:
                    shell: Sh
                    command: |-
                      pip install pytest
                      pytest tests.py --junit-xml=report.xml
                    envVariables:
                      PIP_CACHE_DIR: /root/.cache
                    reports:
                      type: JUnit
                      spec:
                        paths:
                          - report.xml
          platform:
            os: Linux
            arch: Amd64
          runtime:
            type: Cloud
            spec: {}
          sharedPaths:
            - /root/.cache
  properties:
    ci:
      codebase:
        connectorRef: YOUR_CODE_REPO_CONNECTOR_ID
        repoName: YOUR_REPO_NAME
        build: <+input>
```

</details>

<details>

<summary>Pipeline with multiple Python versions</summary>

```yaml
pipeline:
  name: Test a Python app
  identifier: Test_a_Python_app
  projectIdentifier: default
  orgIdentifier: default
  stages:
    - stage:
        name: Test
        identifier: test
        description: ""
        type: CI
        strategy:
          matrix:
            pythonVersion:
              - 3.11.2
              - 3.10.10
        spec:
          cloneCodebase: true
          caching:
            enabled: true
            key: cache-{{ checksum "requirements.txt" }}
            paths:
              - "/root/.cache"
          execution:
            steps:
              - step:
                  type: Action
                  name: Install python
                  identifier: installpython
                  spec:
                    uses: actions/setup-python@v4
                    with:
                      python-version: <+ stage.matrix.pythonVersion >
                      token: <+ secrets.getValue("github_token") >
              - step:
                  type: Run
                  identifier: dependencies
                  name: Dependencies
                  spec:
                    shell: Sh
                    command: |-
                      python -m pip install --upgrade pip
                      pip install -r requirements.txt
                    envVariables:
                      PIP_CACHE_DIR: "/root/.cache"
              - step:
                  type: Run
                  name: Test
                  identifier: test
                  spec:
                    shell: Sh
                    command: |-
                      pip install pytest
                      pytest tests.py --junit-xml=report.xml
                    envVariables:
                      PIP_CACHE_DIR: /root/.cache
                    reports:
                      type: JUnit
                      spec:
                        paths:
                          - report.xml
          platform:
            os: Linux
            arch: Amd64
          runtime:
            type: Cloud
            spec: {}
          sharedPaths:
            - /root/.cache
  properties:
    ci:
      codebase:
        connectorRef: YOUR_CODE_REPO_CONNECTOR_ID
        repoName: YOUR_REPO_NAME
        build: <+input>
```

</details>
{% endtab %}

{% tab title="Self-managed" %}
If you copy this example, replace the placeholder values with appropriate values for your [code repo connector](/continuous-integration/3.0/use-harness-ci/use-harness-ci/codebase-configuration/create-and-configure-a-codebase.md#code-repo-connectors), [kubernetes cluster connector](/harness-ai/use-harness-platform/connectors/cloud-providers/add-a-kubernetes-cluster-connector.md), kubernetes namespace, and repository name. Depending on your project and organization, you may also need to replace `projectIdentifier` and `orgIdentifier`.

<details>

<summary>Pipeline with one specific Python version</summary>

Here is a single-stage pipeline, with steps that use Python 3.10.10.

```yaml
pipeline:
  identifier: Test a Python app
  name: Test_a_Python_app
  orgIdentifier: default
  projectIdentifier: default
  stages:
    - stage:
        identifier: default
        name: default
        spec:
          cloneCodebase: true
          execution:
            steps:
              - step:
                  type: Run
                  name: Test
                  identifier: test
                  spec:
                    connectorRef: YOUR_IMAGE_REGISTRY_CONNECTOR
                    image: python:3.10.10
                    shell: Sh
                    command: |-
                      pip install pytest
                      pytest tests.py --junit-xml=report.xml
                    reports:
                      type: JUnit
                      spec:
                        paths:
                          - "report.xml"
          infrastructure:
            type: KubernetesDirect
            spec:
              connectorRef: YOUR_KUBERNETES_CLUSTER_CONNECTOR_ID
              namespace: YOUR_KUBERNETES_NAMESPACE
              automountServiceAccountToken: true
              nodeSelector: {}
              os: Linux
        type: CI
  properties:
    ci:
      codebase:
        connectorRef: YOUR_CODE_REPO_CONNECTOR_ID
        repoName: YOUR_REPO_NAME
        build: <+input>
```

</details>

<details>

<summary>Pipeline with multiple Python versions</summary>

Here is a single-stage pipeline, with a matrix looping strategy for Python versions 3.11.2 and 3.10.10.

```yaml
pipeline:
  identifier: Test a Python app
  name: Test_a_Python_app
  orgIdentifier: default
  projectIdentifier: default
  stages:
    - stage:
        strategy:
          matrix:
            pythonVersion:
              - 3.11.2
              - 3.10.10
        identifier: default
        name: default
        spec:
          cloneCodebase: true
          execution:
            steps:
              - step:
                  type: Run
                  name: Test
                  identifier: test
                  spec:
                    connectorRef: YOUR_IMAGE_REGISTRY_CONNECTOR
                    image: python:<+ stage.matrix.pythonVersion >
                    shell: Sh
                    command: |-
                      python -m pip install --upgrade pip
                      pip install -r requirements.txt
                      pip install pytest
                      pytest tests.py --junit-xml=report.xml
                    reports:
                      type: JUnit
                      spec:
                        paths:
                          - "report.xml"
          infrastructure:
            type: KubernetesDirect
            spec:
              connectorRef: YOUR_KUBERNETES_CLUSTER_CONNECTOR_ID
              namespace: YOUR_KUBERNETES_NAMESPACE
              automountServiceAccountToken: true
              nodeSelector: {}
              os: Linux
        type: CI
  properties:
    ci:
      codebase:
        connectorRef: YOUR_CODE_REPO_CONNECTOR_ID
        repoName: YOUR_REPO_NAME
        build: <+input>
```

</details>
{% endtab %}
{% endtabs %}

### Next steps <a href="#next-steps" id="next-steps"></a>

Now that you have created a pipeline that builds and tests a Python app, you could:

* Create [triggers](/harness-ai/use-harness-platform/triggers.md) to automatically run your pipeline.
* Add steps to [build and upload artifacts](/continuous-integration/3.0/use-harness-ci/use-harness-ci/build-and-upload-artifacts.md).
* Add a step to [build and push an image to a Docker registry](/continuous-integration/3.0/use-harness-ci/use-harness-ci/build-and-upload-artifacts/build-and-push/build-and-push-to-docker-registry.md).