Use CI Run steps

You can use a Run step to run commands or scripts in a CI pipeline. Here are some examples of different ways you can use Run steps.

    - step:
        type: Run
        name: Run pytest
        identifier: Run_pytest
        spec:
          connectorRef: YOUR_IMAGE_REGISTRY_CONNECTOR
          image: python:latest
          shell: Sh
          command: |-
            echo "Welcome to Harness CI"
            uname -a
            pip install pytest
            pip install pytest-cov
            pip install -r requirements.txt

            pytest -v --cov --junitxml="result.xml" test_api.py test_api_2.py test_api_3.py

            echo "Done"
          reports:
            type: JUnit
            spec:
              paths:
                - "**/*.xml"

Add the Run step

You need a CI pipeline with a Build stage where you'll add the Run step.

In order for the Run step to execute your commands, the build environment must have the necessary binaries for those commands. Depending on the stage's build infrastructure, Run steps can use binaries that exist in the build environment or pull an image, such as a public or private Docker image, that contains the required binaries. For more information about when and how to specify images, go to the Container registry and image settings.

Run step settings

The CI Run step has the following settings.

Metadata

• Name: Enter a name summarizing the step's purpose. Harness automatically assigns an ID based on the Name.

• Description: Optional text string describing the step's purpose.

Container Registry and Image

Container Registry and Image ensure that the build environment has the binaries necessary to execute the commands that you want to run in this step. For example, a cURL script may require a cURL image, such as curlimages/curl:7.73.0.

What are the expected values for Container Registry and Image

For Container Registry settings, provide a Harness container registry connector, such as a Docker connector, that connects to a container registry, such as Docker Hub, where the Image is located.

For Image, provide the FQN (fully-qualified name) or artifact name and tag of the Docker image to use when this step runs commands, for example us-docker.pkg.dev/gar-prod-setup/harness-public/harness/cache:latest or maven:3.8-jdk-11. If you don't include a tag, Harness uses the latest tag. Depending on the connector and feature flags set, an FQN may be required.

You can use any Docker image from any Docker registry, including Docker images from private registries. Different container registries require different name formats, for example:

• Docker Registry: Input the name of the artifact you want to deploy, such as library/tomcat. Wildcards aren't supported. FQN is required for images in private container registries.

• ECR: Input the FQN of the artifact you want to deploy. Images in repos must reference a path, for example: 40000005317.dkr.ecr.us-east-1.amazonaws.com/todolist:0.2.

• GAR: Input the FQN of the artifact you want to deploy. Images in repos must reference a path starting with the project ID that the artifact is in, for example: us-docker.pkg.dev/gar-prod-setup/harness-public/harness/cache:latest.

  

  Configuring a Container Registry and Image settings.

Pulling images from JFrog Artifactory Docker registries

If you need to pull images from a JFrog Artifactory Docker registry, create a Docker connector that connects to your JFrog instance. Don't use the Harness Artifactory connector - The Artifactory connector only supports JFrog non-Docker registries.

To create a Docker connector for a JFrog Docker registry:

1. Go to Connectors in your Harness project, organization, or account resources, and select New Connector.

2. Select Docker Registry under Artifact Repositories.

3. Enter a Name for the connector. The Description and Tags are optional.

4. For Provider Type, Select Other.

5. In Docker Registry URL, enter your JFrog URL, such as https://mycompany.jfrog.io.

6. In the Authentication settings, you must use Username and Password authentication.

   • Username: Enter your JFrog username.

   • Password: Select or create a Harness text secret containing the password corresponding with the Username.

7. Complete any other settings and save the connector. For information all Docker Registry connector settings, go to the Docker connector settings reference.

One completed, please remember to use the FQN location of the image unless you have set the appropriate feature flags as listed above

Shell and Command

Use these fields to define the commands that you need to run in this step.

For Shell, select the shell type. Options include: Bash, PowerShell, Pwsh (PowerShell Core), Sh, and Python. If the step includes commands that aren't supported for the selected shell type, the build fails. Required binaries must be available on the build infrastructure or through a specified Container Registry and Image.

In the Command field, enter POSIX shell script commands for this step. The script is invoked as if it were the entry point. If the step runs in a container, the commands are executed inside the container.

Reference background services

You can reference services started in Background steps by using the Background step's Id in your Run step's Command. For example, a cURL command could call BackgroundStepId:5000 where it might otherwise call localhost:5000. The exact format depends on your build infrastructure. For more information, go to Background step settings - Name and ID and Background step settings - Port Bindings.

If the Background step is inside a step group, you must include step group ID, such as StepGroupId_BackgroundStepId:5000, even if both steps are in the same step group.

Scripts that produce output variables

If your script produces an output variable, you must declare the output variable in the Run step's Output Variables. For example, the following step runs a python script that defines an output variable called OS_VAR, and OS_VAR is also declared in the outputVariables.

Images without a shell

To support Docker images without a shell, the Command field is optional. You must provide either Image, Command, or both. If you provide only Image, Harness runs the image entrypoint.

If Command is empty or omitted, then Harness ignores Shell. The default value for Shell is Sh; however the presence of Shell doesn't require Command.

Report Paths

If relevant to the commands in your Run step, you can specify one or more paths to files that store test results in JUnit XML format. You can add multiple paths. If you specify multiple paths, make sure the files contain unique tests to avoid duplicates. Glob is supported.

This setting is required for the Run step to be able to publish test results.

For example, this step runs pytest and produces a test report in JUnit XML format.

Environment Variables

You can inject environment variables into the step container and use them in the commands executed in this step. You must input a Name and Value for each variable.

You can reference environment variables by name in commands. For example, a Bash script would use $var_name or ${var_name}, and a Windows PowerShell script would use $Env:varName.

Variable values can be fixed values, runtime inputs, or expressions. For example, if the value type is expression, you can input a value that references the value of some other setting in the stage or pipeline.

Output Variables

Output variables expose values for use by other steps or stages in the pipeline.

Create an output variable

To create an output variable, do the following in the step where the output variable originates:

1. In the Command field, export the output variable. For example, the following command exports a variable called myVar with a value of varValue:

2. In the step's Output Variables, declare the variable name, such as myVar.

Reference an output variable

To reference an output variable in a later step or stage in the same pipeline, use a variable expression that includes the originating step's ID and the variable's name.

Use either of the following expressions to reference an output variable in another step in the same stage:

To reference an output variable in a stage other than the one where the output variable originated, use either of the following expressions:

Early access feature: Secret type selection

Currently, this early access feature is behind the feature flag CI_ENABLE_OUTPUT_SECRETS. Contact Harness Support to enable the feature.

You can enable type selection for output variables in Run steps.

If you select the Secret type, Harness treats the output variable value as a secret and applies secrets masking where applicable.

Scoping output variables using aliases

To prevent variable name conflicts, you can use Publish Variable Names (Alias) to scope output variables to different entities.

1. Export the variables: Use Output Variables to export the variables.

2. Define an alias: In Publish Variable Names (Alias), enter an alias to use to reference the exported output variables.

3. Select a scope: In Scope, select the scope for the exported output variable.

The following screenshot shows the output alias configured with Stage scope:

The following screenshot shows the output alias configured with Pipeline scope:

You can scope output variables to the following entities:

• Step group:

  • The output variable must be used in the same step group, including nested child step groups.

  • The format for referencing an exported step group output variable using its alias is:

• Stage:

  • The output variable can be used anywhere in the same stage, including step groups in the same stage. It cannot be used outside of the same stage.

  • The format for referencing an exported stage output variable using its alias is:

• Pipeline:

  • The output variable can be used anywhere in the same pipeline but not in a chained pipeline.

  • The format for referencing an exported pipeline output variable using its alias is:

To reference a map of exported output variables, reference the alias in the format <+exportedVariables.getValue("SCOPE.ALIAS_NAME")>, like <+exportedVariables.getValue("stepGroup.info")>.

For more information, go to Scoping output variables using aliases in the Shell Script step documentation.

Additional container settings

Settings specific to containers are not applicable in a stages that use VM or Harness Cloud build infrastructure.

Privileged

Enable this option to run the container with escalated privileges. This is equivalent to running a container with the Docker --privileged flag.

Image Pull Policy

If you specified a Container Registry and Image, you can specify an image pull policy:

• Always: The kubelet queries the container image registry to resolve the name to an image digest every time the kubelet launches a container. If the kubelet encounters an exact digest cached locally, it uses its cached image; otherwise, the kubelet downloads (pulls) the image with the resolved digest, and uses that image to launch the container.

• If Not Present: The image is pulled only if it is not already present locally.

• Never: The image is assumed to exist locally. No attempt is made to pull the image.

Run as User

If you specified a Container Registry and Image, you can specify the user ID to use for running processes in containerized steps.

For a Kubernetes cluster build infrastructure, the step uses this user ID to run all processes in the pod. For more information, go to Set the security context for a pod.

Set Container Resources

Maximum resources limits for the resources used by the container at runtime:

• Limit Memory: Maximum memory that the container can use. You can express memory as a plain integer or as a fixed-point number with the suffixes G or M. You can also use the power-of-two equivalents, Gi or Mi. Do not include spaces when entering a fixed value. The default is 500Mi.

• Limit CPU: The maximum number of cores that the container can use. CPU limits are measured in CPU units. Fractional requests are allowed. For example, you can specify one hundred millicpu as 0.1 or 100m. The default is 400m. For more information, go to Resource units in Kubernetes.

Timeout

Set the timeout limit for the step. Once the timeout limit is reached, the step fails and pipeline execution continues. To set skip conditions or failure handling for steps, go to:

• Step Skip Condition settings

• Step Failure Strategy settings

Run step logs and test results

During and after pipeline runs, you can find step logs on the Build details page.

If your pipeline runs tests, you can view test reports on the Build details page.

Run step and Pipeline Annotations

Pipeline Annotations allow you to publish rich, structured insights directly into the Harness pipeline execution page. Instead of sifting through thousands of lines of logs to find critical information like test summaries, security scan results, or deployment notes, you can use the hcli annotate command to push this data to a dedicated Annotations tab in the Harness UI.

For more information visit Pipeline Annotations

Hidden/Invisible Characters

Customers may occasionally encounter unexplained behavior in their scripts caused by hidden or invisible characters. These characters often appear when copying and pasting from non-plain-text sources and can lead to unexpected script errors. Harness includes a feature to display invisible characters, which is enabled by default.

When invisible characters are present, users will see a highlighted space in their scripts.

They can hover over the highlight to view the character and click "Adjust settings" to manage the display.

If a selection was accidentally made, the user can right-click within the script area and open the Command Palette.

A dialog box will appear, allowing the user to search for and toggle the setting that controls how invisible characters are highlighted.

Troubleshoot script execution (Run steps)

Go to the CI Knowledge Base for questions and issues related to script execution and using Run steps, such as:

• Can I use an image that doesn't have a shell in a Run step?

• Is a Docker image required to use the Run step on local runner build infrastructure?

• When attempting to export an output variable from a Run step using a Python shell, the step fails with "no such file or directory"

• What does the "Failed to get image entrypoint" error indicate in a Kubernetes cluster build?

• Does the Harness Run step overwrite the base image container entry point?

• Why is the default entry point not running for the container image used in the Run step?

• How do I start a service started in a container that would usually be started by the default entry point?

• How do I run the default entry point of the image used in the Run step?

• Does CI support running Docker-in-Docker images?

• Can't connect to Docker daemon with Docker-in-Docker Background step.

• Concatenated variable values in PowerShell scripts print to multiple lines