Run Tests step settings
This topic describes settings for the Harness CI Run Tests step. Use the Run Tests step to run tests and enable Test Intelligence in CI pipelines.
- Some settings are located under Additional Configuration in the Pipeline Studio's visual editor.
- Some settings are only applicable to certain languages or build tools.
- Settings specific to containers, such as Set Container Resources, are not applicable when using the step in a stage with VM or Harness Cloud build infrastructure.
Enter a name summarizing the step's purpose. Harness automatically assigns an Id (Entity Identifier Reference) based on the Name. You can edit the Id.
Optional text string.
Container Registry and Image
The Container Registry is a Harness container registry connector for the image that you want Harness to run build commands on, such as Docker Hub.
The Image is the FQN (fully-qualified name) or artifact name of the Docker image to use when this step runs commands, for example
us.gcr.io/playground-123/quickstart-image. The image name should include the tag. If you don't include a tag, Harness uses the
You can use any Docker image from any Docker registry, including Docker images from private registries. Different container registries require different name formats:
- Docker Registry: Enter the name of the artifact you want to deploy, such as
library/tomcat. Wildcards aren't supported. FQN is required for images in private container registries.
- ECR: Enter the FQN (fully-qualified name) of the artifact you want to deploy. Images in repos must reference a path, for example:
- GCR: Enter the FQN (fully-qualified name) of the artifact you want to deploy. Images in repos must reference a path starting with the project ID that the artifact is in, for example:
The stage's build infrastructure determines whether these fields are required or optional:
- Kubernetes cluster build infrastructure: Container Registry and Image are always required.
- Local runner build infrastructure: Container Registry and Image are always required.
- Self-hosted cloud provider VM build infrastructure: Run Tests steps can use binaries that you've made available on your build VMs. The Container Registry and Image are required if the VM doesn't have the necessary binaries. These fields are located under Additional Configuration for stages that use self-hosted VM build infrastructure.
- Harness Cloud build infrastructure: Run Tests steps can use binaries available on Harness Cloud machines, as described in the image specifications. The Container Registry and Image are required if the machine doesn't have the binaries you need. These fields are located under Additional Configuration for stages that use Harness Cloud build infrastructure.
Select the source code language to build: C#, Java, Kotlin, or Scala.
Additional settings appear if you select C# or Java.
Select the build environment to test.
Select the framework version to test.
This setting is only available if you select DOTNET as the Build Tool.
Supply a comma-separated list of namespace prefixes that you want to test.
Do you want to enable Error Tracking?
Error tracking helps you be more proactive at discovering and remediating errors early in the software development lifecycle. It help s you more easily discover issues and assess the quality of code before it reaches production.
Select Yes to enable error tracking. When enabled, a set of commands are auto-populated in the Pre-Command field. Review these commends to ensure they are compatible with your build. The auto-populated commands are enclosed between
#ET-SETUP-END, for example:
if [ $arch = "x86_64" ]; then
if cat /etc/os-release | grep -iq alpine ; then
wget -qO- https://get.et.harness.io/releases/latest/alpine/harness-et-agent.tar.gz | tar -xz
wget -qO- https://get.et.harness.io/releases/latest/nix/harness-et-agent.tar.gz | tar -xz
elif [ $arch = "aarch64" ]; then
wget -qO- https://get.et.harness.io/releases/latest/arm/harness-et-agent.tar.gz | tar -xz
# export ET_SHUTDOWN_GRACETIME=30000
# Uncomment the line below if using Java version 10 or above
# export JAVA_TOOL_OPTIONS="-Xshare:off -XX:-UseTypeSpeculation -XX:ReservedCodeCacheSize=512m -agentpath:/opt/harness/lib/libETAgent.so"
Error tracking output is reported on the Error Tracking tab when the pipeline runs.
This setting is located under Additional Configuration.
You can provide a comma-separated list of test annotations used in unit testing. Any method with a specified annotation is treated as a test method. If not specified, the defaults are:
org.junit.Test, org.junit.jupiter.api.Test, org.testing.annotations.Test
Select the build automation tool. Supported tools vary by Language.
Enter the arguments for the build tool. These are used as input for the chosen build tool.
The following languages and build tools have specific build argument requirements:
- Java: Provide runtime arguments for the tests, for example:
Test -Dmaven.test.failure.ignore=true -DfailIfNoTests=false.
- C#: Provide runtime arguments for the tests, for example:
/path/to/test.dll /path/to/testProject.dll. Do not inject another instrumenting agent, such as a code-coverage agent, in the argument string.
- NUnit C#: Provide runtime executables and arguments for the tests, for example:
. "path/to/nunit3-console.exe" path/to/TestProject.dll --result="UnitTestResults.xml" /path/to/testProject.dll. You must include the executable in the string. Do not inject another instrumenting agent, such as a code-coverage agent, in the string.
Test Report Paths
Specify one or more paths to files that store test results in JUnit XML format. You can add multiple paths. If you specify multiple paths, make sure the files contain unique tests to avoid duplicates. Glob is supported.
This field is required for the Run Tests step to publish test results.
Enter the commands for setting up the environment before running the tests. For example,
printenv prints all or part of the environment.
If a script is supplied here, select the corresponding Shell option.
Enter the commands used for cleaning up the environment after running the tests. For example,
sleep 600 suspends the process for 600 seconds.
If a script is supplied here, select the corresponding Shell option.
Run Only Selected Tests
This option must be selected to enable Test Intelligence.
If this option is unchecked, Test Intelligence is disabled and all tests run on every build.
Leave blank or provide a comma-separated list of source code package prefixes, such as
com.company., io.company.migrations. If you do not provide a list, Harness auto-detects the packages.
Variables passed to the container as environment variables and used in the step's commands.
Output variables expose values for use by other steps or stages in the pipeline.
To create an output variable, do the following in the step where the output variable originates:
In the Command field, export the output variable. For example, the following command exports a variable called
myVarwith a value of
In the step's Output Variables, declare the variable name, such as
To call a previously-exported output variable in a later step or stage in the same pipeline, use a variable expression that includes the originating step's ID and the variable name.
To reference an output variable in another step in the same stage, use either of the following expressions:
To reference an output variable in a different stage than the one where it originated, use the following expression:
YAML example: Output variable
In the following YAML example, step
a exports an output variable called
myVar, and then step
b references that output variable.
command: export myVar=varValue
- name: myVar
Image Pull Policy
If you specified a Container Registry and Image, you can specify an image pull policy:
- Always: The kubelet queries the container image registry to resolve the name to an image digest every time the kubelet launches a container. If the kubelet encounters an exact digest cached locally, it uses its cached image; otherwise, the kubelet downloads (pulls) the image with the resolved digest, and uses that image to launch the container.
- If Not Present: The image is pulled only if it isn't already present locally.
- Never: The image is not pulled.
Run as User
If you specified a Container Registry and Image, you can specify the user ID to use for running processes in containerized steps.
For a Kubernetes cluster build infrastructure, the step uses this user ID to run all processes in the pod. For more information, go to Set the security context for a pod.
Set Container Resources
These settings specify the maximum resources used by the container at runtime:
- Limit Memory: The maximum memory that the container can use. You can express memory as a plain integer or as a fixed-point number using the suffixes
M. You can also use the power-of-two equivalents
Mi. The default is
- Limit CPU: The maximum number of cores that the container can use. CPU limits are measured in CPU units. Fractional requests are allowed. For example, you can specify one hundred millicpu as
100m. The default is
400m. For more information go to Resource units in Kubernetes.
The timeout limit for the step. Once the timeout is reached, the step fails and pipeline execution continues.
To change what happens when steps fail, go to Step Failure Strategy settings.
To configure when pipelines should skip certain steps, go to Step Skip Condition settings.