Skip to main content

Format test reports

Results on the Tests tab are parsed from test reports specified in the Report Paths setting in Run and Run Tests steps. Test reports must be in JUnit XML format to appear on the Tests tab, because Harness parses test reports that are in JUnit XML format only.

For optimal rendering in the Harness UI, there is a limit of 8,000 characters per field. If a field in your XML file contains more than 8,000 characters, the output might render incorrectly on the Tests tab.

JUnit XML format resources

Use these resources to learn about JUnit XML formatting.

Tools with built-in JUnit XML output

Here are some Harness YAML examples for test tools that produce JUnit XML output by default.

C, C++

CTest Output JUnit

You can use the --output-junit command with CTest.

              - step:
type: Run
identifier: test
name: Test
spec:
shell: Sh
command: |-
mkdir build
cmake -S . -B build
ctest --test-dir build --output-junit out.xml
reports:
type: JUnit
spec:
paths:
- /harness/build/out.xml

Java - Gradle

Gradle

This example runs Gradle with Test Intelligence. You can also run Java tests in Run steps.

              - step:
type: RunTests
identifier: Run_Tests_with_Intelligence
name: Run Tests with Intelligence
spec:
args: test --tests
buildTool: Gradle
enableTestSplitting: true
language: Java
reports:
spec:
paths:
- "/harness/results.xml"
type: JUnit
runOnlySelectedTests: true

PHP

PHPUnit
              - step:
type: Run
identifier: test
name: Test
spec:
shell: Sh
command: |-
mkdir -p /harness/phpunit
phpunit --log-junit /harness/phpunit/junit.xml tests
reports:
type: JUnit
spec:
paths:
- /harness/phpunit/junit.xml

Python

For Python, use pytest or unittest. You can also use pytest to run unittest.

Pytest in a Run step

This example runs pytest in a Run step.

              - step:
type: Run
identifier: test
name: Test
spec:
shell: Sh
command: |-
. venv/bin/activate
mkdir /harness/test-results
pytest --junitxml=harness/test-results/junit.xml
reports:
type: JUnit
spec:
paths:
- /harness/test-results/junit.xml
info

If you use test splitting with pytest in a Run step, you must set junit_family=xunit1 in your code repo's pytest.ini file or include -o junit_family="xunit1" in the step's command.

Pytest with Test Intelligence (Run Tests step)

This example runs pytest with Test Intelligence.

              - step:
type: RunTests
name: Run Python Tests
identifier: Run_Python_Tests
spec:
language: Python
buildTool: Pytest
args: "--junitxml=out_report.xml"
runOnlySelectedTests: true
preCommand: |
python3 -m venv .venv
. .venv/bin/activate

python3 -m pip install -r requirements/test.txt
python3 -m pip install -e .
reports:
type: JUnit
spec:
paths:
- out_report.xml*

Ruby - Cucumber

Cucumber
              - step:
type: Run
identifier: test
name: Test
spec:
shell: Sh
command: |-
bundle check || bundle install
mkdir -p /harness/cucumber
bundle exec cucumber --format junit --out /harness/cucumber/junit.xml
reports:
type: JUnit
spec:
paths:
- /harness/cucumber/junit.xml

JUnit converters, formatters, and plugins

If your test tool doesn't automatically produce test results in JUnit XML format, there are JUnit converters, formatters, and plugins available for all major languages. Some examples of conversion tools and corresponding Harness YAML are provided below.

C# - .NET Core, NUnit

Example: NUnit to JUnit
              - step:
type: Run
identifier: test
name: Test
spec:
shell: Powershell
command: |-
cd dotnet-agent/TestProject1
wget -UseBasicParsing https://dot.net/v1/dotnet-install.ps1 -o dotnet-install.ps1
.\dotnet-install.ps1
dotnet build

wget https://raw.githubusercontent.com/nunit/nunit-transforms/master/nunit3-junit/nunit3-junit.xslt -o nunit3-junit.xslt

"C:/Program Files (x86)/NUnit.org/nunit-console/nunit3-console.exe" dotnet-agent/TestProject1/bin/Debug/net48/TestProject1.dll --result="UnitTestResults.xml;transform=nunit3-junit.xslt"
reports:
type: JUnit
spec:
paths:
- UnitTestResults.xml

Clojure

Go

Go Junit Report

You can use the go-junit-report tool.

              - step:
type: Run
identifier: test
name: Test
spec:
shell: Sh
command: |-
go install github.com/jstemmer/go-junit-report/v2@latest
go test -v ./... | tee report.out
cat report.out | $HOME/go/bin/go-junit-report -set-exit-code > report.xml
reports:
type: JUnit
spec:
paths:
- report.xml

Java - Maven

Maven Surefire Plugin

This example uses the Maven Surefire Plugin and runs tests with Maven and Test Intelligence.

              - step:
type: RunTests
identifier: Run_Tests_with_Intelligence
name: Run Tests with Intelligence
spec:
args: test -Dmaven.test.failure.ignore=true -DfailIfNoTests=false
buildTool: Maven
enableTestSplitting: true
language: Java
reports:
spec:
paths:
- "target/surefire-reports/*.xml"
type: JUnit
runOnlySelectedTests: true

JavaScript

ESLint

ESLint JUnit Formatter

  - step:
type: Run
name: Run ESLint Tests
identifier: run_eslint_tests
spec:
shell: Sh
command: |
mkdir -p /harness/reports
eslint ./src/ --format junit --output-file /harness/reports/eslint.xml
reports:
type: JUnit
spec:
paths:
- "/harness/reports/eslint.xml"
Jest

Jest JUnit Reporter

  - step:
type: Run
name: Run Jest Tests
identifier: run_jest_tests
spec:
shell: Sh
command: |
yarn add --dev jest-junit
jest --ci --runInBand --reporters=default --reporters=jest-junit
envVariables:
JEST_JUNIT_OUTPUT_DIR: "/harness/reports"
reports:
type: JUnit
spec:
paths:
- "/harness/reports/*.xml"
Karma

Karma JUnit Reporter

  - step:
type: Run
name: Run Karma Tests
identifier: run_karma_tests
spec:
shell: Sh
command: |
npm install
mkdir /harness/junit
karma start ./karma.conf.js
envVariables:
JUNIT_REPORT_PATH: /harness/junit/
JUNIT_REPORT_NAME: test-results.xml
reports:
type: JUnit
spec:
paths:
- "/harness/junit/test-results.xml"
Mocha

Mocha JUnit Reporter

  - step:
type: Run
name: Run Mocha Tests
identifier: run_mocha_tests
spec:
shell: Sh
command: |
npm install
mkdir /harness/junit
mocha test --reporter mocha-junit-reporter --reporter-options mochaFile=./path_to_your/file.xml
reports:
type: JUnit
spec:
paths:
- "/harness/junit/test-results.xml"

Ruby - Minitest, RSpec

Minitest

Add the Minitest Junit Formatter to your Gemfile.

  - step:
type: Run
name: Run Ruby Tests
identifier: run_ruby_tests
spec:
shell: Sh
command: |
bundle check || bundle install
bundle exec rake test --junit
reports:
type: JUnit
spec:
paths:
- "/harness/report.xml"
RSpec

Add the RSpec JUnit formatter to your Gemfile.

  - step:
type: Run
name: Run RSpec Tests
identifier: run_rspec_tests
spec:
shell: Sh
command: |
bundle check --path=vendor/bundle || bundle install --path=vendor/bundle --jobs=4 --retry=3
mkdir /harness/rspec
bundle exec rspec --format progress --format RspecJunitFormatter -o /harness/rspec/rspec.xml
reports:
type: JUnit
spec:
paths:
- "/harness/rspec/rspec.xml"

Code coverage reports and test report artifacts

For information about code coverage reports and publishing report URLs to the Artifacts tab, go to Code Coverage.

Troubleshoot test reports

Go to the CI Knowledge Base for questions and issues related to test reports in Harness CI, including: