Test Management
CI pipelines generate large volumes of test data, but most teams lack visibility and control over test health. Unstable tests block releases, failing pipelines slow delivery, and limited insight into coverage makes it hard to understand real risk.
Harness Test Management turns test results into a complete test health system. After you upload test results, Harness continuously analyzes execution history to detect flaky tests, enforce automated policies, enable safe quarantine, and provide coverage insights across builds and pull requests.
This allows teams to keep pipelines fast and reliable while continuously improving test quality and release confidence.
Test Management Workflow
Test Management follows a simple lifecycle:
- Upload test results from your pipeline.
- Build historical test intelligence across executions.
- Detect instability automatically (flaky tests).
- Apply governance rules with policies.
- Unblock pipelines safely using quarantine.
- Measure test effectiveness with code coverage.
Once enabled, this workflow runs continuously without requiring changes from developers.
Key Features
| Capability | What It Does | Value |
|---|---|---|
| Flaky Detection | Continuously analyzes historical test results to identify tests with inconsistent pass/fail behavior across runs and environments | Eliminates trust issues in CI by surfacing unreliable tests early and reducing wasted debugging time |
| Quarantine | Automatically isolates identified flaky tests from pipeline pass/fail status while continuing to execute and track them | Preserves delivery velocity without hiding risk, enabling teams to ship while fixing test instability |
| Policies | Applies user-defined rules to automatically mark or quarantine tests based on failures, flakiness, or performance signals | Automates governance and enforces consistent quality standards without manual intervention |
| Code Coverage | Collects and visualizes test coverage for you builds, to show how much code is exercised by tests | Increases release confidence by quantifying risk and guiding targeted improvements to test quality |
How It Works
Test Management is powered by the Harness CLI (hcli) and supports the following functions:
- Stores test execution history per repository
- Analyzes test behavior within a 14-day observation window
- Detects flaky tests when the same test passes and fails on the same commit
- Evaluates policies during pipeline test execution
- Tracks quarantined tests without blocking releases
- Displays coverage insights in pipeline executions and PRs
Key CLI Commands
Test Management is powered by the Harness CLI (hcli).
Some commands are typically used inside your CI pipelines (for example uploading test results or coverage on every PR), while others are usually run by platform or QA teams from a local environment or admin pipeline (for example managing policies).
| CLI Endpoint | Purpose | Example |
|---|---|---|
hcli test-reports | Upload and manage test result data from pipelines | hcli test-reports upload test-results.xml |
hcli cov | Analyze and upload code coverage reports | hcli cov upload --file=coverage.out ... |
hcli test-management flaky | View and manage flaky test status | hcli test-management flaky get ... |
hcli test-management quarantine | Manage quarantined tests | hcli test-management quarantine set ... |
hcli test-management policy | Configure automated test policies | hcli test-management policy set ... |
To explore all available commands and options, use --help:
hcli --help
hcli test-management --help
hcli cov --help
Installing hcli
hcli is available automatically when running builds through Harness using any build infrastructure.
For local usage, download the binary:
# macOS (Apple Silicon)
curl -fsSL -o hcli "https://storage.googleapis.com/harness-ti/hcli/v0.7/hcli-darwin-arm64"
chmod +x hcli
# macOS (Intel)
curl -fsSL -o hcli "https://storage.googleapis.com/harness-ti/hcli/v0.7/hcli-darwin-amd64"
chmod +x hcli
# Linux (amd64)
curl -fsSL -o hcli "https://storage.googleapis.com/harness-ti/hcli/v0.7/hcli-linux-amd64"
chmod +x hcli
Move to your PATH or use directly: ./hcli --help
Quick Setup
Automatic Flaky Test detection will be enabled by default.
- Run tests in you pipeline.
- Upload test results to Harness.
- Python
- Java/Gradle
- Go
- run:
script: |-
pytest --junitxml=test-results.xml -v
hcli test-reports upload test-results.xml
- step:
type: Run
name: Run Tests
spec:
shell: Sh
command: |
./gradlew clean test
hcli test-reports upload "build/test-results/test/*.xml"
- run:
script: |-
go install github.com/jstemmer/go-junit-report/v2@latest
go test -v ./... 2>&1 | go-junit-report > report.xml
hcli test-reports upload report.xml