Skip to main content

Pipelines overview

Last updated on

A Harness pipeline is a workflow that defines a sequence of stages and steps. You can use a pipeline to build and test code, deploy services, run security checks, manage infrastructure, or perform other tasks in a workflow.

This topic introduces the basic pipeline concepts and shows how they fit together.

note

All pipeline configurations and examples in this documentation use YAML version v0.

What you will learn from this topic​

Before you begin​

  • Harness account and project: You need access to a Harness account and project. If you are new to Harness, go to Harness Platform overview to get started.
  • Basic DevOps concepts: Familiarity with source code, builds, artifacts, deployments, and Git can help you understand pipeline examples.

What is a pipeline?​

A pipeline is the top-level workflow in Harness. It contains the stages and steps that define the work you want Harness to perform.

A pipeline can be used to:

  • Build and test code.
  • Build and publish an artifact.
  • Deploy a service or other workload.
  • Run security scans.
  • Provision and manage infrastructure.
  • Run feature flag operations.
  • Run other tasks as part of an automated workflow.

A pipeline can contain one or more stages. Each stage contains the logic for one part of the workflow.

Access and permissions​

Pipeline access is controlled through Harness role-based access control (RBAC). Your permissions determine which pipeline resources you can view, create, edit, and execute.

For more information, refer to RBAC in Harness.

Pipelines across Harness modules​

Pipelines are a core Harness platform construct used across multiple modules. Depending on the module and stage type, you can build, deploy, provision infrastructure, run security checks, manage databases, operate feature flags, and automate other workflows.

You can combine stages from different modules in a single pipeline when the workflow requires it.

ModuleTypical pipeline use
Continuous Integration (CI)Build and test code and publish artifacts.
Continuous Delivery & GitOps (CD)Deploy services and other workloads.
Infrastructure as Code Management (IaCM)Provision and manage infrastructure through pipeline workflows.
Internal Developer Portal (IDP)Automate developer workflows and self-service operations.
Supply Chain Security (SCS)Secure software supply chain workflows through pipelines.
Security Testing Orchestration (STO)Run security scans and use scan results in pipeline workflows.
Database DevOps (DB DevOps)Orchestrate database changes through pipeline workflows.
Chaos EngineeringRun chaos experiments as steps in supported pipeline stages.
Feature Management & Experimentation (FME)Run feature flag operations as part of a pipeline.

The exact stages and steps available to you depend on the modules enabled for your project and your permissions.

Pipeline storage​

Pipelines can be stored in Harness or in your Git repository. Storing pipelines in Git enables version control, code reviews, and collaboration through your existing Git workflows. For more information, refer to Git Experience.

Pipeline studio​

Pipeline Studio is the visual editor in Harness where you create and configure pipelines. It provides a drag-and-drop canvas for adding stages and steps, and a YAML editor for advanced configuration. You can switch between visual and YAML views at any time. For more information, refer to Write pipelines in YAML.

Pipeline structure​

A pipeline is the top-level workflow in Harness. It belongs to a project and can span multiple Harness modules.

Each pipeline has:

  • A name and a unique identifier (ID). The ID is set when the pipeline is created and cannot be changed after saving.
  • The organization and project it belongs to.
  • One or more stages, and each stage contains steps.
  • Optional variables, input sets, triggers, notifications, and policy sets.

Stages​

A stage is a major segment of a pipeline. Stages are often based on workflow milestones such as building, approving, and deploying. Every stage has a type that determines its available settings and steps.

For more information, refer to Add a stage.

Each stage has configuration organized across tabs. The tabs available depend on the stage type.

  • Overview: Name, identifier, description, tags, and stage variables.
  • Infrastructure (Build and Deploy stages): Where the stage runs. Options include Harness Cloud, a Kubernetes cluster, a VM, or a cloud provider runtime.
  • Execution: The steps that run inside the stage, arranged on the execution canvas.
  • Advanced: Looping strategy (matrix, repeat, parallelism), conditional execution (when), failure strategy, and delegate selector.

Stage variables​

Stage variables are values defined at the stage level that can be referenced within the stage and in later stages. You can use stage variables to pass data between stages or parameterize stage configuration. For more information, refer to Stage variables.

Services​

A service represents the application or workload you want to deploy. Services are primarily used in CD Deploy stages and define what you deploy, including artifacts, manifests, and configuration files. For more information, refer to Services and environments overview.

Environment and Infrastructure​

An environment represents where you deploy your service, such as development, staging, or production. Infrastructure defines the actual target, such as a Kubernetes cluster or cloud provider account, where the deployment runs. For more information, refer to Services and environments overview.

Execution strategies​

Execution strategies determine how Harness deploys your service to the target infrastructure. Common strategies include rolling, canary, and blue-green deployments. Each strategy has different rollout behavior and rollback capabilities. For more information, refer to Deployment concepts.

Advanced settings​

You can configure advanced settings at both the stage and step level to control execution behavior, handle failures, and integrate with external systems.

Conditional execution​

Conditional execution lets you skip a stage or step based on the outcome of previous stages or steps, or based on an expression. For example, you can configure a stage to run only when a previous stage succeeds. For more information, refer to Define conditional executions for stages and steps.

Looping strategy​

Looping strategies allow you to run a stage or step multiple times using matrix, repeat, or parallelism configurations. This is useful for deploying to multiple environments or testing multiple configurations in parallel. For more information, refer to Looping strategies.

Failure strategy​

A failure strategy defines what Harness does when a stage or step fails, such as retrying the step, marking it as successful, or rolling back changes. You can configure different strategies for different error types. For more information, refer to Define failure strategies.

Send status to Git​

The Send Status to Git setting allows Harness to send the stage execution status to your Git provider for pipelines triggered by Git events. This displays the stage status as a check on pull requests and commits. For more information, refer to Git Experience.

Step​

A step is a single task within a stage's execution. Steps are the smallest unit of work in a pipeline. Examples include running a shell script, building and pushing a Docker image, deploying a Kubernetes manifest, or sending a Slack notification.

Steps can use inputs, expressions, variables, connectors, secrets, and other configuration depending on the step type.

Each step has:

  • A name and unique identifier.
  • A type that determines what the step does and what settings it exposes.
  • Optional timeout, conditional execution, and failure strategy settings.

Available step types depend on the stage type and the modules you have licensed. For example, the Run step is available in CI Build stages, while the K8s Rolling Deploy step is available in CD Deploy stages.

Go to the following step references for the steps available in each module:

  • CI steps: Steps available in CI Build stages, such as Run, Build and Push, and Git Clone.
  • CD steps: Steps available in CD Deploy stages, such as Kubernetes, Helm, and Terraform deployment steps.

Step group​

A step group is a collection of steps organized under a single named group. Step groups let you run multiple steps in parallel or sequentially, apply shared failure strategies, and reuse groups as templates. For more information, refer to Organize steps in step groups.

Pipeline templates​

Pipeline templates let you create reusable pipeline configurations that can be shared across projects and teams. Templates enforce standardization and reduce duplication by defining common pipeline patterns once and using them multiple times. For more information, refer to Templates.

Pipeline configuration​

You can configure pipeline-level settings in addition to stages and steps. The available settings depend on your Harness modules, permissions, and pipeline configuration.

Inputs and input sets​

Inputs allow you to parameterize pipelines with runtime values. Input sets save collections of runtime input values for reuse across multiple executions. For more information, refer to Input sets and overlays.

Triggers​

Triggers start pipelines automatically based on events such as Git commits, pull requests, webhooks, or schedules. You can configure triggers to run pipelines without manual intervention. For more information, refer to Triggers overview.

Variables​

Variables store and reference values across stages and steps in a pipeline. You can define variables at the pipeline, stage, or step level and use expressions to reference them. For more information, refer to Add and reference variables.

Notifications​

Notifications send alerts on pipeline events such as success, failure, or approval requests. You can send notifications to Slack, email, PagerDuty, or Microsoft Teams. For more information, refer to Notifications.

Flow control​

Flow control features let you coordinate execution across stages and steps. This includes barriers for synchronizing parallel stages, conditional execution for skipping stages based on conditions, and looping strategies for running stages multiple times. For more information, refer to Barriers.

Execution settings​

Execution settings configure pipeline-level defaults such as timeouts, concurrency limits, and execution behavior. These settings apply to all stages unless overridden at the stage level. For more information, refer to Pipeline settings.

Policy sets​

Policy sets enforce governance rules on pipelines using Open Policy Agent. You can define policies that validate pipeline configuration, control deployments, and enforce organizational standards. For more information, refer to Harness governance overview.

Advanced configurations​

The Advanced Options panel in Pipeline Studio provides additional pipeline-level settings for timeout, stage execution, re-run behavior, and access control.

  • Pipeline timeout: Set a maximum execution time for the entire pipeline.
  • Selective stage execution: Allow users to select which stages to run during manual execution.
  • Re-run settings: Control whether input data is editable when re-running a failed pipeline.
  • Delegate selector: Specify which delegates can execute the pipeline using delegate tags and selectors.
  • Public access: Mark a pipeline for public viewing without requiring authentication.

For more information, refer to Pipeline settings.

Additional pipeline capabilities​

Advanced pipeline features provide additional capabilities for complex workflows and enterprise requirements.

Analytics​

Pipeline analytics provide visibility into pipeline performance, success rates, execution times, and resource usage. You can use dashboards and reports to track trends and identify bottlenecks. For more information, refer to Dashboards.

Execution history​

Execution history shows all past runs of a pipeline, including status, duration, inputs, and logs. You can view, compare, and re-run previous executions from the history view. For more information, refer to View and compare pipeline executions.

DAG pipelines​

DAG pipelines use directed acyclic graphs to define stage dependencies and execution order. Instead of sequential or parallel execution, you can specify exactly which stages must complete before others start. For more information, refer to DAG pipelines.

Pipeline chaining​

Pipeline chaining lets you run one pipeline as a stage inside another pipeline. This allows you to break complex workflows into smaller reusable pipelines and control execution order between them. For more information, refer to Pipeline chaining.

Annotations​

Annotations let you publish custom summaries and reports to a dedicated Annotations tab in the pipeline execution view. You can use annotations to surface test results, security scan findings, or deployment notes. For more information, refer to Pipeline annotations.

Reconciliation​

Reconciliation updates a pipeline when a referenced entity such as a template, environment, or service changes. Harness prompts you to reconcile the pipeline so it reflects the updated entity. For more information, refer to Templates.

FAQs and Troubleshooting​

Next steps​