Skip to main content

Synchronize parallel stages and step groups using Barriers

Last updated on

This topic describes how to synchronize stages and step groups in your pipeline using barriers.

note

Harness provides multiple options for controlling resource usage and protecting capacity limits. Go to controlling resource usage with Barriers, Resource Constraints, and Queue steps to compare barriers, resource constraints, and queue steps.

What will you learn in this topic?​

Barriers and synchronization​

In complex pipelines that orchestrate interdependent services or components, you may need to coordinate the execution flow across different stages or step groups. For example, you might want to verify a group of services only after all of them are deployed successfully.

Harness provides Barriers to help with this kind of synchronization. A barrier pauses execution at defined points so that multiple parallel entities, such as stages or step groups, proceed only after all required parts reach the same barrier.

note

Barrier steps are only supported inside Deploy and Custom stage types.

How barriers work​

Barriers follow these rules:

  • Barriers take effect only when two or more stages or step groups use the same barrier name (configured through the Barrier Reference field in the Barrier step) and are executed in parallel.
  • All stages or step groups referencing the same barrier must reach the barrier point. Only then do they all proceed simultaneously past that point.
  • If any one of the stages or step groups fails before reaching the barrier, the remaining ones are signaled to fail as well.
  • Each stage or step group then follows its configured failure strategy.
note

Barriers are also supported across child pipelines. A parent pipeline can define and use a barrier, and any child pipeline can reference and synchronize using the same barrier.

Currently, this feature is behind the feature flag PIPE_BARRIERS_FOR_CHAINED_PIPELINES. Contact Harness Support to enable the feature.

Barriers can also be used with looping strategies. Additional constraints apply when you use barriers in looped parallel executions. Go to Using barriers with looping strategies to review those constraints.

Example​

Here is a visualization of how barriers synchronize parallel stages:

  • Stage A and Stage B both wait at Barrier X and proceed only when both reach it.
  • Stage B and Stage C both wait at Barrier Y and proceed together once both are ready.
Diagram showing stages synchronizing at shared barriers

This allows you to control complex coordination logic within a pipeline without resorting to manual delays or checks.

Add a barrier​

A barrier is a name added in a pipeline's Flow Control settings.

Perform the following steps to add a barrier:

  1. In your pipeline, select Flow Control.

    Flow Control option in a pipeline

  2. In Flow Control, select Add Barrier.

    Add Barrier button in Flow Control

  3. In Barrier Name, enter a unique name, and then click outside of the settings. The barrier is created.

    Entering a unique barrier name

Next, select the name using the Barrier step in the stages where you want to synchronize.

Use a barrier in a step​

Perform the following steps to apply a barrier:

  1. In your stage, under Execution, select Add Step, and then select Barrier.

    Adding a Barrier step under Execution

  2. Enter a name for the step.

  3. In Timeout, enter the timeout period in milliseconds. For example, 600000 milliseconds is 10 minutes. The timeout period determines how long each stage with a barrier must wait for the other stages to reach the barrier point. When the timeout expires, it is considered a deployment failure.

  4. Barrier timeouts are not hard timeouts. A barrier can fail anytime between the timeout value and timeout + 1 minute.

  5. In Barrier Reference, select the name of an existing barrier.

    Selecting an existing barrier in Barrier Reference

  6. Select Apply Changes.

Avoid rollback barrier cycles​

Do not reuse the same Barrier Reference in both the normal execution flow and a rollback section. A barrier waits for every position that shares its reference, including any position in a rollback section. Because rollback steps run only when the normal flow fails, that rollback position never arrives during a successful run, so the barrier keeps standing and the pipeline hangs until the step timeout.

For example, if the reference wall3 is used in the normal steps of two parallel stages and also in the rollback steps of one stage, the two normal-flow positions reach the barrier and wait for the rollback position. The rollback position does not execute unless the stage fails, so the pipeline stalls and eventually times out.

To synchronize a rollback flow, use a dedicated barrier reference for rollback instead of the one used in the normal flow.

Barrier reference cycle (anti-pattern) and the corrected configuration
# Anti-pattern: barrier "wall3" is referenced in both normal and rollback steps
pipeline:
  stages:
    - parallel:
        - stage:
            name: deploy
            spec:
              execution:
                steps:
                  - step:
                      type: Barrier
                      spec:
                        barrierRef: wall3        # normal flow
                rollbackSteps:
                  - step:
                      type: Barrier
                      spec:
                        barrierRef: wall3        # rollback flow -> cycle
        - stage:
            name: deploy2
            spec:
              execution:
                steps:
                  - step:
                      type: Barrier
                      spec:
                        barrierRef: wall3        # normal flow
  flowControl:
    barriers:
      - identifier: wall3
# Corrected: use a dedicated reference for the rollback flow
  flowControl:
    barriers:
      - identifier: wall3            # normal flow
      - identifier: wall3_rollback   # rollback flow
# Set barrierRef: wall3_rollback in the rollback step so it no longer shares "wall3".

Harness detects this anti-pattern and fails validation with a Barrier Deadlock Detected error that names the barrier reference and whether it was found in the normal flow, the rollback flow, or both. The check runs both when you save the pipeline and when it starts running, so a barrier reference supplied as a runtime input is caught at execution time. To resolve the error, give the rollback barrier a different reference.

Barrier Deadlock Detected: 'wall3' (mixed in normal flow and rollback flow)

Problem:
Barriers with the same reference must execute concurrently (at the same time).
Your pipeline has barriers with 'wall3' that execute sequentially,
creating a deadlock where each barrier waits for the others that haven't started yet.

The same validation applies to any barriers that share a reference but cannot execute concurrently, such as the same reference reused across sequential stages. Barriers that share a reference and run in parallel remain valid.

note

This validation is behind the feature flag PIPE_DETECT_BARRIER_CYCLES. Contact Harness Support to enable the feature.

Barrier configuration rules​

Keep the following rules in mind when you configure barriers:

  • You can have multiple Barrier steps in a stage or step group. Every Barrier step in the same stage or step group must use a unique Barrier Reference.
  • Ensure the Barrier Reference string for each related barrier across the different stages or step groups matches.
  • Do not use the same Barrier Reference in sequential stages. This results in the first stage with the Barrier step never ending, and the flow never reaches the next stage with the second Barrier step.
  • You can use the same barrier only within a single pipeline. If you attempt to use a barrier name from one pipeline in the Barrier step of another pipeline, it does not function.

Using barriers with looping strategies​

There are a few behaviors to note when using barriers within a looping strategy, for example, when setting up a matrix that creates multiple stages that run in parallel, and the stages contain a Barrier step:

  • In general, barriers are supported in all the looping strategies. You can use them when repeating stages, looping, or in matrices, or when using multi-service or multi-environment stages in pipelines.
  • When setting up the Barrier step, use the same Barrier Reference in all of the looped stages so that all of them execute up to the Barrier step, and then continue or fail together.
  • You cannot use the maxConcurrency parameter when setting up looping. When this parameter is used, not all the stages start up in parallel, and some wait for the first few to end. Barriers prevent the initial set of stages from ending, so the pipeline gets stuck.
  • When using barriers with a multi-service deployment, select the Deploy Services in Parallel option, so that the pipeline does not wait for a stage to complete before beginning the next one.
  • You can use barriers to coordinate between multiple sets of looped stages, or between a single stage and a group of looped stages. As mentioned before, the same Barrier Reference must be used across all the sets of stages. The stages all execute up to the Barrier step and wait for the others. This applies even if one of the groups starts later than another.
  • Barriers are also supported across child pipelines. A parent pipeline can define and use a barrier, and any child pipeline can reference and synchronize using the same barrier.

Next steps​

You have synchronized parallel stages and step groups using barriers. Continue your learning journey with the following: