Skip to main content

Organize steps in step groups

Last updated on

Step groups organize related steps within a pipeline and can run their steps sequentially or in parallel, providing fine-grained control over execution flow and resource utilization.

Step groups can have conditional execution, failure strategies, and looping strategies that apply to all steps in the group.

Step groups help you to:

What you will learn from this topic​

Before you begin​

  • Harness project access: You need View and Edit permissions on Pipelines. An administrator must assign you a role that includes these permissions. For more information, refer to RBAC in Harness.
  • Existing pipeline: You need a pipeline with at least one stage. For more information, refer to Add a stage.

Add a step group​

Perform the following steps to add a step group:

  1. Go to the pipeline stage where you want to add a step group.

  2. In the stage's Execution tab, click Add Step, and then select Add Step Group.

    Add Step menu showing Step Group option

  3. Enter a name for the step group.

  4. When you manually add a step group, you can enable containerized step groups by selecting the Enable container based execution option. For more information, refer to Containerize step groups

  5. Select Apply Changes.

  6. Add or drag steps into the step group, and then configure the steps as usual.

    Step group with steps added in the pipeline editor
YAML example: Step group in a Build stage

The following YAML example includes one stage with three steps. Two steps are in a step group, and one step is not in the step group.

- stage:
    type: CI
    name: build
    identifier: build
    spec:
      cloneCodebase: false
      infrastructure:
        ...
      execution:
        steps: # Step container for the entire stage.
          - stepGroup: # Step group flag.
              name: sg1 # Step group name.
              identifier: sg1 # Step group ID.
              steps: # Step container for the step group. The following two steps are in the step group.
                - step:
                    type: Run
                    name: Run_1
                    identifier: Run_1
                    spec:
                      ...
                - step:
                    type: Run
                    name: Run_2
                    identifier: Run_2
                    spec:
                      ...
          - step: # This step is outdented so that it is not in the step group.
              type: Run
              name: Run_3
              identifier: Run_3
              spec:
                ...

Organize steps and step groups​

You can have multiple step groups in the same stage. Step groups can run in parallel.

Within each step group, you can arrange the steps to run sequentially, in parallel, or a combination.

Running groups or steps in parallel can have significant impacts on resource allocation.

Resource consumption with parallel execution​

When you use self-managed build infrastructures, running steps or step groups in parallel can improve execution time by running multiple sets of tasks at the same time. However, this can also increase the resource consumption of the infrastructure during pipeline execution.

When you use Harness-managed build infrastructures, Harness bills your account based on the compute resources used during each pipeline execution. Running steps or step groups in parallel increases your resource consumption, and your account is billed accordingly. For more information, refer to Harness Cloud billing and build credits.

Diagram showing parallel step execution and resource usage

Step group nesting​

You cannot create a step group inside another step group, but you can configure the steps within a group to run in parallel. You can also run multiple step groups in parallel.

Visual editor showing parallel step groups

Reference step group data​

To reference a step, step setting, variable, output, or other data from a step in a step group, you must include the step group ID in the expression, such as:

<+execution.steps.STEP_GROUP_ID.steps.STEP_ID.SETTING_NAME>
Expression showing step group reference syntax

For example, the following reference calls an output variable named outVar1 that comes from a step with the ID run_script that is inside a step group with the ID StepGroup1:

<+execution.steps.StepGroup1.steps.run_script.output.outputVariables.outVar1>

Referencing outputs from parallel steps​

Steps running in parallel cannot reference output data from each other, since the data might not be available if a step in the group is still running. A step can reference outputs only from other steps that have already finished running.

Diagram showing parallel step output reference limitation

Step group advanced configuration​

You can configure Conditional Execution, Failure Strategy, and Looping Strategy settings for step groups. These settings apply to all steps in the group, unless an individual step has its own settings.

However, if a step within the group has its own step-level Conditional Execution or Failure Strategy, then the individual step's settings override the step group's settings for that single step.

If you configure stage-level Conditional Execution and Failure Strategy settings, then those settings apply to any step groups or steps that do not have their own Conditional Execution and Failure Strategy settings.

Perform the following steps to access Conditional Execution, Failure Strategy, and Looping Strategy settings for step groups:

  1. Select the step group name in the Pipeline Studio.

  2. Select Advanced on the Step Group settings panel.

  3. Configure the required settings.

    Step Group Advanced settings showing Conditional Execution, Failure Strategy, and Looping Strategy options

Next steps​

Last updated on