> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/continuous-delivery/use-continuous-delivery/cd-building-blocks/cd-steps/step-groups.md). # Group steps using step groups Add steps to a pipeline individually or as a step group, and run the steps serially or in parallel. You can add steps to a Continuous Delivery (CD) pipeline individually or together in a step group. Individual steps and steps in a step group can run serially or in parallel. Unlike an individual step, a step group can apply conditional execution and a failure strategy to every step in the group. You can also run pipeline stages in parallel and deploy multiple services at the same time. Go to [synchronize deployments using barriers](/continuous-delivery/use-continuous-delivery/cd-building-blocks/cd-steps/flow-control/synchronize-deployments-using-barriers.md) to control that flow. This topic describes how to add a step group in a stage. *** ## What you will learn from this topic * How to [run steps in parallel](#run-steps-in-parallel) inside a step group * How to use a [containerized step group](#containerized-step-groups) on your own Kubernetes cluster * How to reference [step group variables](#step-group-variables), [output variables](#output-variables), and [step group expressions](#step-group-expressions) * How to set [conditional execution](#conditional-execution) and a [failure strategy](#step-group-failure-strategy) for every step in the group * How to [reference a step setting](#reference-steps-in-a-step-group) with the step group ID in the expression * How to [override service variables](#override-service-variables-in-step-groups) for the duration of the step group * How to [add and name a step group](#add-and-name-the-step-group), then [add steps](#add-steps-to-the-step-group) so the group runs in the stage *** ## Before you begin Confirm the following before you add a step group: * **Supported stages**: Harness supports step groups in Build, Deploy, and Custom stages only. Go to [step groups](https://developer.harness.io/docs/platform/pipelines/use-step-groups) to review platform step group behavior. * **Kubernetes CD quickstart**: Review [Kubernetes CD quickstart](/continuous-delivery/use-continuous-delivery/deploy-services-on-different-platforms/kubernetes/kubernetes-cd-quickstart.md) before you add step groups. {% hint style="warning" %} **Parallel outputs and delegate selectors** The following limits apply when you configure a step group: * When you run steps in parallel, you cannot reference the output of one step in another step. The output for one step might not be available when another step requests it. * Configure delegate selectors on each step in the step group. You cannot configure a delegate selector at the group level. {% endhint %} *** ## Run steps in parallel You can run steps in parallel in cases such as the following: * Simulate load with multiple [HTTP steps](/continuous-delivery/use-continuous-delivery/cd-building-blocks/cd-steps/utilities/http-step.md). * Run multiple [Verify steps](/continuous-delivery/use-continuous-delivery/verify-deployments/verify-deployments-with-the-verify-step.md) for different providers (AppDynamics, Splunk, Prometheus, and others). * Run independent steps that do not need to run serially. * Run multiple Kubernetes [Apply steps](/continuous-delivery/use-continuous-delivery/deploy-services-on-different-platforms/kubernetes/cd-k8s-ref/kubernetes-apply-step.md) to deploy multiple Kubernetes resources at once. * Delete multiple resources at once. Go to [delete Kubernetes resources](/continuous-delivery/use-continuous-delivery/deploy-services-on-different-platforms/kubernetes/kubernetes-executions/delete-kubernetes-resources.md) to remove them. * Create or update multiple Jira issues with the [Jira Create](/continuous-delivery/use-continuous-delivery/cd-building-blocks/cd-steps/ticketing-systems/create-jira-issues-in-cd-stages.md) and [Jira Update](/continuous-delivery/use-continuous-delivery/cd-building-blocks/cd-steps/ticketing-systems/update-jira-issues-in-cd-stages.md) steps. * Provision multiple resources with Terraform. Go to [provision with the Terraform Apply step](/continuous-delivery/use-continuous-delivery/provision-infrastructure/terraform-infra/run-a-terraform-plan-with-the-terraform-apply-step.md) to configure the Apply step. * Save time when steps are independent. You might have 5 steps and run steps 2 and 3 in parallel because they do not depend on each other. Step 4 runs after those steps complete. *** ## Containerized step groups By default, Harness runs CD step tasks on the Harness Delegate host system, for example, the Kubernetes cluster where a Kubernetes delegate is running. To provide greater control over the resources used for CD steps, you can use your own Kubernetes cluster as the runtime infrastructure for those steps. You can use a CD step group that points to your cluster as the runtime infrastructure. In that step group, add the steps that containerized step groups support. {% hint style="info" %} **Nested step groups** Containerized step groups cannot have nested step groups. You can put groups of steps next to each other in a containerized step group. {% endhint %} The steps inside each group run in parallel, and each group runs serially. ![Step group named Group 1 with HTTP steps HEAD and DELETE side by side, followed by HTTP steps Serial and OPTIONS side by side](https://3694223630-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fy1JhZ4oKIppwY7d5AhPj%2Fuploads%2Fgit-blob-f40df3ecc574e0433161a9eb48ea201e7be2cfc4%2Fstep-groups-04.png?alt=media) Go to [Containerize step groups](/continuous-delivery/use-continuous-delivery/cd-building-blocks/cd-steps/containerized-steps/containerized-step-groups.md) to configure a containerized step group. *** ## Visual summary This short video walks through step groups. {% embed url="" %} Step groups in a pipeline stage {% endembed %} *** ## Add a step group Add a step group in the stage execution, then add the steps that run inside it. Conditional execution and failure strategy on the group apply to those steps unless a step sets its own. ### Add and name the step group Perform the following steps to add and name a step group: 1. In your stage, go to **Execution** -> **Add Step** -> **Add Step Group**. ![Add step menu with the Add Step and Add Step Group options](https://3694223630-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fy1JhZ4oKIppwY7d5AhPj%2Fuploads%2Fgit-blob-ce05eb9def5bf27f4bc9118e39582d91a822d60b%2Fstep-groups-00.png?alt=media) 2. Enter a name for the step group. The step group settings appear. ### Add steps to the step group Perform the following steps to add steps to the step group: 1. In the step group **Execution** section, select **Add Step**. 2. Select a step and configure its **Advanced** settings. A step's **Advanced** settings override the **Advanced** settings of the step group. ### Conditional execution A step group can have its own conditional execution settings, separate from the conditional execution settings for the stage. The conditional execution settings of the step group apply to all of its steps. Go to [Define conditional executions for stages and steps](https://developer.harness.io/docs/platform/pipelines/step-skip-condition-settings) to configure skip conditions. The conditional execution settings of any step in a step group override the conditional execution settings of the step group. If you do not use step group conditional execution settings, Harness uses the stage conditional execution settings. ### Step group failure strategy A step group can have its own failure strategy, separate from the failure strategy for the stage. The failure strategy can execute the rollback steps for the step or stage. Go to [Define a failure strategy on stages and steps](https://developer.harness.io/docs/platform/pipelines/failure-handling/define-a-failure-strategy-on-stages-and-steps) to configure the failure strategy. The failure strategy of any step in a step group overrides the failure strategy of the step group. If you do not use a step group failure strategy, Harness uses the stage failure strategy. *** ## Step group variables Step group variables are custom variables that you can add and refer to in your pipeline. They are available across the pipeline. You can add variables of type string, secret, or number to a step group. Their values can be a fixed value, a runtime input, or an expression. Refer to step group variables within the step group with this expression: `<+execution.steps.[step group id].variables.[variable name]>`. Refer to step group variables outside the step group with this expression: `<+pipeline.stages.[stage Id].spec.execution.steps.[step group id].variables.[variable name]>`. {% hint style="info" %} **Execution input** Execution input is not supported for step group variables. {% endhint %} ### Output variables You can reference step group output variables with expressions that use the step group name. A sample expression is `<+execution.steps..steps.stepID.outputVariables.[variable name]>`. If you access the output variable from a step in the same step group, you can use a shorter notation: `<+stepGroup.steps.step1Identifier.output.outputVariables.[variable name]>`. If the step group is configured as a loop, Harness adds an index at the end of the identifier to identify the unique loop instance. Use this index to reference output variables. For example, `<+execution.steps..steps.stepID.outputVariables.[variable name]>`. ### Step group expressions The following expressions are for step groups in pipeline stages: * `<+stepGroup.variables>`: References a collection of variables within the step group. * `<+stepGroup.getParentStepGroup>`: References a collection of variables from the parent of the step group. *** ## Reference steps in a step group To reference a step in a step group, use the step group ID in the step's fully qualified name (FQN). The following image shows a step group with the ID `Group1`. ![Step Group Configuration tab with Name set to Group 1 and the ID Group1](https://3694223630-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fy1JhZ4oKIppwY7d5AhPj%2Fuploads%2Fgit-blob-fdf683ff3d669baf7e72447379e16f08e641dc7a%2Fstep-groups-03.png?alt=media) The FQN format is: `<+pipeline.stages.[stage Id].spec.execution.steps.[step group Id].steps.[step Id].[step setting name]>` For example, to reference the URL setting in an HTTP step in a step group with the ID `Group1`, the FQN is: `<+pipeline.stages.HTTP.spec.execution.steps.Group1.steps.OPTIONS.spec.url>` *** ## Override service variables in step groups You can customize and override service variables during step group execution. The expression `<+serviceVariableOverrides.VARIABLE_NAME>` overrides the original value of `VARIABLE_NAME`. You can use this expression in values YAML files and in other parts of the pipeline where variables are used. {% hint style="warning" %} **Override scope** An overridden value is available only within the scope of the step group execution. When a child group is included in a parent group, the child group override takes precedence. Step group overrides are not recommended for artifacts and other objects fetched in the **Service** section of a pipeline stage. The one exception is scripts in [custom remote manifests](/continuous-delivery/use-continuous-delivery/deploy-services-on-different-platforms/kubernetes/cd-kubernetes-category/add-a-custom-remote-script-and-manifests.md), because they can also be used during the **Execution** section of a pipeline stage. The pipeline overrides variables in the following order: 1. Step group override (child group, during execution) 2. Step group override (parent group, during execution) 3. Service environment overrides 4. Environment configuration 5. Service variable value (default) {% endhint %}
Pipeline example: overriding variables in step groups The following pipeline illustrates how to override service variables. The service definition has two defined variables, `svar1` and `svar2`. The pipeline overrides these variables as follows: * The `parent` step group overrides `svar1` with the value `fromStepGroup_OverriddenAtParentStepGroup`. * The `parent` step group also has a variable `svarParent` with the value `DefinedAtParentLevel`. The `child` step group overrides this with the value `OverridenByChildStepGroup`. * The `child` step group overrides `svar2` with the value `fromStepGroup_OverriddenAtChildLevel`. ![Execution graph of parent and child step groups next to Shell Script logs that print the overridden variable values](https://3694223630-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fy1JhZ4oKIppwY7d5AhPj%2Fuploads%2Fgit-blob-d5f8ae7a27ee60dde07aeb309dc23bf46e770354%2Foverride-service-variables-pipeline-example.png?alt=media) ```yaml pipeline: projectIdentifier: svcredesignhinger orgIdentifier: harness tags: {} stages: - stage: name: dep identifier: dep description: "" type: Deployment spec: deploymentType: Kubernetes service: serviceRef: TestStepGroupOverridesService environment: environmentRef: qa environmentInputs: identifier: qa type: PreProduction variables: - name: rtvar1123 type: String value: <+input> deployToAll: false infrastructureDefinitions: - identifier: delegateInfra2 execution: steps: - stepGroup: name: parent identifier: parent steps: - step: type: ShellScript name: ShellScript_2 identifier: ShellScript_2 spec: shell: Bash onDelegate: true source: type: Inline spec: script: echo <+serviceVariableOverrides.svarParent> environmentVariables: [] outputVariables: [] timeout: 10m - stepGroup: name: child identifier: child steps: - step: type: ShellScript name: ShellScript_1 identifier: ShellScript_1 spec: shell: Bash onDelegate: true source: type: Inline spec: script: |+ echo <+serviceVariableOverrides.svar1> echo <+serviceVariableOverrides.svar2> echo "child overriding parent value" echo <+serviceVariableOverrides.svarParent> environmentVariables: [] outputVariables: [] timeout: 10m variables: - name: svar2 type: String value: fromStepGroup_OverriddenAtChildLevel description: "" required: false - name: svarParent type: String value: OverridenByChildStepGroup description: "" required: false variables: - name: svar1 type: String value: fromStepGroup_OverriddenAtParentStepGroup description: "" required: false - name: svarParent type: String value: DefinedAtParentLevel description: "" required: false rollbackSteps: [] tags: {} failureStrategies: - onFailure: errors: - AllErrors action: type: StageRollback identifier: StepGroupLevelOverrides name: StepGroupLevelOverrides ```
*** ## Next steps * [Containerize step groups](/continuous-delivery/use-continuous-delivery/cd-building-blocks/cd-steps/containerized-steps/containerized-step-groups.md): Run CD steps on a Kubernetes cluster that you manage. * [Synchronize deployments using barriers](/continuous-delivery/use-continuous-delivery/cd-building-blocks/cd-steps/flow-control/synchronize-deployments-using-barriers.md): Coordinate stages that run in parallel. * [Define conditional executions for stages and steps](https://developer.harness.io/docs/platform/pipelines/step-skip-condition-settings): Set skip conditions for a step group or for an individual step. * [Define a failure strategy on stages and steps](https://developer.harness.io/docs/platform/pipelines/failure-handling/define-a-failure-strategy-on-stages-and-steps): Choose what the step group does when a step fails. {% @harness-feedback/feedback %}