Insert step and stage in existing Template

It allows you to insert step and stage in existing templates wihthout a need to create a new version of the template.

NOTE

Currently this feature is behind the feature flag PIE_FLEXIBLE_TEMPLATES and PIE_FLEXIBLE_TEMPLATES_PHASE2. Contact Harness support to enable this feature.

Nesting a stage template with an insert step inside an insert stage, described in Nest a stage template with an insert step inside an insert stage, is available to any account that already has these flags enabled. It does not require a separate feature flag.

Insert blocks provide a way to customize pipelines without affecting the main template.

Steps and stages included in the insert block will behave the same as normal steps and stages in the pipeline.

NOTE

Insert block is supported for CI, CD, Custom and Approval Stages.

Pros of using insert blocks in a template

Insert blocks give template editors and template users the following advantages:

• Only the Template Editor has the flexibility to allow additional steps or stages at any given point where they want.(At beginning of all steps/stage or at the end of all steps/stages or in between the steps/stages)

• The YAML is simple and inline with existing Harness steps/stages YAML. Here the Insert is simply a new type of step which starts with the key insert.

Now, let's dive into who can add insert block in the pipeline and stage template and how other users can utilise it in their pipelines.

Insert stage block in pipeline template

Template editors will be able to add insert stage block in the pipeline template at any position between a stage.

Sample YAML of a pipeline template with insert stage block will look like:

template:
  name: pipeline_1
  identifier: pipeline_1
  versionLabel: v2
  type: Pipeline
  projectIdentifier: Krishika_test_autocreation
  orgIdentifier: default
  tags: {}
  spec:
    stages:
      - stage:
          name: custom_2
          identifier: custom_2
          description: ""
          type: Custom
          spec:
            execution:
              steps:
                - step:
                    type: ShellScript
                    name: ShellScript_1
                    identifier: ShellScript_1
                    spec:
                      shell: Bash
                      executionTarget: {}
                      source:
                        type: Inline
                        spec:
                          script: echo hello_2
                      environmentVariables: []
                      outputVariables: []
                    timeout: 10m
          tags: {}
      - insert:
          name: insert_2
          identifier: insert_2
          stages: <+input>
      - stage:
          name: stage_3
          identifier: stage_3
          tags: {}
          template:
            templateRef: stage_1
            versionLabel: v2
            templateInputs:
              type: Custom
              spec:
                execution:
                  steps:
                    - parallel:
                        - insert:
                            identifier: insert_1
                            steps: <+input>

Insert step block in stage template

Similarly, as a Template Editor you can add a insert step block in the stage template at any position between a step.

Sample YAML of a stage template with an insert step block will look like:-

template:
  name: stage_1
  identifier: stage_1
  versionLabel: v2
  type: Stage
  projectIdentifier: Krishika_test_autocreation
  orgIdentifier: default
  tags: {}
  spec:
    type: Custom
    spec:
      execution:
        steps:
          - parallel:
              - step:
                  type: ShellScript
                  name: ShellScript_1
                  identifier: ShellScript_1
                  spec:
                    shell: Bash
                    executionTarget: {}
                    source:
                      type: Inline
                      spec:
                        script: |
                          echo hello
                    environmentVariables: []
                    outputVariables: []
                  timeout: 10m
              - insert:
                  name: insert_1
                  identifier: insert_1
                  steps: <+input>

This allows you, as the template editor, to maintain control over the template, ensuring its integrity is preserved.

Now, if you use a template with a insert step/stage block in a pipeline, suppose you are using a pipeline template while creating a pipeline in a pipeline studio those insert stages will come under templateInputs.

Sample YAML:

pipeline:
  name: pipeline_insert_sample
  identifier: pipeline_insert_sample
  tags: {}
  template:
    templateRef: pipeline_insert_template
    versionLabel: v2
    templateInputs:
      stages:
        - insert:
            identifier: insertStages1
            stages: <+input>
        - insert:
            identifier: insertStages2
            stages: <+input>
  projectIdentifier: Insert_block
  orgIdentifier: default

In the above YAML as you can see, we have used pipeline template pipeline_insert_template which are having two insert blocks and those insert blocks are under templateInputs.

Template user can add additional step and stage wherever an insert block has been defined. The insert block support inclusion of stages and steps along with runtime inputs, failure strategies, and conditional execution.

Consider a YAML using stage template in a pipeline with an insert step block:-

pipeline:
  name: pipeline_sample
  identifier: pipeline_sample
  projectIdentifier: Krishika_test_autocreation
  orgIdentifier: default
  tags: {}
  stages:
    - stage:
        name: stage_1
        identifier: stage_1
        tags: {}
        template:
          templateRef: stage_insert_template
          versionLabel: v2
          templateInputs:
            type: Custom
            spec:
              execution:
                steps:
                  - insert:
                      identifier: insertSteps1
                      steps:
                        - parallel:
                            - step:
                                identifier: shell1
                                type: ShellScript
                                name: shell1
                                spec:
                                  shell: Bash
                                  executionTarget: {}
                                  source:
                                    type: Inline
                                    spec:
                                      script: echo hello_3
                                  environmentVariables: []
                                  outputVariables: []
                                timeout: 10m
                                failureStrategies:
                                  - onFailure:
                                      errors:
                                        - AllErrors
                                      action:
                                        type: Ignore
                  - insert:
                      identifier: insertSteps2
                      steps: <+input>

In this, under the first insert block we have added one Shell Script step. Now, when we run the pipeline the execution will look like :-

If no actions are provided in the insert block the pipeline will proceed without any additional steps and stages.

For example, in the below yaml, we have used this stage template in the pipeline with 2 insert blocks and we have not added any additional steps in it:-

pipeline:
  name: pipeline_insert_sample
  identifier: pipeline_insert_sample
  projectIdentifier: Krishika_test_autocreation
  orgIdentifier: default
  tags: {}
  stages:
    - stage:
        name: stage_1
        identifier: stage_1
        tags: {}
        template:
          templateRef: stage_insert_template
          versionLabel: v2
          templateInputs:
            type: Custom
            spec:
              execution:
                steps:
                  - insert:
                      identifier: insertSteps1
                      steps: <+input>
                  - insert:
                      identifier: insertSteps2
                      steps: <+input>

Now when we will run the pipeline the execution will look like:-

If you will check the compiled YAML it will show the steps input as empty and thus will not fail the pipeline as well with a null error.

Nest a stage template with an insert step inside an insert stage

Harness supports one additional level of insert nesting: a pipeline template with an insert stage block can accept a stage template that itself contains an insert step block. This lets a template user insert a stage built from a stage template, and still add their own steps inside that inserted stage, without the pipeline template author having to anticipate every team's steps in advance.

The nesting is limited to this single extra level. An insert block still cannot contain another insert block of the same kind, so a stage inserted at this second level cannot itself declare another insert stage, and a step inserted at this second level cannot itself declare another insert step.

Consider a stage template with its own insert step block:

YAML example: stage template with an insert step block

template:
  name: stage_insert_template
  identifier: stage_insert_template
  versionLabel: v2
  type: Stage
  projectIdentifier: Krishika_test_autocreation
  orgIdentifier: default
  tags: {}
  spec:
    type: Custom
    spec:
      execution:
        steps:
          - step:
              type: ShellScript
              name: ShellScript_1
              identifier: ShellScript_1
              spec:
                shell: Bash
                executionTarget: {}
                source:
                  type: Inline
                  spec:
                    script: echo hello  # Replace with your script content
                environmentVariables: []
                outputVariables: []
              timeout: 10m
          - insert:
              name: insertSteps1        # Replace with a name for the insert block
              identifier: insertSteps1
              steps: <+input>

Now consider a pipeline template with an insert stage block, used in a pipeline where the inserted stage comes from this stage template:

YAML example: pipeline that inserts a stage template with an insert step

pipeline:
  name: pipeline_sample
  identifier: pipeline_sample
  projectIdentifier: Krishika_test_autocreation
  orgIdentifier: default
  tags: {}
  template:
    templateRef: pipeline_insert_template
    versionLabel: v2
    templateInputs:
      stages:
        - insert:
            identifier: insertStages1
            stages:
              - stage:
                  name: custom_stage
                  identifier: custom_stage
                  template:
                    templateRef: stage_insert_template
                    versionLabel: v2
                    templateInputs:
                      type: Custom
                      spec:
                        execution:
                          steps:
                            - insert:
                                identifier: insertSteps1
                                steps:
                                  - step:
                                      type: ShellScript
                                      name: ShellScript_1
                                      identifier: ShellScript_1
                                      spec:
                                        shell: Bash
                                        source:
                                          type: Inline
                                          spec:
                                            script: echo hello

In this example, insertStages1 (from the pipeline template) is filled with custom_stage, which references stage_insert_template. That stage template has its own insertSteps1 block, which the template user fills with a Shell Script step. This is the deepest level of nesting Harness supports: a step insert nested inside a stage insert.

In the Pipeline Studio, this looks like a stage insert whose stage contains a step insert of its own:

A stage insert filled with a stage that contains its own step insert, shown alongside the step's execution graph

When you pick a template to fill an insert block in the Pipeline Studio, Harness disables Use Template and shows a tooltip if using that template at this position would exceed the supported nesting, for example if the template itself contains an insert stage or step at a level that would create a third level of nesting.

If you write YAML directly, you can create a pipeline with unsupported nesting, since backend validation does not block saving it. Harness flags this while the pipeline is being saved or opened, in a Pipeline validation failed dialog with the message This pipeline has unsupported nesting of inserts under Template issues. Running such a pipeline fails, so fix the reported nesting before you try to run it.

Pipeline validation failed dialog showing "This pipeline has unsupported nesting of inserts" under Template issues

Limitations

Keep the following limitations in mind when you use insert blocks:

1. Insert block can not be output of any step, it has to be provided.

2. Nesting an insert block inside another insert block of the same kind is not allowed. A step insert cannot contain another step insert, and a stage insert cannot contain another stage insert. Go to Nest a stage template with an insert step inside an insert stage for the one level of nesting Harness does support.

3. Insert blocks cannot be added in parallel to any other stage.

4. In the step group template, insert step cannot be added.

5. Only service, environment, and infrastructure definitions can be propagated within an insert block; they cannot be propagated outside the insert block for other stages that are not part of it.

Demo video

Expressions

If we intend to utilize expressions for the properties within the insert, it will be necessary to specify the complete path for each one.

Example: <+execution.steps.insert1.steps.ShellScript_1.description>

RBAC required

Using insert blocks requires the following permissions:

• Users must possess the Template Create/Edit Permission in order to insert an insert block into the template at any desired location.

• In order to provide the steps/stages input to insert block when specifying runtime inputs in the pipeline, users must have Pipeline Create/Edit Permission. Otherwise, if they intend to provide input values in the parent template, Template Create/Edit Permission will be required.

Next steps

• Go to Override template advanced settings to let callers override a controlled set of a template's advanced settings.

• Go to Reconcile pipeline template changes to reconcile pipelines after a template change.