For the complete documentation index, see llms.txt. This page is also available as Markdown.

Define failure strategies for stages and steps

A failure strategy defines how your stages and steps handle different failure conditions.

A failure strategy defines how steps, stages, and step groups respond to different failure conditions.

A failure strategy consists of error conditions that trigger the strategy and actions to take when those conditions occur.

Failure strategies are a critical component of pipeline design. They determine which failures cause a step or stage to fail and how the pipeline should respond when a failure occurs.

Configure failure strategies

What you will learn from this topic


Before you begin

  • Pipeline access: You need Create or Edit permissions on pipelines to configure failure strategies. For more information, refer to RBAC in Harness.

  • Pipeline basics: You should understand stages and steps in pipelines. For more information, refer to Add a stage.


Configure failure strategies

You can apply failure strategies to:

  • Stages: A stage failure strategy applies to all steps and step groups in the stage that do not have step-level failure strategies.

  • Steps: This failure strategy overrides or enhances the stage failure strategy.

  • Step groups: You can set up a failure strategy for all steps in the group. Individual steps in the group do not have a failure strategy.

  • Pipelines: Pipeline rollback is a failure strategy for all stages in a pipeline. The pipeline rolls back if any of the stages in the pipeline fails.

Add a stage failure strategy

The stage failure strategy applies to all steps in the stage that do not have their own failure strategy configured.

  1. In your pipeline, select the stage where you want to add the failure strategy. For more information, refer to Add a stage.

  2. Select the Advanced tab.

  3. Under Failure Strategy, the default stage failure strategy is shown:

    You cannot remove the default strategy, but you can edit it to choose a different Action, Timeout, and Post timeout action.

  4. To add an additional stage failure strategy, select Add, and then configure the failure strategy settings:

    • On failure of type: Select one or more of the error types to trigger the failure strategy.

    • Perform Action: Select the action that should occur when the specified failure event happens.

    • Timeout and Post timeout action: These are available if you selected Manual Intervention for the Action. The manual intervention action allows a user to intervene and choose an Action when the specified failure event occurs. You can enter a Timeout for the user to select an action, and a Post Timeout Action to fallback on if the user does not manually select an action in a certain amount of time.

    • Retry Count, Retry Intervals, and Post retry failure action: These are available if you selected Retry for the Action. Enter the number of times to retry the stage, the retry interval between attempts, and the Post retry failure action to specify what action to take when all retry attempts have been exhausted and failed. For example, you can configure the stage to rollback or abort after all retries fail.

Stage failure strategy YAML examples

The following examples demonstrate how to configure stage-level failure strategies in YAML.

Stage rollback on all errors

This example shows a deployment stage configured to automatically roll back when any error occurs during stage execution.

Retry stage with rollback on retry failure

This example demonstrates a deployment stage that retries up to 2 times with a 1-hour interval between retries. If all retry attempts fail, the stage performs a rollback.

Add a step failure strategy

Steps do not have a default failure strategy. Instead, steps inherit the stage failure strategy if there is no step-level failure strategy.

When you add a step failure strategy, you override the stage failure strategy for that step.

To add a step failure strategy:

  1. Edit the step where you want to add the failure strategy.

  2. Select the Advanced tab.

  3. Select Failure Strategy, select Add, and then configure the failure strategy settings:

    • On failure of type: Select one or more of the error types to trigger the failure strategy.

    • Perform Action: Select the action that should occur when the specified failure event happens.

    • Timeout and Post timeout action: These are available if you selected Manual Intervention for the Action. The manual intervention action allows a user to intervene and choose an Action when the specified failure event occurs. You can enter a Timeout for the user to select an action, and a Post Timeout Action to fallback on if the user does not manually select an action in a certain amount of time.

    • Retry Count, Retry Intervals, and Post retry failure action: These are available if you selected Retry for the Action. Enter the number of times to retry the step, the retry interval between attempts, and the Post retry failure action to specify what action to take when all retry attempts have been exhausted and failed. For example, you can configure the step to perform a stage rollback or abort after all retries fail.

Step failure strategy YAML examples

The following examples demonstrate how to configure step-level failure strategies in YAML.

Manual intervention on failure

This example shows a Kubernetes rolling deployment step with a failure strategy that triggers manual intervention when any error occurs. If no manual action is taken within 1 hour, the pipeline automatically aborts.

Ignore failure

This example shows a deployment step configured to ignore all errors and continue pipeline execution. In the UI, the step will be marked as success (failure ignored), and the pipeline will proceed to the next step.

Retry with stage rollback on retry failure

This example demonstrates a deployment step that retries up to 2 times with a 1-hour interval between retries. If all retry attempts fail, the stage performs a rollback.

Failure strategies as runtime input

You can also define stage, step, and step group failure strategies at runtime by configuring them as runtime inputs.

To do this, go to the Failure Strategy settings where you want to configure a failure strategy to be specified at runtime, select the Thumbtack icon, and change the input type to Runtime Input.

When you run the pipeline, you will be prompted to define the failure strategy settings for that run.

Due to the potential complexity of failure strategies, input sets are useful for failure strategies as runtime input. Input sets contain pre-defined runtime inputs that you select at runtime. This eliminates the need to manually define the entire failure strategy each time.

Failure strategies for CD steps and stages

For guidance on configuring failure strategies for Continuous Delivery (CD) stages and steps, refer to Define a failure strategy on Harness CD stages and steps.


Failure strategy settings

Error types

The following error types can be selected in a failure strategy.

Error Type

Description

Authentication Errors

Credentials provided in a connector are not valid. Typically, the Harness secret used for one of the credentials is incorrect. If Harness cannot determine if the error is for authentication or authorization, it is treated as an authentication error.

Authorization Errors

The credentials are valid but the user permissions needed to access the resource are not sufficient. If Harness cannot determine if the error is for authentication or authorization, it is treated as an authentication error.

Connectivity Errors

A Harness Delegate cannot connect to a specific resource, such as a repository, VM, or secrets manager.

Delegate Provisioning Errors

No available delegate can accomplish the task, or the task is invalid. For example, if an HTTP step attempts to connect to a URL but there is no available delegate to perform the task.

Timeout Errors

A Harness Delegate fails to complete a task within the stage/step timeout limit. For example, if the Kubernetes workload you are deploying fails to reach a steady state within the step timeout limit.

Unknown Errors

Errors that do not fall into any other category. This includes Harness application errors.

Verification Failures

A Harness Continuous Verification step fails.

Policy Evaluation Failures

An Open Policy Evaluation (OPA) applied on a step fails.

Execution-time Inputs Timeout Errors

A step times out when running a pipeline due to the unavailability of a runtime input.

Approval Rejection

An approval step is rejected. You can select specific failure strategies for approval rejection across steps and stages.

Delegate Restart

An error triggered when the delegate is unreachable when running a pipeline.

User Marked Failure

A failure manually marked by the user during pipeline execution.

All Errors

Any error, whether defined by the other error types or not.

Error scope

The scope of a failure strategy is confined to where it is set.

For example, a failure strategy set on a step does not impact the failure strategy set on a stage. Likewise, the failure strategy set at the stage does not override any failure strategies on its steps.

Rollback stage

Both step and stage failure strategies include the Rollback Stage action option. There is no rollback step option.


Failure strategy actions

The following table lists the failure strategy actions and how they work at the step, step group, and stage levels.

These actions can be applied to the failure strategy as primary action and timeout action.

Action
Step
Step Group
Stage

Manual Intervention

A Harness user can perform a manual intervention when the error type occurs. There are several options to select from: Mark as Success Ignore Failure Retry Abort Rollback StageHarness pauses the pipeline execution when waiting for manual intervention. The pipeline execution state appears as Paused.

Same as step.

Same as step, but applies to all steps.

Mark as Success

The step is marked as Successful and the stage execution continues.

Same as step.

The failed step is marked as Successful and the pipeline execution continues.

Ignore Failure

The stage execution continues. In the UI, the step is marked as success (failure ignored), and rollback is not triggered.

Same as step.

Same as step.

Proceed with Default Values

The step proceeds with default values when the failure occurs. This is typically used when optional inputs or runtime values are not provided.

Same as step.

Same as step.

Retry Step

Harness retries the execution of the failed step automatically. You can set Retry Count and Retry Intervals. You can also configure a Post retry failure action to specify what happens when all retry attempts fail (for example, abort or rollback stage). Additionally, you can define a JEXL condition to retry the step only when a specific condition is met.

Same as step.

Same as step.

Retry Step Group

N/A

Harness will retry the execution of the complete step group automatically, from the beginning. You can set Retry Count and Retry Intervals. You can also configure a Post retry failure action to specify what happens when all retry attempts fail.

N/A

Abort

Pipeline execution is aborted. If you select this option, no timeout is needed.

Same as step.

Same as step.

Rollback Stage

The stage rolls back to the state prior to stage execution. How the stage rolls back depends on the type of build or deployment it was performing.

Same as step.

Same as step.

Rollback Step Group

N/A

The step group rolls back to the state prior to step group execution. How the step group rolls back depends on the type of build or deployment it was performing.

N/A

Mark As Failure

Harness marks the step as Failed.

Harness marks the step group as Failed.

Harness marks the stage as Failed and executes the next stage.

Note

Mark As Failure as a Failure Strategy marks the stage, step group, or step as failed and moves the execution to next step or stage according to when conditions applied on the next step or stage.

Manual interventions

Here is what a Manual Intervention action looks like when a failure occurs:

You can select an Action. If the Manual Intervention exceeds the Timeout setting, Harness automatically selects the Post Timeout Action.

Customize available actions for manual intervention

Currently, this feature is behind the feature flag CDS_MANUAL_INTERVENTION_CUSTOM_ACTIONS. Contact Harness Support to enable the feature.

You can now restrict the set of available actions shown to the pipeline executor during a manual intervention. This applies when a step or stage fails and the configured Failure Strategy is Manual Intervention.

By customizing the available actions, you can:

  • Prevent unsafe actions like Retry or Pipeline Rollback

  • Ensure consistency and control over failure handling

  • Tailor manual intervention choices based on stage or step specific requirements

At pipeline design time, you can whitelist the allowed manual intervention actions using a new Allowed Actions dropdown.

Available Actions

  • Retry Step

  • Mark as Success

  • Mark as Failure

  • Ignore Failure

  • Retry from Stage

  • Rollback Pipeline

The list of available actions may vary slightly based on the stage type (e.g., Deploy, Approval, Custom stage, etc.).

You can also use the checkbox All Actions to allow all available actions.

After selecting the allowed actions, you must also configure the following:

  • Timeout: Duration to wait for manual intervention.

  • Post-Timeout Action: Action to take automatically if no manual decision is made within the timeout.


Prioritization and handling

Failure strategies are evaluated and resolved based on several factors, including the type of failure, the scope at which the strategy is defined, and the conditions under which the failure occurs. The following sections describe how failure strategies are prioritized and handled in different scenarios.

Failure strategies take precedence over conditional executions

Harness pipeline stages and steps can include both conditional executions and failure strategies.

When you use these settings together across multiple stages, it is important to understand how they interact.

For example, consider a pipeline with two stages, stage1 and stage2. Suppose stage2 is configured to Execute this stage only if prior pipeline or stage failed, and stage1 is configured with a Failure Strategy of Rollback Stage for All Errors.

If stage1 encounters an error, it rolls back instead of being marked as failed. As a result, stage2 does not run because its conditional execution is not triggered.

To make stage2 run when stage1 fails, configure stage1 to Ignore Failure instead. This allows the pipeline to continue while still marking stage1 as failed, which triggers the conditional execution of stage2.

Run steps when a stage fails

If you want specific steps to run when a stage fails, add those steps to the stage's Rollback failure strategy.

By default, a rollback stops if an error occurs. If you want a specific step to run even when the rollback encounters an error, configure that step's Conditional Execution to Always and set the preceding step's failure strategy to Mark as failure for All Errors. This ensures that the required step runs even if the preceding step fails.

Stage, step, and step group failure strategy priority

When multiple failure strategies are configured, the more specific strategy takes precedence:

Step → Step Group → Stage

  • A step failure strategy takes precedence over a step group failure strategy.

  • A step group failure strategy takes precedence over a stage failure strategy.

  • The stage failure strategy applies to steps that do not have their own failure strategy.

  • Failure strategies are evaluated in the order the steps execute.

  • If a step does not have its own failure strategy, the failure strategy configured at the step group or stage level is used, depending on where the step belongs.

Multiple failure strategies in a stage

A stage can have multiple failure strategies.

When using multiple failure strategies in a stage, consider the following:

  • For failure strategies that do not overlap (different types of failures selected), they behave as expected.

  • Two failures cannot occur at the same time. Whichever error occurs first, that failure strategy is used.

When you use failure strategies at different levels or configure multiple strategies at the same level, it is important to understand which strategy takes precedence.

Failure strategy conflicts

Failure strategy conflicts can occur when multiple strategies apply to the same failure, either at the same level or at different levels.

Same level

If multiple failure strategies are configured at the same level, the first applicable strategy is used. Any remaining strategies are ignored.

For example, consider these two strategies:

  1. Abort on verification failure or authentication failure.

  2. Ignore on verification failure or connectivity error.

The following behavior occurs:

  • For a verification failure, the first strategy applies, so the stage is aborted.

  • For an authentication failure, the first strategy applies, so the stage is aborted.

  • For a connectivity error, only the second strategy applies, so the error is ignored.

Tip: When multiple strategies at the same level can handle the same error, the strategy listed first takes precedence.

Different levels

Failure strategies can also conflict when they are configured at different levels, such as the step and stage levels.

When both a step-level and stage-level strategy apply to the same failure, the step-level strategy takes precedence and the stage-level strategy is ignored.

In general, failure strategies are evaluated from the most specific level to the least specific level:

Step → Step Group → Stage


Next steps

Last updated

Was this helpful?