Skip to main content

DORA profile

The DORA profile in Harness SEI sets the definition for measuring the DORA metrics across your organization. In this topic we cover how you can create and set up the DORA profile and configure the definition for the DORA metrics across various combination of tools.

Create DORA profiles​

To add or edit Workflow profiles:

  • In your Harness project, go to the SEI module.
  • Select Account Management.
  • Select Workflow under Profiles.
  • To create a profile, select +New Workflow Profile. To edit a profile, select the profile's name in the profiles list.

  • Choose DORA Profile as the Workflow profile type.

  • Enter a Name for the profile.
  • Add a profile description. (Optional)

tip

You can create profiles by copying existing profiles. Make sure to edit copies accordingly and that your Lead Time widgets reference the correct profile.

Set up Lead Time for Changes​

Lead Time for Changes measures how long it takes for a task to move from development to production. This involves defining stages in your workflow that reflect your software delivery process.

Configuration steps​

  • Choose your ticketing platform: Select the issue management system (e.g., Jira) that your team uses to track tasks like new features, stories, or epics.
  • Select the Starting event: Define when Lead Time tracking begins:
    • Ticket Created: Starts tracking when a ticket is created in the issue management system.
    • Commit Created: Starts tracking when the first commit is made.
    • API Event: Uses a custom API event to trigger Lead Time calculation.
  • Define Workflow Stages:
    • Add stages that represent key phases in your delivery process, such as issue management and CI/CD activities.
    • Default development stages (tracked via your SCM) are by default configured. You can enable or disable these as needed.

Here's an example configuration of how you could configure Lead Time across various tools.

This section covers how to set up the Lead Time for Changes metric definition using only an issue management tool like Jira.

  • Select the Issue Management Platform that you want to use to track tasks in your team.

  • Select Ticket Created as the start event. This triggers Lead Time tracking whenever a new ticket is created.

  • Disable development stages as this is a issue management specific definition.

  • Add custom stages that align with your Jira workflow:
    • Click the + button within the workflow.

  • Add the stage name and description.

  • Define the Stage Trigger Event (e.g., Issue Management) and map the stage to statuses in your issue management system.

  • Set acceptable time ranges for the stage, such as Ideal Time and Acceptable Time.

  • Create a series of custom stages to reflect your entire delivery process within the issue management system.

Here’s how a fully configured Issue Management-only DORA Lead Time for Changes definition might look:

Set up Deployment Frequency​

Deployment Frequency measures how frequently a team successfully deploys code to production.

Configuration steps​

  • Select the tool your team uses to track and measure deployments.
  • Select any existing integrations you wish to use for calculating deployment frequency.
  • Defind the settings for how you want to calculate deployment frequency. The additional filters being used to define the deployments will be applicable to all the integrations that you selected.
RECOMMENDATION

For accurate tracking, use CI/CD tools (e.g., Harness CD, GitHub Actions) or ITSM tools (e.g., ServiceNow) that provide detailed deployment and change request tracking data.

Here's an example configuration of how you could configure Deployment Frequency across various tools.

  • Choose Harness CD as the tool that you use to measure a deployment in your team.

  • Select all the integrations that you would like to use to calculate the deployment frequency.

  • Select the category of pipelines ad Continuous Delivery

  • Configure the additional attributes to calculate the deployment frequency. By default, all pipelines are included in the calculation, but you can customize the selection by manually selecting the pipelines.

  • Refine the deployment frequency calculation by specifying detailed filters. This filters pipelines that should contribute to the deployment frequency calculations.
    • Pipeline Filters:
      • Use filters to include or exclude pipelines based on specific properties, conditions, and values from the Harness CD pipeline configuration.
      • Filters combine using an AND operation, meaning all specified conditions must be met.
      • Example: Status = Success AND Environment = Production.

  • Execution Filters:
    • Narrow down the data to specific pipeline executions by defining key-value pairs.
    • Filters combine using an OR operation, meaning any execution meeting the specified conditions will be included.
    • Example: Include executions where JAVA_VERSION = 11 OR Branch = main.

  • Stage Parameter Filters:
    • Apply filters at the stage level within pipelines to focus on specific steps in the deployment process.
    • Filters combine using an OR operation, allowing flexibility to include stages that meet any of the conditions.

  • Define how deployment frequency should be tracked:
    • Pipelines Started: Tracks deployment frequency based on the initiation of deployment pipelines.
    • Pipelines Completed: Tracks frequency based on successfully completed deployments.

Set up Mean Time to Restore​

Mean Time to Restore represents the duration it takes a team to recover from a production failure.

Configuration steps​

  • Choose the tool used for measuring the incident recovery time in your team.
  • Configure the stages/filters to identify incident tickets based on the requirements.
RECOMMENDATION

For accurate tracking, use ITSM tools (i.e. ServiceNow or PagerDuty) that provide detailed incident tracking data.

Here's an example configuration of how you could configure Mean Time to Restore across various tools.

  • Choose ServiceNow as the tool that you use to measure mean time to restore in your team.

  • Select the associated ServiceNow integration. To learn about how to configure the integration, go to ServiceNow integration.

  • Select the ticket type for mean time to restore tracking:
    • Incidents is the recommended option for measuring DORA MTTR.
    • You can alternatively select Change Requests, though this is not recommended for MTTR tracking.

  • Define incident criteria for calculating MTTR by selecting various filters that define incidents you want to track or measure for MTTR calculations.
    • The DORA profile definition supports all the ServiceNow fields including Priority, Urgency, Status etc to be configured as filters. When configuring the filters, the custom fields available dynamically changes based on the selected ticket type.
    • Filters are combined using an AND operation, meaning all specified conditions must be met.
    • Example: STATUS EQUALS CLOSED AND CATEGORY EQUALS = <CATEGORY_VALUES>.

  • Define how mean time to restore should be tracked:
    • Incident Resolved: Tracks mean time to restore based on when incidents are resolved (e.g., status changes to Closed).
    • Incident Updated: Tracks mean time to restore whenever incidents are updated (e.g., fields like resolution date or status are modified).
    • Incident Created: Tracks mean time to restore based on the creation of new incidents.

Set up Change Failure Rate​

Change Failure Rate is computed by dividing the total number of deployments causing failure by the overall number of deployments.

The Change Failure Rate (CFR) is calculated by dividing the total number of deployments that cause a failure by the total number of deployments.

Change Failure Rate = Deployments causing failure / Total deployments

By default, it is recommended to calculate CFR using the total number of deployments as the denominator, aligning with industry standards. However, you also have the flexibility to calculate CFR focusing solely on deployments that result in failure, based on your specific needs.

Configuration settings​

  • Specify the tool your team uses to measure Change Failure Rate.
  • Choose any existing integrations you want to utilize for calculating the change failure rate.
  • Add attributes and filters to identify and define both the total deployments and the deployments causing failure.
RECOMMENDATION

For accurate tracking, use CI/CD tools (e.g., Harness CD, GitHub Actions) or ITSM tools (e.g., ServiceNow) that provide detailed deployment and incident tracking data.

Here's an example of how you could configure CFR across various tools.

Define the deployments causing failure​

  • Choose Harness CD as the tool to identify failed deployments within your team.

  • Select the integrations you want to use for calculating CFR.

  • Select the category of pipelines as Continuous Delivery

  • Configure additional attributes to identify deployments that result in failure. By default, all pipelines are included, but you can customize the selection by manually specifying which pipelines should be included.

  • Refine the calculation by specifying detailed filters. This filters pipelines that should contribute to the change failure rate calculations.
    • Pipeline Filters:
      • Use filters to include or exclude pipelines based on specific properties, conditions, and values from the Harness CD pipeline configuration.
      • Filters combine using an AND operation, meaning all specified conditions must be met.
      • Example: Status = Failed AND Environment = Production.

  • Execution Filters:
    • Narrow down the data to specific pipeline executions by defining key-value pairs.
    • Filters combine using an OR operation, meaning any execution meeting the specified conditions will be included.
    • Example: Include executions where JAVA_VERSION = 11 OR Branch = main.

  • Stage Parameter Filters:
    • Apply filters at the stage level within pipelines to focus on specific steps in the deployment process.
    • Filters combine using an OR operation, allowing flexibility to include stages that meet any of the conditions.

  • Define how deployment causing failure should be tracked:
    • Pipelines Started: Tracks change failure rate based on the initiation of deployment pipelines.
    • Pipelines Completed: Tracks change failure rate based on completed pipelines.

Define the total deployments​

Total deployments represent all deployments that have occurred within a specified time range, regardless of whether they resulted in success or failure.

Similar to the deployments causing failure definition, you can configure the definition for measuring total deployments. This definition may align with the DORA Deployment Frequency definition. However, note that the total deployments are not linked with Deployment Frequency and are tracked separately for Change Failure Rate calculations.

Associate profile with Collection​

Associate the DORA profile with the Collection and Project under which you have set up the DORA Insight.

Once you have finished configuring the profile setting click on Save to save the profile.

Note that you can also edit existing Collections and associate them with the DORA profile if required.

tip

Note that for calculating DORA metrics, each profile can be associated with only one Collection in a one-to-one mapping

You can also associate Collections to existing DORA profiles from the Collections Tab. Collection categories are shown as tabs on the Collection Settings page.

  • To associate a DORA profile with the existing Collections, click on the Associate Workflow Profile option under the Associated Profiles column.

  • Select the DORA profile from the available options. The pop-up dialog box will display the list of all the profiles that can be associated with the current collection.

  • Select the DORA profile and click on the Associate Profile button.

Common set up issues​

While configuring the DORA profile, you may encounter some common issues. Below are the detailed explanations and troubleshooting steps to resolve them:

Missing or unavailable filters​

Possible Causes​

  • Data Ingestion Not Completed: If the selected integration has not yet completed data ingestion, filters may not appear in the settings.
  • Field Not Ingested: The required field might not be included as part of the integration ingestion process.

Resolution Steps​

  • Check Diagnostics: Navigate to the Diagnostics section and verify whether data is ingested for the selected integrations. If data is not yet available, wait for the ingestion process to complete.
  • Request Field Ingestion: If the field is not ingested, create a Harness Support ticket and request the ingestion of the specific field as part of the integration.

Values for selected filters don't populate​

Possible Cause​

  • Data ingestion for the selected integration may still be in progress or incomplete.

Resolution Steps​

  • Verify Data Availability: Use the Diagnostics section to confirm whether the data has been successfully ingested. If the data is not present, wait until ingestion is complete to configure the filters.

Projects/Collections not available in the association settings​

Possible Causes​

  • Existing DORA Profile Association: The collection might already be associated with another DORA profile, as each collection can only be linked to one DORA-type workflow profile. A mismatch between collections and the intended DORA profile can result in the collection not appearing in the association settings.

Resolution Steps​

  • Verify Collection Settings: Check the collection settings to ensure the collection is correctly associated with the desired DORA profile. Remember, while a single DORA profile can be associated with multiple collections, each collection can only be linked to one DORA-type workflow profile. Set up the associations accordingly.

What's next​

After setting up the DORA profile, proceed to create the DORA Insight using the available DORA widgets. These widgets enable you to visualize and monitor key DORA metrics, providing actionable insights into your team’s performance.