Skip to main content

Create OPA policies to stop STO pipelines automatically

You can use Harness Policy as Code to write and enforce policies against your security tests, and to stop your pipelines if a security test has any issues that violate those policies.

You can use Harness Policy as Code to enforce policies such as:

  • A security test cannot include any issues in a list of severities such as Critical or New Critical.

  • A security test cannot include any issues for CVEs past a certain age, for example no critical-severity CVEs more than three years old.

  • A security test cannot include any issues in a list of titles such as libsqlite3 or javascript.express.security.audit.

  • A security test cannot include any more than 75 occurrences of TAR-related issues (issue title matches regex ".*tar.*").

  • A security test cannot include any issues in a list of reference IDs such as CWE-78 or CVE-2023-52138.

Important notes​

Security Test policy samples​

The Harness Policy Library includes the following policy samples that make it easy to create security test policies and enforce them against your scan results.

Exclude vulnerabilities by severity​

Apply a policy to a scan step to warn or block on any vulnerabilities with the specified severity.

You must copy the entire sample code from the OPA policy library, as described in Create a new Security Tests OPA policy.

Here is a sample policy that you can evaluate using the On Step event for a scan step.

note

This policy sample supports the following vulnerabilities only: Critical, High, Medium, Low, and Info. To create policies based on output variables such as NEW_CRITICAL, go to Exclude vulnerabilities using STO output variables.


package securityTests

import future.keywords.in
import future.keywords.if

# Define a set of severities that are denied (Critical, High, Medium, Low, Info)
# The following example denies if the scan results include any issue with a severity of Critical or High.

deny_list := fill_defaults([
  {
    "severity": {"value": "Critical", "operator": "=="}
  },
  {
    "severity": {"value": "High", "operator": "=="}
  }
])

Exclude vulnerabilities by reference ID​

Apply a policy to a scan step to warn or block on any vulnerabilities in a specific list of CVEs or CWEs.

You must copy the entire sample code from the OPA policy library, as described in Create a new Security Tests OPA policy.

Here is a sample policy that you can evaluate using the On Step event for a scan step.


package securityTests

import future.keywords.in
import future.keywords.if

# Define a set of reference-identifiers that are denied
# The following policy denies if the scan results include any occurrence of 
#  - cwe-772
#  - cve-2019-14250
#  - CWE-772
#  - CVE-2019-14250

deny_list := fill_defaults([
  {
    "refId": {"value": "772", "operator": "=="},
    "refType": {"value": "cwe", "operator": "=="}
  },
     {
    "refId": {"value": "772", "operator": "=="},
    "refType": {"value": "CWE", "operator": "=="}
  },
  {
    "refId": {"value": "2019-14250", "operator": "=="},
    "refType": {"value": "cve", "operator": "=="}
  },
 {
    "refId": {"value": "2019-14250", "operator": "=="},
    "refType": {"value": "CVE", "operator": "=="}
  }
])

Exclude vulnerabilities by title​

Apply a policy to a scan step to warn or block on any vulnerabilities in a specific list of issue titles.

You must copy the entire sample code from the OPA policy library, as described in Create a new Security Tests OPA policy.

You can use the ~ operator to find titles based on Python regular expressions.

Here is a sample policy that you can evaluate using the On Step event for a scan step.


package securityTests

import future.keywords.in
import future.keywords.if

# Define a set of titles that are denied
# The following example denies if the scan results include any issues related to `tar@1.34` or `libsqlite3`

deny_list := fill_defaults([
  {
    "title": {"value": "tar@1.34", "operator": "~"}
  },
  {
    "title": {"value": "libsqlite3", "operator": "~"}
  }
])

Exclude vulnerabilities by number of occurrences​

Apply a policy to a scan step to warn or block vulnerabilities based on a set of titles and the maximum allowed number of occurrences for each vulnerability.

You must copy the entire sample code from the OPA policy library, as described in Create a new Security Tests OPA policy.

You can use the ~ operator to find titles based on Python regular expressions.

Here is a sample policy that you can evaluate using the On Step event for a scan step.


package securityTests

import future.keywords.in
import future.keywords.if

# Define a set of titles and maximum occurrences that are denied
# The following example denies on scan results with more than 25 occurrences of TAR- or cURL-related issues

deny_list := fill_defaults([
  {
    "title": {"value": ".*tar.*", "operator": "~"},
    "maxOccurrences": {"value": 25, "operator": ">="},
  },
  {
    "title": {"value": ".*curl.*", "operator": "~"},
    "maxOccurrences": {"value": 25, "operator": ">="},
  }
])

Exclude vulnerabilities by CVE age​

Apply a policy to a scan step to warn or block vulnerabilities based on CVEs by severity and age.

You must copy the entire sample code from the OPA policy library, as described in Create a new Security Tests OPA policy.

Here is a sample policy that you can evaluate using the On Step event for a scan step.


package securityTests

import future.keywords.in
import future.keywords.if

# Define a set of CVE ages (as old/older than given year) and severities (equal/greater than) that are denied
# This example denies CVEs for any of the following filters:
#   - Critical severities, new (2021 or earlier)
#   - High severities, old (2018 or earlier)
#   - Medium severities, very old (2015 or earlier)

deny_list := fill_defaults([
  {
    "year": {"value": 2023, "operator": "<="},
    "severity": {"value": "Critical", "operator": "=="}
  },
    {
    "year": {"value": 2018, "operator": ">="},
    "severity": {"value": "High", "operator": "=="}
  },
  {
    "year": {"value": 214, "operator": "<="},
    "severity": {"value": "Medium", "operator": "=="}
  }
])

Exclude vulnerabilities using STO output variables​

You can create policies based on the output variables generated by an STO scan step.

For example, suppose you want a policy to warn or block if a scan step finds any new vulnerabilities with severities of Critical or High. In this case, you can create a policy with the following OPA code:

 package pipeline_environment

 # Warn or block if the scan step detects any NEW_CRITICAL or NEW_HIGH vulnerabilities 

deny[sprintf("Scan can't contain any NEW_CRITICAL vulnerability '%s'", [input[_].outcome.outputVariables.NEW_CRITICAL])] {
    input[_].outcome.outputVariables.NEW_CRITICAL != "0"
}

deny[sprintf("Scan can't contain any high vulnerability '%s'", [input[_].outcome.outputVariables.NEW_HIGH])] {
    input[_].outcome.outputVariables.NEW_HIGH != "0"
}

Block the pipeline based on the code coverage results​

Apply a policy to the scan step to either warn or block the pipeline based on the code coverage value. You can use the sample policy Security Test - Code Coverage. Below is a sample policy for reference:

package securityTests

import future.keywords.in
import future.keywords.if

# Define a set of Output Variables that are denied
deny_list :=([
# Fail if CODE_COVERAGE is less than 50.0
  {
    "name": "CODE_COVERAGE", "value": 50.0, "operator": "<"
  },
# Optionally define more Output Variables here
#  {
#    "name": "HIGH", "value": 0, "operator": ">"
#  }
])

Workflow description​

The following steps describes the end-to-end workflow:

  1. Create your policies using Security test policy samples.

  2. Create a policy set with the policies you want to enforce.

  3. Enforce the policy set in your scan step.

Create a new Security Tests OPA policy​

  1. You can create policies at the account or the project scope. Go to your account or project, then select Security and Governance > Policies.

  2. Select Policies (top right) and then New Policy.

  3. Select a Security Tests policy from the Policy samples library.

  4. Select Use this sample (bottom). This copies the entire policy sample to the edit pane (left).

  5. Configure the policy as needed. In this example, the policy excludes vulnerabilities with a severity of Critical.

  6. Test your policy to verify that it works as intended.

    Each policy sample includes a set of test data that you can use. In the Testing Terminal, examine the test data and edit it as needed. Then click Test to verify the results.

    It is good practice to test both a Success and Failure case for your policy. The following example illustrates this workflow.

    In this example, the policy denies on reference ID CWE-1230. In this case, you would do the following:

    1. Search the test results for the string 1230. In this case, the ID is not found.

    2. Click Test. The test succeeds.

    3. Search the test results for the string cwe and edit an entry so it matches the reference ID.

    4. Click Test again. The test fails because the data includes the specified CWE.

  7. Once you're satisfied that the policy works as intended, save it.

Create a policy set​

A policy set is a collection of one or more policies. You combine policies into a set and then include it in a scan step.

  1. Go to Security and Governance > Policies. Then click Policy Sets (top right) and then New Policy Set.

  2. Click New Policy Set. The Policy Set wizard appears.

  3. Overview:

    1. Name — Enter a descriptive name such as myorg/myimage policies.

    2. Entity type this policy applies to = Security Tests

    3. On what event should the policy be set to = On Step

      These settings allow you to apply the member policies to a specific step, which you'll define below.

  4. Policy evaluation criteria:

    1. Click Add Policy.

    2. Select the policy you just created and set the pull-down to Error and Exit. This is the action to take if any policies in the set are violated.

    3. Click Apply to add the policy to the set, then Finish to close the Policy Set wizard.

  5. ❗ In the Policy Sets page, enable Enforced for your new policy set.

Enforce the policy in your scan step​

Now you can set up your scan step to stop builds automatically when the policy gets violated.

  1. Go to the scan step and click Advanced.

  2. Under Policy Enforcement, click Add/Modify Policy Set and add the policy set you just created.

  3. Click Apply Changes and then save the updated pipeline.

Set up email notifications for pipeline failures​

You have a Policy that fails the pipeline based on an OPA policy. Now you can configure the stage to send an email notification automatically whenever the pipeline fails.

  1. Click Notifications (right-side menu). The New Notification wizard appears.

  2. Set up the notification as follows:

    1. Overview page — Enter a notification name such as Pipeline failed -- NEW_CRITICAL or NEW_HIGH issues detected.

    2. Pipeline Events page — Select Stage Failed for the event that triggers the notification. Then select the stage that has the Policy step you just created.

    3. Notification Method page — Specify Email for the method and specify the recipient emails.

YAML pipeline example​

The following pipeline that can generate two different notifications. If the code scan detects any CRITICAL or NEW_CRITICAL issues, it sends an automated email like this:

"STO scan of sto-notification-example found the following issues:
Critical : 1
New Critical : 0
High: 0
New High: 0
Medium: 0
New Medium: 0
See https://app.harness.io/ng/#/account/XXXXXXXXXXXXXXXXXXXXXX/sto/orgs/default/"

If the scan finds any NEW_CRITICAL or NEW_HIGH issues, it stops the pipeline execution and sends an email like this:

Stage Block_on_New_Critical_and_New_High_issues failed in pipeline stonotifyexample_-_v3
triggered by D*** B******
Started on Fri Apr 07 14:53:34 GMT 2023 and StageFailed on Fri Apr 07 14:53:36 GMT 2023
Execution URL  https://app.harness.io/ng/#/account/XXXXXXXXXXXXXXXXXXXXXX/sto/orgs/default/projects/myProject/pipelines/stonotifyexample_-_v3/executions/XXXXXXXXXXXXXXXXXXXXXX/pipeline
2s

Here's the full pipeline. Note that the policy and policy set are referenced, but not defined, in the pipeline itself.

pipeline:
  name: sto-notification-example
  identifier: stonotifyexample
  projectIdentifier: default
  orgIdentifier: default
  tags: {}
  properties:
    ci:
      codebase:
        connectorRef: YOUR_CODE_REPO_CONNECTOR_ID
        build: <+input>
  stages:
    - stage:
        name: banditScanStage
        identifier: banditScanStage
        description: ""
        type: SecurityTests
        spec:
          cloneCodebase: true
          execution:
            steps:
              - step:
                  type: Bandit
                  name: Bandit_1
                  identifier: Bandit_1
                  spec:
                    mode: orchestration
                    config: default
                    target:
                      name: dvpwaScanStep-v3
                      type: repository
                      variant: <+codebase.branch>
                    advanced:
                      log:
                        level: info
          infrastructure:
            type: KubernetesDirect
            spec:
              connectorRef: YOUR_KUBERNETES_CLUSTER_CONNECTOR_ID
              namespace: YOUR_NAMESPACE
              automountServiceAccountToken: true
              nodeSelector: {}
              os: Linux
    - stage:
        name: Block on New-Critical and New-High issues
        identifier: Block_on_New_Critical_and_New_High_issues
        description: ""
        type: Custom
        spec:
          execution:
            steps:
              - step:
                  type: Email
                  name: emailOnNotification
                  identifier: Email_1
                  spec:
                    to: john.smithh@myorg.org
                    cc: ""
                    subject: "STO ALERT: Critical issues found in <+pipeline.name>"
                    body: |-
                      "STO scan of <+pipeline.name> found the following issues: <br> 
                       Critical : <+pipeline.stages.banditScanStage.spec.execution.steps.Bandit_1.output.outputVariables.CRITICAL> <br>
                       New Critical : <+pipeline.stages.banditScanStage.spec.execution.steps.Bandit_1.output.outputVariables.NEW_CRITICAL> <br>
                       High: <+pipeline.stages.banditScanStage.spec.execution.steps.Bandit_1.output.outputVariables.HIGH> &#10; <br>
                       New High: <+pipeline.stages.banditScanStage.spec.execution.steps.Bandit_1.output.outputVariables.NEW_HIGH> <br>
                       Medium: <+pipeline.stages.banditScanStage.spec.execution.steps.Bandit_1.output.outputVariables.MEDIUM> <br>
                       New Medium: <+pipeline.stages.banditScanStage.spec.execution.steps.Bandit_1.output.outputVariables.NEW_MEDIUM>  <br>
                       See https://app.harness.io/ng/#/account/MY_ACCOUNT_ID/sto/orgs/default/"
                  timeout: 1d
                  when:
                    stageStatus: All
                    condition: <+pipeline.stages.banditScanStage.spec.execution.steps.Bandit_1.output.outputVariables.NEW_CRITICAL> > 0 || <+pipeline.stages.banditScanStage.spec.execution.steps.Bandit_1.output.outputVariables.CRITICAL> > 0
              - step:
                  type: Policy
                  name: Policy_1
                  identifier: Policy_1
                  spec:
                    policySets:
                      - account.Security_Set_Block_on_Issue_Severity
                    type: Custom
                    policySpec:
                      payload: |-
                        {
                        "NEW_CRITICAL": <+pipeline.stages.banditScanStage.spec.execution.steps.Bandit_1.output.outputVariables.NEW_CRITICAL>, 
                        "NEW_HIGH": <+pipeline.stages.banditScanStage.spec.execution.steps.Bandit_1.output.outputVariables.NEW_HIGH>
                        }
                  timeout: 10m
                  failureStrategies: []
        tags: {}
  notificationRules:
    - name: example sto test
      identifier: example_sto_test
      pipelineEvents:
        - type: StageFailed
          forStages:
            - Block_on_Critical_and_High_issues
      notificationMethod:
        type: Email
        spec:
          userGroups: []
          recipients:
            - john.smithh@myorg.org
      enabled: true