Custom Ingest step

Ingest third-party scanner results into a Continuous Delivery stage using SARIF or custom JSON

The Custom Ingest step ingests results from any third-party scanner. Harness Security Testing Orchestration (STO) supports two generic data formats:

• SARIF (Static Analysis Results Interchange Format) is an open data format supported by many scan tools, especially tools available as GitHub Actions. You can ingest SARIF 2.1.0 data from any tool that supports this format. If your scanner can export to SARIF, use this format. Go to Ingest SARIF scan results to configure SARIF ingestion.

• Harness STO supports a custom JSON format that is useful for ingesting data from scanners that have no integration in STO and that cannot publish to SARIF. Go to Ingest results from unsupported scanners to configure JSON ingestion.

Use a supported scanner when one exists

The Custom Ingest step is intended for scanners that have no supported integration in STO. Use the documented workflow for supported scanners. Go to Supported Security Scanners for the list of scanners STO supports.

---

What you will learn from this topic

• How to choose SARIF or custom JSON for a scanner that STO does not integrate directly

• How to find the step settings for scan, target, and ingestion

• How to open an ingestion workflow for SARIF or JSON results

• How to read a sample pipeline that writes a JSON results file and ingests it

---

Before you begin

Review the following before you add a Custom Ingest step:

• Containerized step group: The sample pipeline runs the step inside a step group on Kubernetes. Go to Containerize step groups to enable container-based execution.

• Supported scanners: Go to Supported Security Scanners and use a dedicated scanner step when STO already supports the scanner.

• Result file: The step reads a SARIF or custom JSON file. Go to Ingest SARIF scan results or Ingest results from unsupported scanners to prepare that file.

---

Related resources

The following topics cover scanner setup in Security Testing Orchestration:

• What's supported in STO

• Security Testing Orchestration FAQs

• Optimize STO pipelines

---

Custom Ingest step settings for STO scans

The step settings are grouped below. This topic lists the setting names. Go to Custom Ingest step for SARIF and JSON scan results for the field descriptions maintained with the Security Testing Orchestration step reference.

Scan

The scan settings on this step are:

• Scan mode

• Scan Configuration

Target

Type

The target type of the scanned object. You can ingest data for the following target types:

• Repository: Ingest scan results for a code repository.

• Container image: Ingest scan results for the layers, libraries, and packages in a container image.

• Instance: Ingest scan results for a running application.

• Configuration: Ingest scan results for your cloud environment, generated by gathering configuration data through the cloud provider's APIs.

Name

Variant

Ingestion

The ingestion setting on this step is Ingestion File.

Log Level, CLI flags, and Fail on Severity

The log and failure settings on this step are:

• Log Level

• Additional CLI flags

• Fail on Severity

Additional Configuration

Go to Custom Ingest step for SARIF and JSON scan results for the fields under Additional Configuration.

Advanced settings

Go to Custom Ingest step for SARIF and JSON scan results for Conditional Execution, Failure Strategy, Looping Strategy, and Policy Enforcement.

---

Proxy settings

Go to Custom Ingest step for SARIF and JSON scan results for the proxy variables on this step.

---

Custom Ingest pipeline examples

The following topics describe end-to-end workflows for ingesting SARIF and JSON data:

• Gitleaks step configuration

• Checkmarx step configuration

• Run STO scans using GitHub Action and Drone Plugin steps

• Ingest results from unsupported scanners

---

Sample pipeline

The following pipeline writes a custom JSON results file in a Run step, then ingests that file with a Custom Ingest step:

Sample YAML

pipeline:
  name: abc
  identifier: abc
  projectIdentifier: your_project_identifier
  orgIdentifier: your_org_identifier
  tags: {}
  stages:
    - stage:
        name: s1
        identifier: s1
        description: ""
        type: Deployment
        spec:
          deploymentType: Kubernetes
          service:
            serviceRef: your_service_identifier
          environment:
            environmentRef: your_environment_identifier
            deployToAll: false
            infrastructureDefinitions:
              - identifier: your_infrastructure_identifier
          execution:
            steps:
              - stepGroup:
                  name: sg1
                  identifier: sg1
                  steps:
                    - step:
                        type: Run
                        name: Run_1
                        identifier: Run_1
                        spec:
                          connectorRef: a
                          image: alpine
                          shell: Sh
                          command: |-
                            cat <<EOF >> example.json
                            {  
                               "meta":{  
                                  "key":[  
                                     "issueName",  
                                     "fileName"  
                                  ],  
                                  "subproduct":"MyCustomScanner"  
                               },  
                               "issues":[  
                                  {  
                                     "subproduct":"MyCustomScanTool",  
                                     "issueName":"Cross Site Scripting",  
                                     "issueDescription":"Lorem ipsum...",  
                                     "fileName":"homepage-jobs.php",  
                                     "remediationSteps":"Fix me fast.",  
                                     "risk":"high",  
                                     "severity":8,  
                                     "status":"open",  
                                     "referenceIdentifiers":[  
                                        {  
                                           "type":"cwe",  
                                           "id":"79"  
                                        }  
                                     ]  
                                  }  
                               ]  
                            }
                            EOF
                            ls
                            cat example.json
                    - step:
                        type: CustomIngest
                        name: CustomIngest_1
                        identifier: CustomIngest_1
                        spec:
                          mode: ingestion
                          config: default
                          target:
                            type: repository
                            detection: manual
                            name: external-scanner-test
                            variant: main
                          advanced:
                            log:
                              level: info
                          ingestion:
                            file: example.json
                  stepGroupInfra:
                    type: KubernetesDirect
                    spec:
                      connectorRef: your_k8s_connector
            rollbackSteps: []
        tags: {}
        failureStrategies:
          - onFailure:
              errors:
                - AllErrors
              action:
                type: StageRollback

---

Next steps

• Go to Ingest SARIF scan results to ingest SARIF 2.1.0 output.

• Go to Ingest results from unsupported scanners to ingest the custom JSON format.

• Go to Custom Ingest step for SARIF and JSON scan results for the field descriptions of the settings listed above.

• Go to Containerize step groups to run the step on your Kubernetes cluster.