CodeQL scanner reference for STO
You can scan your code repositories using CodeQL, an analysis engine used by developers to automate security checks, and by security researchers to perform variant analysis.
The following steps outline the basic workflow:
-
Run a CodeQL scan, either externally or as part of a Run step, and publish the results to SARIF.
-
Add the SARIF data to your pipeline. If you ran the scan outside the pipeline, do the following:
-
In the stage where you ingest the results, go to Overview > Shared Paths and create a folder under
/sharedsuch as/shared/scan_results. -
Use a Run step to add your scan results to the shared folder.
-
-
Use a CodeQL step to ingest the results.
This topic includes an end-to-end YAML pipeline that illustrates this workflow.
Important notes for running CodeQL scans in STO
Root access requirements
If you want to add trusted certificates to your scan images at runtime, you need to run the scan step with root access.
You can set up your STO scan images and pipelines to run scans as non-root and establish trust for your proxies using custom certificates. For more information, go to Configure STO to Download Images from a Private Registry.
For more information
The following topics contain useful information for setting up scanner integrations in STO:
CodeQL step settings for STO scans
The recommended workflow is to add a CodeQL step to a Security Tests or CI Build stage and then configure it as described below.
Scan
Scan mode
- Ingestion Configure the step to read scan results from a data file and then ingest, normalize, and deduplicate the data.
Scan configuration
The predefined configuration to use for the scan. All scan steps have at least one configuration.
Target
Type
-
Repository Scan a codebase repo.
In most cases, you specify the codebase using a code repo connector that connects to the Git account or repository where your code is stored. For information, go to Configure codebase.
Detect target and variant
When auto-detect is enabled for code repositories, the step detects these values using git:
- To detect the target, the step runs
git config --get remote.origin.url. - To detect the variant, the step runs
git rev-parse --abbrev-ref HEAD. The default assumption is that theHEADbranch is the one you want to scan.
Note the following:
- Auto-detection is not available when the Scan Mode is Ingestion.
- Auto-detect is the default selection for new pipelines. Manual is the default for old pipelines, but you might find that neither radio button is selected in the UI.
Name
The identifier for the target, such as codebaseAlpha or jsmith/myalphaservice. Descriptive target names make it much easier to navigate your scan data in the STO UI.
It is good practice to specify a baseline for every target.
Variant
The identifier for the specific variant to scan. This is usually the branch name, image tag, or product version. Harness maintains a historical trend for each variant.
Ingestion file
The path to your scan results when running an Ingestion scan, for example /shared/scan_results/myscan.latest.sarif.
-
The data file must be in a supported format for the scanner.
-
The data file must be accessible to the scan step. It's good practice to save your results files to a shared path in your stage. In the visual editor, go to the stage where you're running the scan. Then go to Overview > Shared Paths. You can also add the path to the YAML stage definition like this:
- stage:
spec:
sharedPaths:
- /shared/scan_results
Log Level
The minimum severity of the messages you want to include in your scan logs. You can specify one of the following:
- DEBUG
- INFO
- WARNING
- ERROR
Additional CLI flags
Use this field to run the CodeQL scanner binary with additional flags.
Passing additional CLI flags is an advanced feature. Harness recommends the following best practices:
-
Test your flags and arguments thoroughly before you use them in your Harness pipelines. Some flags might not work in the context of STO.
-
Don't add flags that are already used in the default configuration of the scan step.
To check the default configuration, go to a pipeline execution where the scan step ran with no additional flags. Check the log output for the scan step. You should see a line like this:
Command [ scancmd -f json -o /tmp/output.json ]In this case, don't add
-for-oto Additional CLI flags.
Fail on Severity
Every Security step has a Fail on Severity setting. If the scan finds any vulnerability with the specified severity level or higher, the pipeline fails automatically. You can specify one of the following:
CRITICALHIGHMEDIUMLOWINFONONE— Do not fail on severity
The YAML definition looks like this: fail_on_severity : critical # | high | medium | low | info | none
Settings
You can use this field to specify environment variables for your scanner.
Additional Configuration
In the Additional Configuration settings, you can use the following options:
Advanced settings
In the Advanced settings, you can use the following options:
CodeQL pipeline example
The following pipeline illustrates a simple ingestion scan. It consists of two steps. A Run step generates an example CodeQL data file in SARIF format. A CodeQL step then ingests the data.
pipeline:
projectIdentifier: STO
orgIdentifier: default
tags: {}
stages:
- stage:
name: ingestion
identifier: ingestion
type: SecurityTests
spec:
cloneCodebase: false
infrastructure:
type: KubernetesDirect
spec:
connectorRef: K8S_DELEGATE_CONNECTOR
namespace: harness-delegate-ng
automountServiceAccountToken: true
nodeSelector: {}
os: Linux
execution:
steps:
- step:
type: Run
name: create codeql sarif
identifier: create_codeql_sarif
spec:
connectorRef: CONTAINER_IMAGE_REGISTRY_CONNECTOR
image: alpine
shell: Sh
command: |-
pwd
echo '{
"$schema": "",
"version": "sarif-2.1.0",
"runs": [
{
"tool": {
"driver": {
"name": "CodeQL",
"version": "2.5.7",
"semanticVersion": "2.5.7+1234567890",
"informationUri": "https://github.com/github/codeql",
"properties": {
"analysisTarget": "myproject",
"analysisTimestamp": "2023-04-03T14:00:00Z",
"analysisDuration": 120000,
"query": "detect-external-libs.ql",
"queryUrl": "https://github.com/github/codeql/blob/master/javascript/ql/src/semmle/javascript/Security/CWE/CWE-094/ExternalLibraries.ql"
}
}
},
"results": [
{
"ruleId": "js/detect-external-libs",
"message": {
"text": "The following external libraries were found: jQuery, Lodash"
},
"locations": [
{
"physicalLocation": {
"artifactLocation": {
"uri": "/path/to/myproject/js/script.js"
},
"region": {
"startLine": 10,
"startColumn": 1,
"endLine": 10,
"endColumn": 15
}
}
}
],
"level": "warning",
"properties": {
"severity": "high",
"confidence": "medium"
}
}
]
}
]
}'> codeql.sarif
ls
- step:
type: CodeQL
name: CodeQL_1
identifier: CodeQL_1
spec:
mode: ingestion
config: default
target:
name: login_microservice
type: repository
variant: my_hotfix_branch
advanced:
log:
level: info
fail_on_severity: critical
ingestion:
file: /harness/codeql.sarif
sharedPaths:
- /var/run
- /shared/scan_results/
identifier: codeql_ingestion
name: codeql ingestion