Ctrlk
Sign UpGet a DemoContact Us
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Build Intelligence Overview

Learn about Build Intelligence in Harness CI.

Build Intelligence is part of Harness CI Intelligence, a suite of Harness CI features that improve build times. It stores outputs remotely and retrieves them when inputs do not change.

Build Intelligence is available for Gradle, Bazel, and Maven (version 3.9+) build tools. You can use it with any programming language that uses a supported build tool.

Build Intelligence has the following availability and support details:

  • Build Intelligence is Generally Available (GA) and is enabled by default for newly created CI stages. This is configurable in CI default settings.

  • Build Intelligence supports Linux only (AMD and ARM).

  • Build Intelligence supports Cloud and Kubernetes build infrastructures only.

  • Build Intelligence supports build commands in Run and Test steps only. It does not support compile or package commands that run in a Dockerfile through a Build and Push step.

Use Build Intelligence

Build Intelligence seamlessly integrates into your workflow without requiring changes to your build commands. Harness automatically detects supported build tools in your pipeline and injects the necessary configurations into the relevant files within the build workspace. This ensures Build Intelligence optimizes your builds during Gradle or Bazel operations performed in Test or Run steps.

In the UI editor, enable Build Intelligence on the Build stage overview. Alternatively, enable it in the YAML editor.

The following example shows a CI stage using Build Intelligence:

- stage:
    identifier: build
    name: build
    type: CI
    spec:
      cloneCodebase: true
      buildIntelligence: 
        enabled: true # Build intelligence enabled
      execution:
        steps:
          - step:
              type: Action
              name: Set up Gradle
              identifier: Set_up_Gradle
              spec:
                uses: gradle/gradle-build-action@v2
                with:
                  gradle-version: "8.5"
          - step:
              type: Run
              name: build
              identifier: build
              spec:
                shell: Sh
                command: ./gradlew build --profile  # '--profile' is optional but advised for gradle

You can also enable or disable Build Intelligence based on an expression directly in the YAML. For example, you can conditionally set whether Build Intelligence is on or off using a pipeline variable expression like <+pipeline.variables.someVar>.

Use Build Intelligence when building a Docker image

Build Intelligence does not support compile or package commands that run in a Dockerfile through a Build and Push step, including BuildAndPushDockerRegistry and Kaniko builds. This applies to Gradle, Maven, Bazel, Go, and any other toolchain that you invoke with RUN in the Dockerfile.

The Docker build environment does not have access to the /harness workspace, Build Intelligence environment variables, or the cache proxy. If Build Intelligence is enabled on the stage, injection can still rewrite build files copied into the image. The build then tries to download plugins or cache clients, often from Maven Central, and can fail with HTTP 429 or missing-artifact errors.

To use Build Intelligence:

  1. Run the compile in a Run or Test step. Run the compile or package command in a Run or Test step before the Build and Push step.

  1. Copy artifacts in the Dockerfile. Remove the matching RUN compile command from the Dockerfile. Copy the artifact produced by the preceding step instead.

  1. Keep Build and Push for the image. Use the Build and Push step only to build and push the image. Optionally, set caching: true on this step to enable Docker layer caching.

If the compile or package command must stay in the Dockerfile:

  • Disable Build Intelligence on the CI stage.

  • Enable Docker layer caching to reuse unchanged Docker layers.

  • Use an internal package repository for dependencies, for example Maven, Gradle, Go modules, or npm. This avoids public registry rate limits such as Maven Central HTTP 429.

Docker layer caching and internal package repositories can improve build performance. They do not provide Build Intelligence. Resolve injected plugins from an internal repository Gradle and Maven auto-injection can still try to download io.harness:gradle-cache or the Maven build-cache extension from Maven Central. Mirror the exact coordinates from the build logs in your internal Maven repository. Set the MAVEN_URL stage variable or configure Maven Repository URL in CI default settings. This configuration changes only where the plugin artifact is downloaded. It does not make Build Intelligence supported for commands that run inside a Dockerfile.

The cache storage limit depends on your subscription plan type. Go to Subscriptions and licenses to understand usage limits.

Harness does not limit the number of caches you can store. Once you reach your storage limit, Harness saves new caches by automatically evicting old caches.

The cache retention window is 15 days, which resets whenever a cache is updated.

  • Build Intelligence is only supported for Kubernetes on self-hosted build infrastructure.

  • By default, Build Intelligence uses port 8082, and downloads the Build Intelligence plugin from Maven Central. You can change the default behaviour in CI default settings.

  • When using Build Intelligence with self-hosted infrastructure, you need object storage for cache storage. Build Intelligence supports Azure Blob Storage, GCP Cloud Storage, AWS S3, and any S3-compatible storage. Go to configure default object storage to set up your storage.

  • By default, the Build Intelligence step configures a proxy on port 8082. However, for self-hosted setups, you can configure the port by setting the stage variable CACHE_SERVICE_HTTPS_BIND, or in CI default settings.

  • You can also override the storage connector, region, bucket name, and sidecar container settings at the stage level. Go to Override storage and sidecar settings to configure per-stage overrides.

  • When using Azure Blob Storage, authenticate with the Azure connector using Service Principal or OIDC. Managed Identity authentication is not supported for Build Intelligence.

Example Pipeline YAML:

  • By default, the Build Intelligence plugin is downloaded from Maven Central. If your environment does not have access to Maven Central or you prefer using a custom Maven repository, you can configure this by setting a stage variable named MAVEN_URL, or in CI default settings. See Build Intelligence plugin

How does Build Intelligence work

Harness auto-detects supported build tools (Gradle and Bazel). It auto injects required configuration to appropriate files on the build workspace. This will allow Build Intelligence to automatically optimize your builds when bazel/gradle operation are done in Test or Run steps.

Gradle Config

When using gradle, Harness creates an init.gradle file in ~/.gradle/init.d or $GRADLE_HOME/init.d or $GRADLE_USER_HOME/init.d folder if not found, with the required configuration.

Bazel Config

When using bazel, Harness create a ~/.bazelrc file (if it does not exist), with the required configuration.

The config will look like: build --remote_cache=http://endpoint:port/cache/bazel (endpoint is localhost:8082)

Maven Config

For Maven builds, the following configuration files are injected into the environment:

.mvn/maven-build-cache-config.xml

.mvn/extensions.xml

~/.m2/settings.xml

There are two ways to apply settings.xml:

  • Maven installation-wide: ${maven.home}/conf/settings.xml

  • User-specific: ${user.home}/.m2/settings.xml

These configurations enable build caching and repository access based on your pipeline’s setup.

Use a private Maven repository for Build Intelligence extensions

If your environment restricts access to Maven Central, you can configure Build Intelligence to download required extensions (like the Maven build cache extension) from your private or third-party artifact registry.

This support is built-in and requires the following:

  1. Upload Required Maven Artifacts Ensure the following artifacts are available in your private Maven repository under the correct path structure (i.e., org/apache/maven/extensions/maven-build-cache-extension/1.2.0/):

This ensures Maven clients can resolve the artifacts properly using the standard package coordinates, which are required by Build Intelligence to enable caching for Maven builds.

  1. Update Your pom.xml

Point your Maven build to use your private registry:

  1. (Optional) Add Authentication If your registry requires authentication, add this to your settings.xml file:

Make sure the <id> in settings.xml matches the one in the <repository> section of your pom.xml.

Use --profile

Appending `--profile' to your build command, enables publishing Build Intelligence savings to Harness. This will allow you to clearly view the performance and benefits of using Build Intelligence. Note that even when omitted, Build Intelligence will continue to work and optimize your run as expected, but the savings will not be visible in the UI and relevant dashboards.

For example: ./gradlew build --profile

This is supported only with the Gradle build tool.

Go to Intelligence Savings to understand how savings are calculated.

Override storage and sidecar settings

When running Build Intelligence on self-managed Kubernetes infrastructure, the Build Intelligence sidecar uses storage connector and container settings from your account-level Default Settings by default. You can override any of these individually per stage under stage.spec.buildIntelligence to use different connectors, bucket configurations, or resource allocations for specific stages.

FALLBACK BEHAVIOR

Each field falls back independently. You only need to specify the fields you want to override. For example, if you specify connectorRef but omit region and bucket_name, the stage uses the stage-level connector with the account-level region and bucket name. Any field not specified at the stage level uses the value from CI default settings.

These settings apply only to Kubernetes build infrastructure.

Override the storage connector and bucket

Use connectorRef, region, and bucket_name to override the storage settings configured in your account-level Default Settings.

  • connectorRef specifies the connector ID for cloud storage. This overrides the Cloud Storage Connector from Default Settings. Supported connector types include Azure Blob Storage, GCP Cloud Storage, AWS S3, and any S3-compatible storage. When using an Azure connector, authenticate with Service Principal or OIDC only; Managed Identity is not supported.

  • region specifies the bucket region. This overrides the account-level Region setting.

  • bucket_name specifies the bucket name. This overrides the account-level Bucket Name setting.

Configure run-as-user

Use runAsUser to set the user ID (UID) for the Build Intelligence sidecar container. This corresponds to the Kubernetes securityContext.runAsUser property.

This is useful when your cluster enforces non-root security policies or when you need root access (UID 0) for the sidecar container.

Configure sidecar container resources

Use resources to define Kubernetes resource requests and limits for the Build Intelligence sidecar container. This controls memory and CPU allocation to prevent the sidecar from being OOMKilled or starving other containers in the pod.

Full YAML reference

The following example shows all available buildIntelligence properties:

Troubleshooting

Maven deploy plugin skipped when using Build Intelligence

When Build Intelligence restores cached build outputs, Maven deploy plugins (such as maven-deploy-plugin) may be skipped, preventing artifacts from being uploaded to Artifactory or other artifact repositories. This is expected behavior because Build Intelligence treats the deploy phase as part of the cached build output.

To ensure artifacts are always uploaded to your repository while still benefiting from Build Intelligence caching for other build phases (compile, test, package), use the alwaysRunPlugins flag with your Maven command:

This flag tells the Maven build cache extension to bypass the cache for the specified plugin, ensuring the deploy phase always executes while other phases (build, compile, test) continue to use Build Intelligence caching.

You can also configure this permanently in your .mvn/maven-build-cache-config.xml file by adding a runAlways section:

Go to the Maven Build Cache Extension documentation to learn more about cache configuration options.

License-checking plugins flag Build Intelligence files

Build Intelligence generates XML configuration files in the .mvn/ directory of your build workspace. License-compliance plugins, such as Apache RAT (Release Audit Tool), License Maven Plugin, or similar source-auditing tools, may flag these generated files for missing license headers. This can cause build failures.

Option 1: Exclude Harness-generated files from the scan

Add the Harness-generated directories to your plugin's exclusion list. The key directories to exclude are:

  • Build Intelligence: /harness/.mvn

  • Cache Intelligence: /harness/.m2, /harness/.mvn

Add the following snippet under the <build> section to configure the apache-rat-plugin to ignore these paths:

Example: Apache RAT plugin in pom.xml

Example: .rat-excludes file

Example: License Maven Plugin in pom.xml

Refer to your specific plugin's documentation for the correct exclusion syntax.

Option 2: Inject license headers into generated files

If excluding files is not an option, add a Run step before your build step that injects the required license headers into the generated XML files. The example below uses an Apache Software Foundation header, but you should adapt it to the header your plugin expects:

Place this Run step after the codebase clone and before your build/test steps so the headers are present when the scan runs.

PreviousBuild, Push, Upload, Download
NextBuild and push artifacts and images

Last updated

Was this helpful?