Skip to main content

Share CI data across steps and stages

This topic describes how you can cache and share data across steps, stages, and builds.

Share data between steps in a stage​

When a pipeline runs, it creates a temporary volume called a workspace. During initialization, the stage clones your codebase to the root of the workspace. Then, the steps in the stage run inside the root. The workspace is a volume that persists for the lifetime of the stage and enables steps in that stage to communicate and share state information. The workspace is destroyed when the stage ends.

The workspace is the current working directory for each step in the stage, and the default shared working directory for any stage is /harness. Any step in the stage can create, retrieve, update, and delete files in this folder. If you need to share additional volumes between steps in the stage, you can add Shared Paths in the Build stage settings. Paths must begin with a forward slash, such as /vol.

For example, the maven m2 repo is stored in /root/.m2 by default, which is outside the /harness directory. If your Build stage uses Maven, you can specify /root/.m2 as a Shared Path so that all steps in that stage can access that directory.

Share data across stages​

You can use the following caching methods to share data across stages:

You cannot share access credentials or other Text Secrets across stages.

If you need to maintain a long-running service for the duration of a stage, use a Background step.

Use caching to reduce build time​

Use the following caching methods to reduce build time:

You cannot share access credentials or other Text Secrets with caching.

Multilayer caching

For multilayer caching, use multiple Restore Cache and Save Cache steps according to the pattern described in Multilayer caching.

Remote Docker cache​

Remote Docker caching can dramatically improve build times by sharing data across pipelines, stages, and steps. Remote caching leverages your exising Docker registry to pull previously built layers.

Remote caching isn't available for all build infrastructures. If available for your build infrastructure, you can enable the Remote Cache Image option in the following steps:

info

Harness Cloud can manage the Docker layer cache backend for you, without relying on your Docker registry, which can further reduce build times. To learn more, go to Docker Layer Caching.

For more recommendations for optimizing Docker images, go to Optimize and enhance CI pipelines.