Skip to main content

Kubernetes support bundle for CI

Last updated on

When a CI stage running on a Kubernetes build infrastructure fails, Harness can collect a diagnostic support bundle from the build pod and upload it to your account's CI log storage. Harness Support can then use the bundle, along with your existing pipeline logs, to diagnose cluster-related failures such as network policy denials, image pull errors, scheduling issues, OOM kills, and volume mount failures faster than would be possible from pipeline logs alone.

This page describes exactly what is collected, the Kubernetes permissions required, where the bundle is stored, who can access it, and how to disable collection at the account and stage level.

info

This feature applies only to CI stages running on the Kubernetes cluster build infrastructure. It is not used for Harness Cloud, self-managed VM, or Local/Docker build infrastructures.

When the support bundle is collected​

The collector runs when either of the following is true:

  1. The account-level feature flag CI_SUPPORT_BUNDLE_COLLECTION is enabled for your account by Harness.
  2. The stage-level pipeline variable HARNESS_CI_SUPPORT_BUNDLE_ENABLED is set to true on the specific stage you want diagnostics for.

In practice this means:

  • When the account-level feature flag is on, diagnostics are collected for every failing Kubernetes stage in your account, whether or not the stage sets HARNESS_CI_SUPPORT_BUNDLE_ENABLED.
  • When the account-level feature flag is off, diagnostics are collected only for the specific stages that set HARNESS_CI_SUPPORT_BUNDLE_ENABLED=true.

If both gates are disabled, no data is collected and the rest of this page does not apply.

When collection is active for a stage, the build pod's step log output includes a clearly visible line at the start of the stage so you can confirm collection is on:

[Support Bundle] Diagnostic collection enabled for this stage.
On stage failure, a Kubernetes diagnostic bundle will be uploaded
to your account's CI log storage.

A bundle is only uploaded if the stage fails. Successful stages do not produce a bundle.

What is collected​

On stage failure, the build pod's Lite Engine container queries the in-cluster Kubernetes API for the diagnostic data listed below, packages it into a single support-bundle.tar.gz archive (capped at 50 MB), and uploads it to your CI log storage.

All Kubernetes API calls are read-only (GET) and are scoped to the build pod's own namespace. Go to Kubernetes permissions required to review the single cluster-scoped exception.

File in the bundleSourceSanitization applied before upload
metadata.jsonPipeline / stage / run identifiers, container image references, K8s server versionNone (identifiers only)
pod.jsonBuild pod spec and statusAll container env variables replaced with a REDACTED marker; envFrom removed entirely
all-pods.jsonList of pods in the build pod's namespaceSame env redaction applied to every pod
node.jsonThe single node the build pod is running on (capacity, conditions, kernel version)None (node-level metadata only)
events.jsonKubernetes events in the build pod's namespaceNone
resourcequotas.jsonResource quotas in the namespaceNone
limitranges.jsonLimit ranges in the namespaceNone
networkpolicies.jsonNetwork policies applied to the namespaceNone
persistentvolumeclaims.jsonPVCs in the namespaceNone
configmaps.jsonConfigMaps in the namespace.data and .binaryData are stripped. Only key names are retained, never values.
serviceaccounts.jsonServiceAccounts in the namespaceDefault API response (metadata, labels, annotations, image-pull-secret references by name). No tokens are read or included.
network-diagnostics.txtIn-pod DNS config, route table, TCP reachability probes to common endpointsNone (diagnostic state only)
logs/<container>.logThe most recent 300 lines of stdout/stderr from each container in the build podNone. Refer to the warning below.
logs/<container>-previous.logPrevious container instance logs (when a container restarted)None. Refer to the warning below.
Container logs are included without content sanitization

The bundle includes the most recent 300 lines of stdout/stderr from each container in the build pod. No regex-based scanning or scrubbing is applied to log content before it is packaged.

If your build steps print, echo, or otherwise output sensitive values to stdout/stderr (for example, echo "$GITHUB_TOKEN", a tool that prints an authorization header on failure, or a startup banner that includes a connection string), that content will appear in the bundle as-is.

The same content is already visible in the standard pipeline log viewer in the Harness UI, and bundle access is gated by the same authorization as those logs. Go to Who can access the bundle to review access controls. It does, however, give Harness Support an additional analysis path against the same content.

If you want to ensure no log content is captured from a stage that prints sensitive values, that stage must not be collected from. Because collection is enabled when either gate is on, this means the stage must not set HARNESS_CI_SUPPORT_BUNDLE_ENABLED=true and the account-level feature flag CI_SUPPORT_BUNDLE_COLLECTION must be off for your account. While the account flag is on, all failing stages are collected. Go to How to disable collection to opt out.

What is never collected​

For clarity, the collector contains no path that reads or transmits any of the following:

  • Kubernetes Secrets: There is no GET, LIST, or WATCH call against the secrets resource in any namespace.
  • ServiceAccount tokens: No calls against the serviceaccounts/token subresource or the tokenrequests API. Token values held in Secrets are never read because Secrets themselves are never read.
  • ConfigMap values: Only key names are retained, never values.
  • Pod environment variable values: Replaced with a REDACTED marker in every container of every pod that appears in the bundle.
  • Resources from any namespace other than the build pod's own: Every namespaced API call is pinned to the build pod's namespace.

Kubernetes permissions required​

The build pod's ServiceAccount needs the following permissions for the collector to operate. If a permission is not granted, the collector fails open: the corresponding artifact is omitted from the bundle and a warning is logged, but the build itself is not affected.

Namespaced permissions (in the build pod's own namespace)​

API groupResourceVerbs
(core)podsget, list
(core)pods/logget
(core)eventslist
(core)resourcequotaslist
(core)limitrangeslist
(core)persistentvolumeclaimslist
(core)configmapslist
(core)serviceaccountslist
networking.k8s.ionetworkpolicieslist

Cluster-scoped permission​

API groupResourceVerbsWhy
(core)nodesgetReads the single node that the build pod is running on, to capture node conditions (memory/disk/PID pressure), capacity, allocatable resources, and kernel version. These fields are needed to diagnose eviction, OOM, and ephemeral-storage failures and are not exposed to the pod through any other Kubernetes API.

The cluster-scoped permission is restricted to a single verb (get) on a single resource type (nodes). No wildcard verbs, wildcard resources, or group-level bindings are used. The ClusterRoleBinding references a single ServiceAccount.

If your cluster's RBAC policy does not permit cluster-scoped node reads, you can omit this binding. The bundle will still be produced; it will simply not include node.json. All other diagnostic capability is preserved.

Where bundles are stored​

Storage backendThe same object storage backend (Google Cloud Storage on Harness Cloud; S3-compatible storage on Harness Self-Managed) used for your account's CI step logs.
Object key pathAccount-prefixed under your existing CI log storage prefix, scoped per pipeline run.
Access boundaryIdentical to your CI step logs. The bundle is part of the same authenticated log-service access path. No separate bucket and no new access control surface.
Encryption in transitTLS for all uploads.
Encryption at restProvided by the underlying object storage backend (GCS default encryption on Harness Cloud; per-tenant configuration on Self-Managed).

Retention​

Support bundles inherit the same retention policy as your CI step logs. Retention varies by plan and deployment model.

DeploymentRetention
Harness Cloud (SaaS)The same retention period that applies to your CI step logs. Contact Harness Support to confirm the exact retention period configured for your account.
Harness Self-ManagedDetermined by your local log-service object-storage lifecycle configuration.

If you need a specific bundle deleted before its retention period expires (for example, after a support case is closed), open a request with Harness Support and include the pipeline, run, and stage identifiers.

Who can access the bundle​

Bundles are stored at the same access boundary as CI step logs. Concretely:

  • Customer users: any user in your account who has read access to the pipeline's CI step logs through the Harness UI or API also has read access to the corresponding bundle. No additional role or permission is granted by enabling support bundle collection.
  • Harness employees: when you engage Harness Support on a CI issue, Support engineers can download the bundle for the failing run as part of the existing customer-data access process. Access uses each engineer's own Harness API key and is subject to Harness's standard access auditing on customer log data. There is no shared service account or shared credential with elevated bundle access.

Harness debugging tooling that consumes the bundle uses the same authentication path as the Harness UI log download. There is no separate, broader access channel.

How to disable collection​

Because collection is enabled when either gate is on, stopping collection means turning off every gate that is currently enabled:

  • While the account-level feature flag is on, every failing stage is collected, regardless of the stage variable. Go to Disable for the entire account to turn the flag off.
  • Any stage that sets HARNESS_CI_SUPPORT_BUNDLE_ENABLED=true is collected even when the account flag is off, so the variable must also be removed or set to false.

Disable for a specific stage​

Remove the stage variable, or set it to false:

stages:
  - stage:
      name: build
      type: CI
      spec:
        # ... infrastructure, cloneCodebase, etc.
        variables:
          - name: HARNESS_CI_SUPPORT_BUNDLE_ENABLED
            type: String
            value: "false"   # or simply omit this variable entirely

Removing the stage variable stops collection for that stage only if the account-level feature flag is also off. While the account flag is on, the stage is still collected on failure even without the variable. To stop it in that case, go to Disable for the entire account.

Disable for the entire account​

Contact Harness Support and request that the CI_SUPPORT_BUNDLE_COLLECTION feature flag be turned off for your account.

Turning off the account-level feature flag stops account-wide collection. However, any individual stage that explicitly sets HARNESS_CI_SUPPORT_BUNDLE_ENABLED=true will still be collected on failure. To fully disable collection, turn off the feature flag and ensure no stage sets the variable to true.

Verify collection is off​

When collection is enabled and active for a stage, the build pod's step log output includes the line shown in When the support bundle is collected. If you do not see that line in your build's step output, no collection occurred.

  1. Scope collection to what you need. When the account-level feature flag is off, use the HARNESS_CI_SUPPORT_BUNDLE_ENABLED variable to turn collection on only for stages you actively want diagnostics from, typically stages that have been failing intermittently and are hard to triage from logs alone. Keep in mind that when the account-level flag is on, collection applies to all failing Kubernetes stages in the account, so if you need stage-level control, request that the account flag stay off and rely on the stage variable.
  2. Avoid printing secrets to stdout. Container logs are included in the bundle without content sanitization. If a build step would echo a sensitive value to stdout, redirect or mask it the same way you would for the standard pipeline log view.
  3. Use Harness secrets, not pipeline variables, for credentials. Values injected through Harness Secrets are referenced by name in the pod spec (envFrom/secretKeyRef), and the collector strips both env and envFrom from the pod spec before upload. Plaintext values in pipeline variables, by contrast, may end up in container logs if a step echoes them.
  4. Disable when not needed. If you have stages with sensitive output that you cannot fully sanitize, leave HARNESS_CI_SUPPORT_BUNDLE_ENABLED unset on those stages.
Last updated on