Add a HashiCorp Vault secret manager

This document explains how to store and use encrypted secrets (such as access keys) by adding a HashiCorp Vault Secrets Manager.

To store and use encrypted secrets (such as access keys), you can add a HashiCorp Vault Secret Manager.

The video below provides an overview of how to use HashiCorp Vault as an external secret manager.

Before you begin

Harness' key concepts
Harness Secret Manager Overview
Store authentication credentials
Make sure that the Harness Delegate can connect to the Vault URL.
Make sure you have View and Create/Edit permissions for secrets.

Add a Secret Manager

This topic assumes that you already have a Harness Project set up. If you don’t have one yet, see Create Organizations and Projects for step-by-step instructions.

You can create a Connector at the Project, Organization, or Account scope, depending on your access and sharing requirements.

To add a Connector:

Navigate to the appropriate scope:
- Project Setup (for project-level connectors)
- Organization Resources (for organization-level connectors)
- Account Resources (for account-level connectors)
Select Connectors. Click New Connector (or Connector, depending on the UI version).
From the list of connector types, choose Secret Managers.
Select HashiCorp Vault.
The HashiCorp Vault Secret Manager configuration page opens, where you can enter the required connection and authentication details.

Overview

Enter a Name for the secret manager.
(Optional) Modify the ID, or leave it the same as the name. For more information, see Entity Identifier Reference.
Enter a Description for the secret manager.
Add Tags, if required.
Select Save and Continue.

Configure the Authentication type

On the Vault Details page, Enter the Vault URL.
Enter the Base Secret Path. The Base Secret Path is used when writing secrets to Vault. When reading secrets, Harness uses the complete secret path.
For more information about Vault configuration, see the official Vault documentation.
Select the Authentication Type.
Option: App Role
```
  The App Role option enables the Harness Vault Secret Manager to authenticate with Vault-defined roles.

  The Vault AppRole method allows you to define multiple roles corresponding to different applications, each with different levels of access. The application's **App Role ID** and **Secret ID** are used for authentication with Vault. You need these to log in and fetch a Vault token.

  To assign a **Secret ID**, you can create a new [**Secret**](https://app.gitbook.com/s/3F2TpHXhur2QtQnORSM9/use-harness-platform/secrets/add-use-text-secrets) or choose an existing one.

  The SecretId should not expire and it should be valid until it is manually revoked. Harness uses the App Role ID and Secret ID you supply to fetch a Vault Auth Token dynamically whenever there is a CRUD operation of secrets related to this vault. For example, when creating a secret in this vault, Harness internally uses this App Role Id and Secret ID and makes a call to vault via the delegate to generate a token. 
```
Enable Cache for App Role Token
```
     Now, this token is used to make the actual secret creation call to vault. This token is never received on the Harness side. It resides in the delegate and is cached for the duration that is one percent less than the time-to-live (TTL) of the token to prevent the generation of a new token for each request for improved performance. If you don't want to cache the token, you can disable the caching and token will be destroyed after the operation is successfully completed.
```
Example of Token Caching Refresh
```
     To understand how the number of delegates and caching schedule works, here is an example that we have put together.

     Let’s assume we have set a 15 min TTL. The token is cached for one pipeline during those 15 minutes (minus 1 percent), but it is determined by the **delegate** which is being used with the connector.  If the delegate has been used previously with the connector within the 15 min, then a cached version of the Vault token will exist on the delegate.  

     If we have three pipeline, and two delegates involved, the assignment in our example is:
     - `Pipeline A` and `Pipeline C` use the delegate, `delegate-mumbai`
     - `Pipeline B` uses the delegate, `delegate-sanfran`

     We will see the following behavior for the Vault connector occur in the executions, along with the **unique refresh timer** for each delegate
     ```
     Pipeline A executed at time 0 (Token initial new fetch using role ID and Secret ID, stored on delegate-mumbai)
     Pipeline B executed at time +2min (Token initial new fetch using role ID and Secret ID, because this is a different delegate delegate-sanfran)
     Pipeline A executed at time +5min
     Pipeline C executed at time +10 min
     Pipeline B executed at time +11 min
     Pipeline A executed at time +16 min (Token expired on delegate-mumbai, Token new fetch performed using role ID and Secret ID,)
     Pipeline C executed at time +17 min
     Pipeline B executed at time +20 min (Token expired on delegate-sanfran, Token new fetch performed using role ID and Secret ID,)
     ```

     Each delegate being utilized with the connector has a local cache of the token which refreshes according to the TTL rule of 15 minutes (minus 1 percent).
```
Additional Information
```
  For more information, go to [RoleID](https://www.vaultproject.io/docs/auth/approle.html#roleid) and [Authenticating Applications with HashiCorp Vault AppRole](https://www.hashicorp.com/blog/authenticating-applications-with-vault-approle) in the HashiCorp documentation.

  If you encounter errors, setting [token_num_uses](https://www.vaultproject.io/api-docs/auth/approle#token_num_uses) to `0` can often resolve problems.
```
Permissions
The Vault AppRole ID or the Periodic Token used in either of the authentication options must have an ACL policy attached so that Harness can use it. Typically, you create the policy first, then create the AppRole or Periodic Token and attach the policy.
IMPORTANT
The AppRole option only works with the default authentication path approle. If you change the authentication path to a non-default value like approle-test, the AppRole authentication will fail.
In the policy examples below: if you've created a Read-only Vault Secret Manager, this secret manager needs only read, and list permissions on Vault. It does not need — and cannot assume — create, update, or delete permissions. If the secrets are in the Secret Engine named “secret”, the policy must have the following permissions.
```
path "secret/*" {
capabilities = ["create", "update", "list", "read", "delete"]
}
```
If the secrets are in a subfolder, such as secrets/harness, the policy will look like this:
```
path "secret/harness/*" {
capabilities = ["create", "list", "read", "update", "delete"]
}
path "secret/harness" {
capabilities = ["list", "read"]
}
```
These examples apply only to a v1 secret engine. If you are planning to use a secret engine with version 2 (versioned secret engine), then the policies are different as explained here. Go through this link to understand the correct permissions required for your use case. If the Vault Secret Manager needs to renew tokens, the following permissions are needed:
```
path "auth/token/renew-self" {
capabilities = ["read", "update"]
}
```
Custom Authentication Path for AppRole Login
FEATURE AVAILABILITY
This feature is supported on delegate version 837xx and above.
The appRoleLogin method now supports a custom authentication path, utilizing it if provided, or defaulting to the standard approle endpoint if not specified.
Click to view full size image
For example, the custom AppRole path can be set as <customPath>, providing greater flexibility in configuring the authentication mechanism. This feature is particularly useful in scenarios where the default authentication path requires customization.
Ensure that the custom AppRole path you configure exists in your HashiCorp Vault. This step is necessary for the authentication to succeed using the custom path.
Option: Token
For Harness, the Token option requires periodic tokens (tokens that have renewal options). To create a periodic token, make sure to specify a period in the token creation command:
```
  ```
  vault token create -policy=harness -period=768h
  ```
```
Next, use the newly generated token in Harness. To do this, complete the following steps:
- Select Create or Select a Secret.
- On the Secret Settings page, either create a new Secret or select an existing secret; if creating a new secret, enter the generated token in the Secret Value field.
  For detailed steps on creating a secret, see Add and reference text secrets. If you have already created a secret containing the token, select it from the list as shown below.
- Select Apply. If you want to verify the renewal manually, use the command:
  vault token lookup <token_id>
Option: Vault Agent
This option enables the Harness Vault Secret Manager to authenticate with the Auto-Auth functionality of the Vault Agent.
To authenticate with Vault Agent, make sure you have configured it on the required environment, with entries for method and sinks. For more information, go to Vault Agent.
In the Sink Path field, enter any sink path you have in your Vault Agent Configuration. This is the path of the encrypted file with tokens. The specified delegate reads this file through file protocol (file://).
Option: AWS Auth
This option provides an automated mechanism to retrieve a Vault token for IAM principals and AWS EC2 instances. With this method, you do not need to manually install or supply security-sensitive credentials such as tokens, usernames, or passwords.
In the AWS Auth method, there are two authentication types:
- IAM
- EC2
Harness recommends using the IAM technique for authentication since it is more versatile and complies with standard practices.
To authenticate with AWS Auth, make sure you have configured the vault with entries for Header, Role, and Region. For more information, go to AWS Auth Method in the Vault documentation.
You must add the Server ID Header from Vault as a Harness Encrypted Text Secret and select it for Server Id Header in Harness.
In Role, enter the role you have configured in the Vault.
In Region, enter the AWS Region for the Secret Manager.
Option: Kubernetes Auth
This option uses a Kubernetes Service Account Token to authenticate with Vault. With this method of authentication, you can easily add a Vault token into a Kubernetes pod.
To authenticate with Kubernetes Auth, make sure you have created a role in the vault inside auth/kubernetes/role. This role authorizes the "vault-auth" service account in the default namespace and it gives it the default policy. This is also where you'll find the service account name and namespace that will be used to access the vault endpoint.
For more information, go to Kubernetes Auth Configuration in the Vault documentation.
In Role Name, enter the role you have configured in the Vault.
In Service Account Token Path enter the JSON Web Token (JWT) path. This is the path where the JWT token is mounted. The default path of this token is /var/run/secrets/kubernetes.io/serviceaccount/token.
For more information, go to Service Account Tokens in the Kubernetes documentation.
Option: JWT/OIDC Auth
FEATURE AVAILABILITY
This feature is supported with Delegate version: 25.02.85100 and above.
JWT/OIDC authentication allows you to authenticate with HashiCorp Vault using JWT token in just a few steps.

Steps to enable Vault authentication

Enable the JWT authentication method on custom path:
Enable any custom path with <YOUR_PATH>. For example, we have used harness/jwt below.
```
vault auth enable -path=harness/jwt jwt  
```
Configure the JWT authentication method:
- Set the OIDC discovery URL or manually specify the JWT issuer with https://app.harness.io/ng/api/oidc/account/<YOUR_ACCOUNT_ID/>. Harness exposes endpoints with discovery url for publishing the OpenID configuration and RSA public key.
  vault write auth/harness/jwt/config oidc_discovery_url="<OIDC_DISCOVERY>" bound_issuer="<BOUND_ISSUER>"
  bound_issuer is same as oidc_discovery_url i.e., https://app.harness.io/ng/api/oidc/account/<YOUR_ACCOUNT_ID/>
Verify JWT Configuration:
After setting the OIDC discovery URL or manually specifying the JWT issuer, you can verify the configuration by running the following command:
```
curl --header "X-Vault-Token: YOUR_ROOT_TOKEN" http://<VAULT_DOMAIN_IP>/v1/auth/harness/jwt/config
```
This will return the current configuration, showing details like bound_issuer, oidc_discovery_url, and other settings.
Create a Role for JWT Authentication:
Create a role that maps JWT claims to Vault policies. Define this role in a file, such as role-config.json. Here's an example of how to configure it:
- bound_audiences: Set this to the audience (aud) in the JWT claims, matching the JWT authentication mount path (harness/jwt).
- bound_claims: Specify the claims you want to validate (e.g., sub, iss, account_id).
- policies: Define the policies associated with the client token.
- ttl: Set the time-to-live for the generated token.
- role_type: Roles allow you to group configuration settings together to simplify plugin management. Set this to jwt.
Example role-config.json:
The sub field can have the following format, depending on the scope of the authentication method configured in Harness:
- For account scope only: `<ACCOUNT_ID>` - With the `PL_OIDC_ENHANCED_SUBJECT_FIELD` feature flag enabled: `account/{account_id}:org/{organization_id}:project/{project_id}` - Leave org and/or project blank for an account level or org level connector. For example, account level would be `account/{account_id}:org/:project/`
```
{
"bound_audiences": ["<CUSTOM_MOUNT_PATH>"],
"user_claim": "sub",
"bound_claims": {
   "sub": "<YOUR_ACCOUNT_ID>",
   "iss": "<SAME_AS_OIDC_DISCOVERY_URL>",
   "account_id": "<YOUR_ACCOUNT_ID>"
},
"policies": ["<YOUR_POLICIES>"],
"ttl": "1h",
"max_ttl": "4h",
"role_type": "jwt"
}
```

Set JWT auth role(s) and associated context

These feature is behind the PL_ENABLE_GRANULAR_CLAIMS_FOR_VAULT feature flag. please contact Harness Support to enable the feature.

To configure JWT authentication with Vault, you can add one or more roles. Each role specifies how incoming JWTs are validated and which claims must be present.

Roles can be used in different contexts based on your configuration, as shown below:

General Context – This Vault role is used to validate JWTs in non-pipeline scenarios, such as testing the connector or manually creating secrets.
Pipeline Context – This role is used when the JWT contains pipeline-related claims (for example, pipeline_id, execution_id, or environment_id). It applies only during pipeline executions.

You can also customize the claims to suit your specific use case. For example, you might want to validate a JWT based on a custom claim that is specific to your organization. In this case, you can add the custom claim to the bound_claims object as shown below.

  ```json
  {
  "bound_audiences": ["<CUSTOM_MOUNT_PATH>"],
  "user_claim": "sub",
  "bound_claims": {
     "sub": "<YOUR_ACCOUNT_ID>",
     "iss": "<SAME_AS_OIDC_DISCOVERY_URL>",
     "account_id": "<YOUR_ACCOUNT_ID>",
     "pipeline_id": "<PIPELINE_ID>",
     "execution_id": "<EXECUTION_ID>",
     "environment_id": "<ENVIRONMENT_ID>"
  },
  },
  "policies": ["<YOUR_POLICIES>"],
  "ttl": "1h",
  "max_ttl": "4h",
  "role_type": "jwt"
  }
  ```

OIDC claims supported in Harness for HashiCorp Vault

Trusted Claims:

Harness validates the following claims internally to determine if the principal has the required permissions. When configuring trust on the Cloud Provider side, only these specific claims and their exact values should be accepted. Any claims outside this list must be rejected to avoid unauthorized access.
- account_id
- organization_id
- project_id
- pipeline_id
The following claims are validated for existence in Harness, but do not include an access check:
- environment_id
- connector_id
- service_id

Non-Trusted Claims

The following claims are considered non-trusted. They are not validated for existence or access control and are used for informational context only:
- environment_type
- connector_name
- service_name
- triggered_by_name
- trigger_by_email
- stage_type
- step_type

Custom Parameters

Here are the custom parameters for the Hashicorp Vault (OIDC JWT):

account_id: The account id of your Harness account.
account_name: The name of your Harness account.
organization_id: The organization id of your Harness organization.
project_id: The project id of your Harness project.
pipeline_id: The pipeline identifier.
environment_id: The environment identifier.
environment_type: The type of the environment (for example, Production, Pre-Production).
triggered_by_name: The name of the user who triggered the pipeline.
trigger_by_email: The email address of the user who triggered the pipeline.
stage_type: The type of stage being executed in the pipeline.
step_type: The type of step being executed within the stage.
connector_id: The identifier of the connector used in the execution.
connector_name: The name of the connector used in the execution.
service_id: The service identifier associated with the pipeline execution.
service_name: The name of the service associated with the pipeline execution.

Apply and verify the Role Configuration:

Use the following command to create the role in Vault:

curl --header "X-Vault-Token: YOUR_ROOT_TOKEN" --request POST --data @role-config.json http://<VAULT_DOMAIN_IP>/v1/auth/harness/jwt/role/role_assigned

To ensure that the role was created successfully, run the following command:

curl --header "X-Vault-Token: YOUR_ROOT_TOKEN" http://<VAULT_DOMAIN_IP>/v1/auth/harness/jwt/role/role_assigned

Configure in HashiCorp Vault in Harness

Configure directly through Harness, without using a delegate:
In this step, the customer needs to whitelist the Harness Platform IPs to enable connectivity to Vault.
After setting up your Vault URL, Authentication, Path, and Role, choose the connectivity mode: either using a delegate or without a delegate. For now, we will use the "Connect through Harness Platform" option, as shown in the image below.
Next, set up the engine by either auto-fetching or manually configuring it. Finally, the system will check the connection. Click "Finish" to complete the setup.
Configure directly through Harness Delegate:
Repeat the steps by configuring Vault URL, Authentication, Path and role, choose "Connect through Harness Delegate" option in connectivity mode.
Next, set up the engine (auto-fetch or manual). Once the connection is verified, click "Finish" to complete the setup.

Select Secret Engine and Version

Once you have entered the required fields, you can choose to Fetch Engines or Manually Configure Engine.

Fetch Engines

If you want Harness to automatically fetch secret engines, include this read permission for sys/mounts in the ACL policy.

  ```json
  path "sys/mounts"{
  capabilities = ["read"]
  }
  ```

Select Fetch Engines. Harness will populate the Secret Engine drop-down with the list of engines and their versions.

Select the engine you want to use.

Manually Configure Engine

If you don't want to or cannot add the ACL policy (with read permission for sys/mounts) in the Secret Manager, perform the following steps:

Identify the engine version of the Secret Manager in Vault.
In Secret Engine Name, enter the name of the Secret Engine.
In Secret Engine Version, enter the engine version.
You cannot change the Secret Engine later. Harness blocks editing this setting later since there might be secrets that are created/referenced under this secret engine. Changing the secret engine might break references to those secrets.
In Kubernetes Auth Endpoint, enter the authentication endpoint, if configured. The default value is Kubernetes.

Renewal Interval (minutes)

In Renew Interval, you can (optionally) enter how often the Harness Delegate reloads the Vault access token.

You can expect a delay during the Vault renewal. A periodic job runs to check if there has to be a renewal, resulting in a delay of no more than two minutes.

Review: Validating Non-Read Only Vault Secret Managers

To validate a non-read-only Vault Secret Manager, Harness creates a dummy secret in the secret engine. The path of the secret is as follows:

v2 Secret Engine: <SECRET_ENGINE_NAME>/data/\<BASE_PATH>/harness_vault_validation#value
v1 Secret Engine: <SECRET_ENGINE_NAME>/\<BASE_PATH>/harness_vault_validation#value

The secret can fail because of various reasons.

Using the Token/App Role, the Vault authentication is not successful.
The following permission is not available in any of the policies attached to the Token/App Role. If this permission is not available, the user will not be able to fetch the list of secret engines from the customer vault and Harness will show a single option of Secret Engine named **"secret"**with version 2, which might be incorrect for the customer.
Make sure to add the permission to a policy attached to the Token/App Role as follows:
```
path "sys/mounts"{
capabilities = ["read"]
}
```
The policy attached to the Token/AppRole does not provide the write permission in the specified path. Make sure you update the policies and permissions.

Read-only Vault

If required by your organization's security practices, select the Read-only Vault option. This selection authorizes Harness to read secrets from Vault, but not to create or manage secrets within Vault.

Once you have filled out the required fields, select Finish.

Read-only Limitations

If you select Read-only Vault, there are several limitations on the resulting Harness Vault Secret Manager.

Also a read-only Harness Vault Secret Manager:

Cannot be used in the Add Encrypted File dialog.
Cannot create inline secrets in the Add Encrypted Text modal.

Delegate selector considerations

When configuring a HashiCorp Vault secret manager, consider how delegate selectors affect secret resolution for connectors that reference secrets stored in Vault.

DELEGATE SELECTOR COMPATIBILITY

If you configure this Vault connector with a delegate selector, ensure that other connectors (such as GitHub connectors, Kubernetes cluster connectors, or cloud provider connectors) that reference secrets from this Vault use compatible delegate selectors.

How it works:

When a connector references a secret stored in Vault, the delegate executing the task must be able to access both the connector's endpoint and the Vault secret manager. If the connector and Vault use different, non-overlapping delegate selectors, secret resolution will fail.

Best practices:

Use no delegate selector on your Vault connector to allow any delegate to access it, OR
Ensure that connectors referencing Vault secrets use the same delegate selector as the Vault connector
If using multiple delegate selectors, ensure delegates have overlapping tags between the connector and Vault

For more information about delegate selector priority and secret resolution, go to Use delegate selectors.

Test Connection

Once the Test Connection succeeds, select Finish. You can now see the connector in Connectors.

IMPORTANT

Harness tests connections by creating a dummy secret in the Secret Manager or Vault. For the Test Connection to function successfully, make sure you have Create permission for secrets.

The Test Connection fails if you do not have Create permission. However, Harness still creates the connector for you. You may use this connector to read secrets, if you have View permissions.

Reference JSON secrets

PreviousSecret Manager Template Reconciliation

NextAdd an Azure Key Vault secret manager

Last updated 5 days ago

Was this helpful?