Configure OIDC Authentication for Cloud Spanner and CloudSQL

Set up keyless authentication for GCP Cloud Spanner and CloudSQL databases using OpenID Connect (OIDC) with Workload Identity Federation in Harness Database DevOps.

OpenID Connect (OIDC) authentication enables your database connectors to authenticate with GCP databases without storing long-lived service account keys. Harness exchanges a short-lived OIDC token for temporary GCP credentials at runtime.

GKE-SPECIFIC ALTERNATIVE

If your delegate runs in GKE, you can use GKE Workload Identity instead of OIDC. Go to Set up keyless authentication for Google Cloud databases for a GKE-native approach that maps Kubernetes Service Accounts directly to Google Service Accounts.

This topic assumes you have experience with GCP workload identity providers.

Prerequisites

• Workload Identity Pool: A configured Workload Identity Pool in your GCP project. Go to Set up the GCP workload identity provider in the CI documentation to create the pool.

• OIDC provider: An OIDC provider configured with the correct Harness issuer URL for your account cluster. Go to Set up the GCP workload identity provider in the CI documentation to configure the provider.

• Service account access grant: The service account must grant access to your Harness account ID via workload identity pool attribute conditions. Go to Grant access to the service account in the CI documentation for configuration steps.

• Service account IAM roles: Assign roles to your GCP service account based on the database type you are connecting to:

  Workload Identity (required for all databases):

  • roles/iam.workloadIdentityUser: Allow the Workload Identity SA to impersonate the GCP service account.

  BigQuery:

  • roles/bigquery.admin: Full access to BigQuery datasets and tables.

  • roles/bigquery.jobUser: Run queries and schema operations.

  • roles/iam.serviceAccountTokenCreator: Required for OIDC token exchange.

  Cloud Spanner:

  • roles/spanner.databaseUser: Read/write access to Spanner databases.

  • roles/spanner.databaseAdmin: Schema changes and admin operations.

  CloudSQL:

  • roles/cloudsql.client: Connect to CloudSQL instances.

  • roles/cloudsql.instanceUser: IAM database authentication.

• Database instance: A Cloud Spanner instance, CloudSQL PostgreSQL/MySQL instance with IAM authentication enabled, or BigQuery project with datasets configured.

• Harness project access: Connector creation permissions in your Harness project. Go to RBAC in Harness to configure roles.

• Required GCP APIs: The following APIs must be enabled in your GCP project. Go to Enable required GCP APIs for instructions.

How OIDC authentication works

When you configure a JDBC connector with OIDC authentication:

1. Token generation: Harness generates a short-lived OIDC token (JWT) that identifies the pipeline execution or connection test request.

2. Token exchange: The token is exchanged for a GCP OAuth2 access token using Workload Identity Federation:

   • Connection test: Exchange happens on the delegate.

   • Pipeline execution: Exchange happens inside the plugin container using GCP Security Token Service (STS).

3. Service account impersonation: The access token is used to impersonate the specified service account, which has database permissions.

4. Database connection: The database connection is established using the impersonated service account credentials.

This flow eliminates the need to store service account keys in Harness or your delegate environment. The GCP access token is short-lived (1 hour) and a new token is generated for each pipeline execution or connection test.

Supported databases

OIDC authentication is available for the following GCP database types:

• Cloud Spanner: Uses OAuth2 access token authentication via the oauthToken JDBC connection property.

• CloudSQL PostgreSQL: Uses the CloudSQL Socket Factory with IAM authentication and credentials file.

• CloudSQL MySQL: Uses the CloudSQL Socket Factory with IAM authentication and credentials file.

• BigQuery: Uses OAuth2 access token authentication via the Simba BigQuery JDBC driver.

Generic PostgreSQL or MySQL databases without CloudSQL Socket Factory are not supported.

BigQuery prerequisites

In addition to the common OIDC prerequisites, BigQuery requires:

Required GCP APIs:

# BigQuery API for data access <a href="#bigquery-api-for-data-access" id="bigquery-api-for-data-access"></a>
gcloud services enable bigquery.googleapis.com --project=YOUR_PROJECT_ID

# IAM and STS APIs (required for OIDC token exchange) <a href="#iam-and-sts-apis-required-for-oidc-token-exchange" id="iam-and-sts-apis-required-for-oidc-token-exchange"></a>
gcloud services enable iamcredentials.googleapis.com \
  sts.googleapis.com \
  --project=YOUR_PROJECT_ID

Required IAM roles for the service account:

SA_EMAIL="your-sa@PROJECT_ID.iam.gserviceaccount.com"
PROJECT_ID="your-project-id"

# BigQuery data access (choose based on requirements) <a href="#bigquery-data-access-choose-based-on-requirements" id="bigquery-data-access-choose-based-on-requirements"></a>
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:$SA_EMAIL" \
  --role="roles/bigquery.dataViewer"  # For read-only access

# Or use admin for full access <a href="#or-use-admin-for-full-access" id="or-use-admin-for-full-access"></a>
# --role="roles/bigquery.admin" <a href="#rolerolesbigqueryadmin" id="rolerolesbigqueryadmin"></a>

# Job execution (required for running queries) <a href="#job-execution-required-for-running-queries" id="job-execution-required-for-running-queries"></a>
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:$SA_EMAIL" \
  --role="roles/bigquery.jobUser"

# Service Account Token Creator (for OIDC token exchange) <a href="#service-account-token-creator-for-oidc-token-exchange" id="service-account-token-creator-for-oidc-token-exchange"></a>
gcloud iam service-accounts add-iam-policy-binding $SA_EMAIL \
  --project=$PROJECT_ID \
  --role="roles/iam.serviceAccountTokenCreator" \
  --member="serviceAccount:$SA_EMAIL"

Role descriptions:

• roles/bigquery.dataViewer: Read-only access to BigQuery datasets and tables.

• roles/bigquery.admin: Full access to BigQuery resources.

• roles/bigquery.jobUser: Required to run BigQuery jobs (queries and schema operations).

• roles/iam.serviceAccountTokenCreator: Allows generating access tokens during OIDC exchange.

JDBC URL formats

OIDC authentication requires specific URL formats for each database type.

Cloud Spanner URL format

jdbc:cloudspanner:/projects/PROJECT_ID/instances/INSTANCE_NAME/databases/DATABASE_NAME

Example:

jdbc:cloudspanner:/projects/my-project/instances/spanner-test/databases/cymbal

POSTGRESQL NOT SUPPORTED FOR SPANNER

The PostgreSQL dialect for Spanner is not currently supported and will cause pipeline failures with an image pull error. Use the standard Cloud Spanner GQL.

CloudSQL PostgreSQL URL format

jdbc:postgresql:///DATABASE_NAME?cloudSqlInstance=PROJECT_ID:REGION:INSTANCE_NAME&socketFactory=com.google.cloud.sql.postgres.SocketFactory&enableIamAuth=true

Example:

jdbc:postgresql:///mydb?cloudSqlInstance=my-project:us-central1:my-instance&socketFactory=com.google.cloud.sql.postgres.SocketFactory&enableIamAuth=true

Required parameters:

• cloudSqlInstance: Instance connection name in the format project-id:region:instance-name.

• socketFactory: Must be com.google.cloud.sql.postgres.SocketFactory.

• enableIamAuth: Must be true.

CloudSQL MySQL URL format

jdbc:mysql:///DATABASE_NAME?cloudSqlInstance=PROJECT_ID:REGION:INSTANCE_NAME&socketFactory=com.google.cloud.sql.mysql.SocketFactory&enableIamAuth=true

Example:

jdbc:mysql:///mydb?cloudSqlInstance=my-project:us-central1:my-instance&socketFactory=com.google.cloud.sql.mysql.SocketFactory&enableIamAuth=true

Required parameters:

• cloudSqlInstance: Instance connection name in the format project-id:region:instance-name.

• socketFactory: Must be com.google.cloud.sql.mysql.SocketFactory.

• enableIamAuth: Must be true.

If any required parameter is missing, the connection test will fail with a validation error before attempting to connect.

BigQuery URL format

jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId=PROJECT_ID;DefaultDataset=DATASET_NAME;Location=REGION;

Example:

jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId=cd-play;DefaultDataset=Step_execution_data;Location=asia-south1;

Required parameters:

• ProjectId: Your GCP project ID where BigQuery datasets reside.

• DefaultDataset: The default BigQuery dataset for schema operations.

• Location: The BigQuery dataset location (for example, us-central1, asia-south1).

IMPORTANT

When using OIDC authentication, the OAuth access token is injected automatically during the connection. Do not include OAuthType or OAuthAccessToken parameters in the URL. The Simba BigQuery JDBC driver is included in the Harness Database DevOps plugin images.

Before you configure OIDC

Complete the GCP Workload Identity Federation setup before configuring your JDBC connector. Go to Set up the GCP workload identity provider in the CI documentation, which covers issuer URLs for all Harness account clusters (Prod-1, Prod-2, Prod-3) and the attribute condition required for both connection tests and pipeline executions. Return to this page after completing those steps.

Configure OIDC authentication for Cloud Spanner

1. Create a JDBC connector: In your Harness project, go to Connectors and select New Connector. Choose JDBC.

2. Enter connection details: In the Connection URL field, enter your Cloud Spanner JDBC URL in the format:

   jdbc:cloudspanner:/projects/YOUR_PROJECT_ID/instances/YOUR_INSTANCE/databases/YOUR_DATABASE

   Replace YOUR_PROJECT_ID, YOUR_INSTANCE, and YOUR_DATABASE with your Cloud Spanner resource identifiers.

3. Select OIDC authentication: In the Authentication section, select OIDC as the auth type.

4. Configure GCP OIDC details:

   • Provider Type: Select GCP.

   • Project Number: Enter your GCP project number (numeric identifier, not project ID). Go to the GCP Console dashboard to find the project number.

   • Workload Pool ID: Enter the Workload Identity Pool ID you created in Set up the GCP workload identity provider. This is the Pool ID value shown in the GCP Console under IAM & Admin > Workload Identity Federation.

   • Provider ID: Enter the OIDC provider ID within the pool. This is the Provider ID value shown when you select the provider in the GCP Console.

   • Service Account Email: Enter the email of the service account that has Spanner Database User permissions (for example, [e-mail]).

5. Test the connection: Select Test Connection to verify that the delegate can authenticate and connect to Cloud Spanner.

   The connection test runs on the delegate and exchanges the Harness OIDC token for a GCP access token before connecting to the database.

   

   JDBC connector test for Cloud Spanner with OIDC authentication

Configure OIDC authentication for CloudSQL

1. In your Harness project, go to Connectors and select New Connector. Choose JDBC.

2. In the Connection URL field, enter your CloudSQL JDBC URL with the following required parameters:

   • PostgreSQL format: jdbc:postgresql:///YOUR_DATABASE?cloudSqlInstance=PROJECT_ID:REGION:INSTANCE_NAME&socketFactory=com.google.cloud.sql.postgres.SocketFactory&enableIamAuth=true

   • MySQL format: jdbc:mysql:///YOUR_DATABASE?cloudSqlInstance=PROJECT_ID:REGION:INSTANCE_NAME&socketFactory=com.google.cloud.sql.mysql.SocketFactory&enableIamAuth=true

   In above URLs, replace the placeholders with your CloudSQL resource identifiers:

   • YOUR_DATABASE: The database name within the CloudSQL instance.

   • PROJECT_ID:REGION:INSTANCE_NAME: Your CloudSQL instance connection name (for example, my-project:us-central1:my-instance).

   IMPORTANT

   Required URL parameters for OIDC authentication:**

   • cloudSqlInstance: The CloudSQL instance connection name in the format project-id:region:instance-name.

   • socketFactory: The CloudSQL Socket Factory class for your database type:

     • PostgreSQL: com.google.cloud.sql.postgres.SocketFactory

     • MySQL: com.google.cloud.sql.mysql.SocketFactory

   • enableIamAuth=true: Enables IAM authentication.

   If any of these parameters are missing, the connection test will fail with a validation error.

3. Select OIDC authentication: In the Authentication section, select OIDC as the auth type.

4. Configure GCP OIDC details:

   • Provider Type: Select GCP.

   • Project Number: Enter your GCP project number (numeric identifier, not project ID). Go to the GCP Console dashboard to find the project number.

   • Workload Pool ID: Enter the Workload Identity Pool ID you created in Set up the GCP workload identity provider. This is the Pool ID value shown in the GCP Console under IAM & Admin > Workload Identity Federation.

   • Provider ID: Enter the OIDC provider ID within the pool. This is the Provider ID value shown when you select the provider in the GCP Console.

   • Service Account Email: Enter the email of the service account that has CloudSQL IAM user permissions (for example, [e-mail]).

5. Select Test Connection to verify that the delegate can authenticate and connect to CloudSQL.

   The connection test runs on the delegate and uses the CloudSQL Socket Factory to establish an IAM-authenticated connection.

   

   JDBC connector test for CloudSQL with OIDC authentication

Username derivation: The database username is derived automatically from the service account email:

• PostgreSQL: Strips the .gserviceaccount.com suffix. For example, [e-mail] becomes [e-mail].

• MySQL: Uses the prefix before the @ symbol. For example, [e-mail] becomes db-sa.

Ensure that a database user with this username exists in your CloudSQL instance and is granted appropriate permissions. Go to CloudSQL IAM authentication to create IAM database users.

Configure OIDC authentication for BigQuery

1. In your Harness project, go to Connectors and select New Connector. Choose JDBC.

2. In the Connection URL field, enter your BigQuery JDBC URL:

   jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId=YOUR_PROJECT_ID;DefaultDataset=YOUR_DATASET;Location=YOUR_REGION;

   Replace the placeholders with your BigQuery resource identifiers:

   • YOUR_PROJECT_ID: Your GCP project ID where BigQuery datasets reside.

   • YOUR_DATASET: The default BigQuery dataset for schema operations.

   • YOUR_REGION: Optional. The BigQuery dataset location (for example, us-central1, asia-south1).

3. In the Authentication section, select OIDC as the auth type.

4. Configure GCP OIDC details:

   • Provider Type: Select GCP.

   • Project Number: Enter your GCP project number (numeric identifier, not project ID). Go to the GCP Console dashboard to find the project number.

   • Workload Pool ID: Enter the Workload Identity Pool ID you created in Set up the GCP workload identity provider. This is the Pool ID value shown in the GCP Console under IAM & Admin > Workload Identity Federation.

   • Provider ID: Enter the OIDC provider ID within the pool. This is the Provider ID value shown when you select the provider in the GCP Console.

   • Service Account Email: Enter the email of the service account that has BigQuery permissions (for example, [e-mail]).

5. Select Test Connection to verify that the delegate can authenticate and connect to BigQuery.

   The connection test runs on the delegate and exchanges the Harness OIDC token for a GCP access token before connecting to BigQuery using the Simba BigQuery JDBC driver.

IMPORTANT

The service account must have the following IAM roles:

• roles/bigquery.dataViewer or roles/bigquery.admin (for dataset access)

• roles/bigquery.jobUser (for running queries)

• roles/iam.serviceAccountTokenCreator (for OIDC token exchange)

The BigQuery JDBC driver (Simba) is included in the Harness Database DevOps plugin images. No additional driver configuration is required.

Use OIDC connectors in pipelines

When you reference a JDBC connector with OIDC authentication in a Database DevOps step (Liquibase or Flyway), Harness automatically handles the token exchange and authentication flow.

Pipeline execution flow

1. Token generation: Harness generates a pipeline-scoped OIDC token (JWT) that includes custom attributes identifying the pipeline, organization, project, and connector.

2. Environment variables: The following environment variables are passed to the plugin container:

   • PLUGIN_GCP_OIDC_TOKEN: Harness OIDC JWT.

   • PLUGIN_GCP_OIDC_WORKLOAD_POOL_ID: Workload Identity Pool ID.

   • PLUGIN_GCP_OIDC_PROVIDER_ID: OIDC Provider ID.

   • PLUGIN_GCP_OIDC_PROJECT_ID: GCP Project Number.

   • PLUGIN_GCP_OIDC_SERVICE_ACCOUNT_EMAIL: Service Account Email.

3. Token exchange in plugin: The plugin container exchanges the Harness OIDC token for a GCP access token using GCP Security Token Service (STS). This exchange happens inside the plugin container and the GCP access token never leaves the customer infrastructure.

4. Database authentication: The plugin uses the GCP access token to authenticate with the database:

   • Cloud Spanner: Token is passed as the oauthToken JDBC connection property.

   • CloudSQL: Token is written to a credentials file and the CloudSQL Socket Factory handles authentication.

5. Plugin image selection: For CloudSQL PostgreSQL/MySQL connectors with OIDC authentication, Harness automatically selects the CloudSQL-compatible plugin image that includes the Socket Factory.

No additional configuration is required in the step definition. The token exchange and authentication are fully automated.

Example pipeline execution:

The following screenshot shows a successful Database DevOps Apply step execution using OIDC authentication for both Cloud Spanner and CloudSQL databases:

Database DevOps Apply step execution with OIDC authentication

Connector JSON structure

The OIDC connector uses a polymorphic structure that supports multiple cloud providers. The current implementation supports GCP, and the schema is designed to allow future extension to AWS and Azure without breaking changes.

Connector JSON Structure

CloudSQL PostgreSQL example:

{
  "connector": {
    "name": "CloudSQL OIDC Connector",
    "identifier": "cloudsql_oidc",
    "type": "Jdbc",
    "spec": {
      "connectionUrl": "jdbc:postgresql:///mydb?cloudSqlInstance=my-project:us-central1:my-instance&socketFactory=com.google.cloud.sql.postgres.SocketFactory&enableIamAuth=true",
      "auth": {
        "type": "Oidc",
        "spec": {
          "providerType": "Gcp",
          "providerSpec": {
            "type": "Gcp",
            "spec": {
              "projectNumber": "145904791365",
              "workloadPoolId": "harness-identity-pool",
              "providerId": "harness-oidc-provider",
              "serviceAccountEmail": "[e-mail]"
            }
          }
        }
      }
    }
  }
}

Keyless Connector JSON Structure

Cloud Spanner example:

{
  "connector": {
    "name": "Spanner OIDC Connector",
    "identifier": "spanner_oidc",
    "type": "Jdbc",
    "spec": {
      "connectionUrl": "jdbc:cloudspanner:/projects/my-project/instances/my-instance/databases/my-database",
      "auth": {
        "type": "Oidc",
        "spec": {
          "providerType": "Gcp",
          "providerSpec": {
            "type": "Gcp",
            "spec": {
              "projectNumber": "145904791365",
              "workloadPoolId": "harness-identity-pool",
              "providerId": "harness-oidc-provider",
              "serviceAccountEmail": "[e-mail]"
            }
          }
        }
      }
    }
  }
}

Next steps

Now that you have configured OIDC authentication, you can use your connector in Database DevOps pipelines. Go to Create a Database DevOps pipeline to build automated database change workflows.