Use Custom Script for Harness Database DevOps

Learn how to use a custom script to retrieve and manage a Liquibase changelog in Harness Database DevOps, including DB Schema setup and pipeline configuration.

In certain Database DevOps workflows, users may prefer or require a custom script to retrieve and manage their Liquibase changelog rather than using a connector or direct integration.

This guide provides a comprehensive overview of how to use a custom script for Database DevOps, including the creation of a DB Schema, deployment of the schema, and the use of pipeline variables.

Use case

This guide supports the following scenario:

• The customer maintains a Liquibase changelog in a remote repository (for example, GitHub or JFrog). This is particularly useful when the changelog is stored in a repository for which Harness does not provide a native connector.

• They want to download and extract this changelog dynamically via a custom shell script.

• The script can reference pipeline variables, such as build versions or access tokens.

Before proceeding, ensure you have set up a Docker Registry Connector to define the custom base image used in your pipeline. Go to Docker Connector Settings Reference to configure this connector.

Deploy using a custom script

By following these steps, you can effectively manage your database changes and streamline your DevOps processes.

Create a DB schema

1. Under Database DevOps in the Harness UI, navigate to DB Schema.

2. Select Add New DB Schema.

• Connect to Data Schema: Connect via Harness Connector or use a custom script to connect to the database schema stored in a remote repository.

• Schema Path: The path to the schema file in the repository. This is where the custom script writes the changelog files that Database DevOps should use during pipeline execution.

• Image: The Docker image to be used for the custom script. This should be a base image that has the necessary tools and libraries installed to execute your script.

• Shell: The shell to be used for executing the script. This can be bash or sh, both of which are supported by the Docker image.

• Script: The custom script that runs. This script should include the logic to download and extract the changelog file from the remote repository.

For example:

curl -H 'Authorization: token <+secrets.getValue("github")>'
 -H 'Accept: application/vnd.github.v3.raw'
 -L https://api.github.com/repos/Sonichigo/mux-sql/contents/liquibase.yml?ref=main -o changelog.yml

If your custom script requires variables or parameters, you can reference them using JEXL expressions. For example, you can reference file secrets securely by their ID. Go to Reference by ID to use file secrets in your script. To learn how Database DevOps supports runtime secrets specifically, go to Runtime secrets for DBOps to configure runtime secrets for Database DevOps.

1. Select Save.

2. The DB Schema is created, and you can see the details in the UI.

3. Select the DB Schema, and then select Add New DB Instance.

• Name of DB Instance: The name of the DB Instance. This is the name that is used in the pipeline to refer to this DB Instance.

• Tags (optional): Tags to be used for the DB Instance. This can be used to change which instances are shown on the migration state dashboard. If left blank, the tags are automatically generated.

• Connector: The JDBC Connector to be used for the DB Instance. This is the connector that connects to the database instance.

• Context: The context is used to control which changelogs are deployed to which environments. This is useful when you have multiple environments (for example, dev, test, and prod) and you want to deploy different changelogs to each environment.

Deployment pipeline

1. Under Database DevOps in the Harness UI, navigate to Pipelines.

2. Select Create a Pipeline.

3. Select Add Stage, and then select Custom Stage.

4. In the Stage section, create Add Step Group as the stage type.

Toggle on Enable container based execution.

1. In the Step Group section, select Add Step as the step type. Under DB DevOps, select Apply Schema as the step type.

• Select DB Schema: The DB Schema created earlier.

• Select DB Instance: The name of the DB Instance, which was created earlier in the DB Schema.

• Tags (optional): Tags to be used for the DB Instance. This can be used to identify the DB Instance in the pipeline. If left blank, the tags are automatically generated.

1. Select Apply Changes, and then select Save.

2. The pipeline is created, and you can see the details in the UI.

3. Select Run to run the pipeline.

Final result

Visual Overview

YAML Overview

pipeline:
  name: custom-changelog-script
  identifier: customchangelogscript
  projectIdentifier: default_project
  orgIdentifier: default
  tags: {}
  stages:
    - stage:
        name: deploy
        identifier: deploy
        description: "Deploy DB Schema using Custom Script"
        type: Custom
        spec:
          execution:
            steps:
              - stepGroup:
                  name: apply schema
                  identifier: apply_schema
                  steps:
                    - step:
                        type: DBSchemaApply
                        name: DBSchemaApply_1
                        identifier: DBSchemaApply_1
                        spec:
                          connectorRef: dockerHarness
                          dbSchema: customscript
                          dbInstance: test3
                          tag: my-deployment
                        timeout: 10m
                  stepGroupInfra:
                    type: KubernetesDirect
                    spec:
                      connectorRef: db
            rollbackSteps: []
          serviceDependencies: []
        tags: {}

You have created a DB Schema and deployed it using a custom script. You can use this pipeline to deploy your DB Schema to the database instance.

Next steps

• Go to Automated rollback for database schemas to configure automated rollback for failed deployments.

• Go to Configure Author DB Change to use LLM-powered change authoring.

• Go to the Troubleshooting Guide to resolve common setup and runtime errors.