Skip to main content

Getting started with service onboarding pipeline (using Custom Stage)

This tutorial is designed to help a platform engineer to get started with Harness IDP. We will create a basic service onboarding pipeline that uses a software template and provisions a Next.js application for a developer. After you create the software template, developers can choose the template on the Create page and enter details such as a name for the application and the path to their Git repository. The service onboarding pipeline creates a hello world repository for storing code.

Your users (developers) must perform a sequence of tasks to create the application. First, they interact with a software template. A software template is a form that collects a user's requirements. After a user submits the form, IDP executes a Harness pipeline that onboard the new service. Usually the pipeline fetches a hello-world skeleton code, creates a new repository, and interacts with third-party providers such as cloud providers, Jira, and Slack.


Before you begin this tutorial, make sure that you fulfil the following requirements:

  • Enable Harness IDP for your account.
  • Obtain a CI or CD license if you do not have one. This is a temporary requirement.

Create a pipeline

Begin by creating a pipeline for onboarding the service.

Create a Build or Custom stage

To create a Build or Custom stage, perform the following steps:

  1. In the sidebar of the Harness application, select Projects, and then select a project.

You can also create a new project for the service onboarding pipelines. Eventually, all the users in your account should have permissions to execute the pipelines in this project. For information about creating a project, go to Create organizations and projects.

  1. Select Pipelines, and then select Create a Pipeline.

  2. In Pipeline Studio, select Add Stage.

  1. In Select Stage Type, select Custom Stage. (If you have a Harness CI license, you could also use the Build stage type. However, for this tutorial, we recommend that you use the Custom Stage type.)

  1. In Stage Name, enter a name for the stage, and then click Set Up Stage.

  2. Select Add step, and then, in the menu that appears, select Add Step.

A sidebar with available steps is displayed.

  1. Select Container Step to run a Python CLI called cookiecutter. We need a publicly available Python image for this purpose. You can use Container Step for any such project generators (for example, yeoman).

In the CI or Build stage type, container step is named Run, and it has the same functionality.

  1. Configure the step as follows:

    1. Enter a name for the step. For example, name it Create React app.

    2. You can enter 10m (10 minutes) in the Timeout field.

    3. In Container Registry, create or choose an anonymous Docker connector that connects to DockerHub (

    4. In Image, enter python.

    Before we write the command, we must make an infrastructure choice, which means that we specify where the pipeline executes. You can execute the pipeline on your own infrastructure or on the Harness platform. If you have an existing delegate set up for deployments, you can use the associated connector and specify its Kubernetes namespace. If you want to use the Harness platform, you have to use the CI or Build stage type instead of the Custom stage type and choose the Harness platform as your infrastructure.


    Depending upon our operation, we might have to adjust the memory limit of the container. If required, you can change Limit Memory from 500Mi to 4000Mi.

Cookiecutter Scripts Based on your SCM

  1. Paste the following cookiecutter-based script into Command.

    The script performs the following tasks:

    1. Generates a basic Next.js app.

    2. Creates a repository with the contents. The sample code used in the command is available here, which essentially is a cookiecutter project. You can choose from available cookiecutter projects or create your own project from scratch.

    # Testing path

    # Pre-cleanup in case pipeline fails

    rm -rf idp-samples/
    rm -rf "<+pipeline.variables.project_name>"

    # Clone skeleton
    git clone

    # Generate code to be pushed
    pip install cookiecutter
    cookiecutter idp-samples/idp-pipelines/cookiecutter-react-app/ app_name="<+pipeline.variables.project_name>" --no-input

    # Add catalog-info.yaml content
    echo "apiVersion:
    kind: Component
    name: <+pipeline.variables.project_name>
    description: This is a nextjs app.
    annotations: dir:.
    type: documentation
    lifecycle: experimental" > catalog-info.yaml

    # Create repository
    curl -L -i -X POST -H "Accept: application/vnd.github+json" -H "Authorization: Bearer <+pipeline.variables.github_token>"<+pipeline.variables.github_org>/repos -d "{\"name\":\"<+pipeline.variables.github_repo>\",\"description\":\"<+pipeline.variables.project_name> - A Next.js app\",\"private\":false}"

    # Push the code
    cd <+pipeline.variables.project_name>/
    git init -b main
    git config --global ""
    git config --global "Harness Support"
    git add .
    git commit -m "Project init and Added catalog-info.yaml"
    git remote add origin<+pipeline.variables.github_org>/<+pipeline.variables.github_repo>.git
    git push https://<+pipeline.variables.github_token><+pipeline.variables.github_org>/<+pipeline.variables.github_repo>.git

    # Add catalog-info.yaml location to catalog
    curl --location '<+account.identifier>/idp/api/catalog/locations' \
    --header 'x-api-key: Harness PAT' \
    --header 'Harness-Account: <+account.identifier>' \
    --data-raw '{"type":"url","target":"<+pipeline.variables.github_org>/<+pipeline.variables.github_repo>/blob/main/catalog-info.yaml"}'

In the above script you need to add the Personal Access Token to make the API call to register your catalog,

  1. Click Apply Changes.

    Manage variables in the pipeline

    The script uses several pipeline variables. The variables are as follows:

  • <+pipeline.variables.project_name>
  • <+pipeline.variables.github_username>
  • <+pipeline.variables.github_token>
  • <+pipeline.variables.github_org>
  • <+pipeline.variables.github_repo>

Except for the secrets all the variables should have a runtime input type and the variable name should match with the parameter name used in the template as the values would be pre-populated from the values entered as input in the below IDP template.

For eg: <+pipeline.variables.project_name> variable is pre-populated by project_name: ${{ parameters.project_name }} under input set: in the below given template.

You can use the Variables button on the floating sidebar on the right-hand side to open the Variables page for the pipeline.

You can create any number of pipeline variables and decide their value type. Some variables, such as a GitHub token, a user name, and organization, can have a fixed value. The token used in the code above is a Harness secret whose value is decoded during pipeline execution.

Variables such as project name and GitHub repository are runtime inputs. They are needed at the time of pipeline execution. When creating a new variable, you can specify its type in the UI. For more information about reference variables, go to the reference documentation on pipeline variables.

Create a software template definition in IDP

Now that our pipeline is ready to execute when a project name and a GitHub repository name are provided, let's create the UI counterpart of it in IDP. This is powered by the Backstage Software Template. Create a template.yaml file anywhere in your Git repository. Usually, that would be the same place as your skeleton hello world code. We use the react-jsonschema-form playground to build the template. Nunjucks is templating engine for the IDP templates.


kind: Template
name: onboard-services
title: Create and Onboard a new react app
description: A template to create and onboard a new react app
- nextjs
- react
- javascript
type: service
- title: Next.js app details
- project_name
- github_repo
- github_org
title: Name of your new app
type: string
description: Unique name of the app
title: Name of the GitHub repository
type: string
description: This will be the name of Repository on Github
title: Name of the GitHub Organisation
type: string
description: This will be the name of Organisation on Github
title: GitHub PAT
type: string
ui:widget: password
title: Harness Token
type: string
ui:widget: password
ui:field: HarnessAuthToken
- id: trigger
name: Creating your react app
action: trigger:harness-custom-pipeline
project_name: ${{ parameters.project_name }}
github_repo: ${{ parameters.github_repo }}
github_org: ${{ parameters.github_org }}
github_token: ${{ parameters.github_token }}
apikey: ${{ parameters.token }}
# The final step is to register our new component in the catalog.
- title: Pipeline Details
url: ${{ steps.trigger.output.PipelineUrl }}

This YAML code is governed by Backstage. You can change the name and description of the software template. The template has the following parts:

  1. Input from the user
  2. Execution of pipeline

Let's take a look at the inputs that the template expects from a developer. The inputs are written in the spec.parameters field. It has two parts, but you can combine them. The keys in properties are the unique IDs of fields (for example, github_repo and project_name). If you recall, they are the pipeline variables that we set as runtime inputs earlier. This is what we want the developer to enter when creating their new application.

The YAML definition includes fields such as cloud provider and database choice. They are for demonstration purposes only and are not used in this tutorial.

Adding the owner

By default the owner is of type Group which is same as the User Group in Harness. In case the owner is a user you have to mention it as user:default/debabrata.panigrahi and it should only contain the user name not the complete email id.

Authenticate the request

Once you have written all the inputs that the template requires, you must add the following YAML snippet under

title: Harness Token
type: string
ui:widget: password
ui:field: HarnessAuthToken

Also the token input is used as a parameter under steps as apikey

- id: trigger
name: ...
action: trigger:harness-custom-pipeline
url: ...
key: value
apikey: ${{ parameters.token }}

This is a custom component we created to authenticate the call to execute the pipeline on the basis of the logged-in user's credentials.

Action to trigger the pipeline


The template actions currently supports only custom stage and codebase disabled CI stage with Run step, also all input, except for pipeline input as variables, must be of fixed value.

The spec.steps field contains only one action, and that is to trigger a Harness pipeline. Update the url and replace it with the URL of your service onboarding pipeline. Also, ensure that the inputset is correct and it contains all the runtime input variables that the pipeline needs.

Conditional Inputs in Templates

  1. One Of: Helps you create a dropdown in the template, where only one of all the options available could be selected.
- properties:
- java
java version:
type: "string"
- java8
- java11
  1. All Of: Helps you create a dropdown in the template, where only all the options available could be selected.
type: object
- properties:
- string
- boolean
default: true
- properties:
type: boolean
type: string
  1. Any Of: Helps you to select from multiple properties where both can't be selected together at once.
type: object
type: integer
title: Age
type: array
type: object
- properties:
type: string
- properties:
type: string
- title: First method of identification
type: string
title: First name
default: Chuck
type: string
title: Last name
- title: Second method of identification
type: string
title: ID code

For more such references and validate your conditional steps take a look at the react-json schema project.

Upload a file using template

There are 3 types of file upload.

  1. Single File
  2. Multiple Files
  3. Single File with Accept Attribute
title: Files
type: object
type: string
format: data-url
title: Single file
type: array
title: Multiple files
type: string
format: data-url
type: string
format: data-url
title: Single File with Accept attribute

How to use arrays as Harness Pipeline inputs

Harness Pipelines variables can only be 3 types, string, number and secrets, in case you want to add multiple strings and comma separated values you need to join them and send as single input parameters.

In the following template I want to pick the enum and parse the exampleVar as a string and use it as comma separated value in the inputset for pipeline. As you could see in the example below under inputset, exampleVar takes input as ${{ parameters.exampleVar.join(',') }}.

    - title: Pass Variables Here      
title: Select an option
type: array
type: string
- Option1
- Option2
- Option3
- Option1
title: Select Options
multi: true
- id: trigger
name: Call a harness pipeline, and pass the variables from above
action: trigger:harness-custom-pipeline
url: '*********/home/orgs/default/projects/*************/pipelines/*************/pipeline-studio/?storeType=INLINE'
exampleVar: ${{ parameters.exampleVar.join(',') }}
owner: ${{ parameters.owner }}
apikey: ${{ parameters.token }}

Register the template

Use the URL to the template.yaml created above and register it by using the same process for registering a new software component.

Now navigate to the Create page in IDP. You will see the newly created template appear. Try it out!

Unregister/Delete Template

  1. Navigate to the Catalog page, and select Template under Kind.

  1. Select the Template Name you want to Unregister.
  2. Now on the Template overview page, click on the 3 dots on top right corner and select Unregister Entity.

  1. Now on the Dialog box select Unregister Location.

  1. This will delete the Template.