Skip to main content

Getting Started with Workflows

Goal​

This will help you setup a Workflow in IDP which automates GitHub Repo Onboarding for users.

info

We now have a new Workflows homepage which is behind a Feature Flag IDP_ENABLE_WORKFLOWSV2. To enable the feature flag, please contact Harness Support

Pre-requisite​

Add Connector​

  1. Go to Admin -> Git Integrations.
  2. Add a New Git Integration with type as GitHub.
  3. Now Select the GitHub Connector and Save Changes, follow the steps to add a new connector
info

Make sure the Connector URL used here is pointed towards the org where you want to create the new repo and the token used have write permission for the repo also Enable the API while creating connector.

Create Harness Pipeline with IDP Stage​

  1. Go to Admin -> Pipelines.
  2. Now start with Create a Pipeline.

  1. Add a Name, select the type as Inline and Continue.

  1. Add Stage and Select Stage Type as Developer Portal.

  1. Select the infrastructure as Harness Cloud.

  1. Now let's create some variables which we will be using in further steps, add the following Custom Variables (stick to the exact naming convention for variables as provided as these will be used as a reference to input values into the pipeline)
    • github_org
    • github_repo
    • github_username The variables are of Type String and the Values are set to Runtime Input.

Add another variable github_token as Type Secret and add the GitHub PAT for Values.

Apply the changes

  1. Now Continue to Execution and Add Step, under Git Repository Setup select Create Repo.

  1. Under Step Parameters, select the Repository Type as Public if the connector you have created use a token generated by your GitHub admin, else you might not have access to the private repo created.

  2. Add the Connector you created under Select codebase repo connector.

  3. Now under Organization and Repository Name add the variables you created as Expressions (<+pipeline.variables.github_org> and <+pipeline.variables.github_repo>).

  4. Add a Repository Description, Apply Changes.

  1. Now add a Run Step to add your username as an admin to the repo created, with the following curl command, make sure to add a container registry and image, under optional configuration.
curl -L \
  -X PUT \
  -H "Accept: application/vnd.github+json" \
  -H "Authorization: Bearer <+pipeline.variables.github_token>" \
  -H "X-GitHub-Api-Version: 2022-11-28" \
  https://api.github.com/repos/<+pipeline.variables.github_org>/<+pipeline.variables.github_repo>/collaborators/<+pipeline.variables.github_username> \
  -d '{"permission":"admin"}'

Create Workflow​

If you want to use our sample Workflows you can skip this step and go to register Workflow step

The Workflow is defined in a YAML file named workflow.yaml. The syntax of the template definition is owned by backstage.io while the workflow runs on a Harness pipeline of your choice.

The following is the example Workflow with Workflows Action to create a new repository.

apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
# some metadata about the template itself
metadata:
  name: Workflows-Quickstart
  title: Create a New GitHub Repository
  description: Creates a new GitHub Repository 
spec:
  owner: service.owner
  type: service
  # these are the steps which are rendered in the frontend with the form input
  parameters:
    - title: Fill in the repo details
      required:
        - github_org_name
        - github_repo_name
        - github_username
      properties:
        github_org_name:
          title: GitHub Organization
          type: string
          description: Name the org where you want to add the new repository
        github_repo_name:
          title: Repository Location
          type: string
          description: Name the new repository that you want to create 
        github_username:
          title: GitHub Username
          type: string
          description: Add Your GitHub Username 
        token:
          title: Harness Token
          type: string
          ui:widget: password
          ui:field: HarnessAuthToken       
  # here's the steps that are executed in series in the scaffolder backend
  steps:
    - id: trigger
      name: Onboarding a New GitHub Repository
      action: trigger:harness-custom-pipeline
      input:
        url: "URL_of_the_Harness_Pipeline_Created_Above"
        inputset:
          github_org: ${{ parameters.github_org_name }}
          github_repo: ${{ parameters.github_repo_name }}
          github_username: ${{parameters.github_username}}
        apikey: ${{ parameters.token }}
    # The final step is to register our new component in the catalog.

  output:
    links:
      - title: Pipeline Details
        url: ${{ steps.trigger.output.PipelineUrl }}

In the above Workflow we have used the Harness Trigger Custom Action, which takes the exact variable name github_org and github_repo you created for your pipeline as input and a token as apikey

info

The token property we use to fetch Harness Auth Token is hidden on the Review Step using ui:widget: password, but for this to work the token property needs to be mentioned under the first page in-case you have multiple pages.

# example workflow.yaml
...
parameters:
  - title: <PAGE-1 TITLE>
    properties:
      property-1:
        title: title-1
        type: string
      property-2:
        title: title-2
    token:
      title: Harness Token
      type: string
      ui:widget: password
      ui:field: HarnessAuthToken
  - title: <PAGE-2 TITLE>
    properties:
      property-1:
        title: title-1
        type: string
      property-2:
        title: title-2
  - title: <PAGE-n TITLE>  
...
  1. Copy the above Workflow file in your git provider, and save it as workflow.yaml.

Register Workflow in IDP​

  1. In the left navigation, select Workflows, and then select Register New Workflow.

  1. Enter the URL to your new workflow.yaml.

  2. Select Import.

  1. Now click on the Workflow name

Use Workflows to Create a Repo​

  1. Once you're on the entity page for Workflow, Launch the Workflow.

  1. Now fill the fields as:
    • Repository Location: Add a name for the repository you wanted to create.
    • GitHub Organization: Add the org where you want to add your new repository, usually GitHub URL are of the format https://github.com/org_name/repo_name/

  1. Now Trigger the flow, it will create a new GitHub repo.

Delete the Workflow​

If you want to unregister or delete the Workflow follow the steps mentioned here.

Further Reading​

Self Service Onboarding in IDP is powered by Workflows and Harness Pipeline(IDP Stage). Follow the tutorials to learn more about it.

  1. Create a service onboarding pipeline (using IDP Stage)

  2. Getting started with service onboarding pipeline (using Custom Stage)

  3. Use Harness IDP for self serviced Harness CI/CD onboarding