Setup GitHub repository onboarding with Harness IDP
Create an end-to-end GitHub repository onboarding setup using Harness IDP workflows and pipelines.
This tutorial shows you how to create a self-service GitHub repository onboarding workflow using Harness IDP. The workflow allows developers to create and configure new GitHub repositories automatically through a simple UI form interface. This eliminates manual repository setup and ensures consistency across all projects.
Goals of the workflow
By the end of this tutorial, you will have a working workflow that delivers:
Automated Repository Creation: New GitHub repositories are created instantly with proper configuration
Automated Creation of Branch and Files: New branches and files are created and pushed to the repository automatically without any manual intervention.
Security Best Practices: Branch protection rules are automatically applied to enforce code reviews
CI/CD Integration: Webhooks are configured to trigger automated pipelines on specific changes
Self-Service Experience: Developers can create repositories without platform team intervention
Workflow components
The GitHub repository onboarding workflow consists of two main components that work together:
1. Harness pipeline
This orchestration pipeline contains all the automation logic and executes the actual repository creation steps. This Harness IDP pipeline serves as the orchestration backend for our GitHub repository onboarding workflow. Think of it as the engine that powers the automation - when a developer fills out the workflow form (the UI part), this pipeline executes all the technical steps needed to create and configure a GitHub repository.
2. Harness IDP workflow
The Workflow provides a user-friendly interface where developers input the following details and can create a repository without any intervention and runs the above orchestration pipeline in backend to execute the final actions of the Workflow. The Workflow includes validation to ensure proper input format and triggers the above pipeline execution to ensure all the steps are executed successfully.
The following input details are required for the Workflow:
Repository name and description
GitHub organization
Feature branch name
Before you begin
Harness IDP enabled for your Account: Ensure you have access to Harness with the Internal Developer Portal (IDP) module enabled.
GitHub Connector: A GitHub connector must be set up and configured to authenticate with your GitHub organization. This connector is used by the
CreateRepoandDirectPushsteps to interact with GitHub APIs securely. Go to GitHub Connector to learn more.GitHub PAT Token: A Personal Access Token (PAT) with appropriate permissions (
repo,admin:repo_hook,admin:org_hook) is needed for GitHub API operations. The pipeline uses this token in theRunsteps to execute GitHub API calls for branch creation, branch protection setup, and webhook configuration. Go to GitHub PAT to learn more.Workflow YAML & Harness IDP Pipeline configuration: Basic familiarity with Workflow YAML syntax and Harness pipeline configuration is recommended. Understanding YAML structure helps in configuring pipeline variables and step parameters correctly. Go to Workflow YAML and Harness IDP Pipeline to learn more.
Create the GitHub repository onboarding pipeline
This Harness IDP pipeline serves as the orchestration backend for our GitHub repository onboarding workflow. In this section, we will go through all the steps required to develop this orchestration pipeline for this Workflow.
Pipeline design
Steps involved in this Pipeline:
The following steps will be executed in this pipeline:
Step 1: Create a new GitHub repository with proper configuration
Step 2: Set up a feature branch for development
Step 3: Create a directory
Step 4: Create standard project files
Step 5: Push the files to the feature branch
Step 6: Configure branch protection rules for security
Step 7: Set up webhooks for CI/CD integration
Pipeline Execution:
The pipeline is triggered automatically when a developer submits the IDP workflow form. The workflow acts as the user interface, collecting input parameters, while this pipeline handles the backend automation. For more detailed information about Harness IDP pipelines, refer to the Harness IDP Pipeline Documentation.
Get started
Let us walk through setting up this pipeline step by step:
Step 1: Create a new pipeline
Navigate to your Harness project.
Go to Pipelines in the left sidebar from Harness IDP.
Click + Create a Pipeline
Choose Inline pipeline creation
Enter the following details:
Name:
GitHub Repository OnboardingDescription:
Automated GitHub repository creation and configuration pipeline
Step 2: Add a developer portal stage
Once you are inside the pipeline editor:
Click + Add Stage.
Select Developer Portal as the stage type. This opens a stage configuration screen with four tabs: Overview, Infrastructure, Execution, and Advanced.
Hereβs what youβll do in each tab:
1. Overview
Give your stage a name (for example, GitHub Repo Onboarding).
You can also add a brief description to explain what this stage handles, such as generating code and registering the service.
2. Infrastructure
Next, choose where the pipeline should run.
If you want a simple setup, go with Harness Cloud Infrastructure.
If you prefer running the scripts in your own environment (like Kubernetes or a Docker runner), choose a custom delegate.
3. Execution
This is where you will define the main orchestration logic for the pipeline. You can add execution steps directly in this section.
You donβt need to configure this yet; weβll walk through each of these steps in the next section.
Advanced (optional)
You can leave this tab as-is unless you need to configure timeouts or failure handling. This is optional.
Step 3: Configure pipeline variables
Pipeline variables are essential components that make your pipeline reusable and dynamic. These variables serve as inputs to the onboarding pipeline; some will come from the developer via the workflow form, while others are fixed or derived values used internally by the pipeline.
Pipeline Variables:
These variables are used in the GitHub Repository Onboarding pipeline for our Workflow:
repo_name
String
User Input
Name of the repository to create
my-awesome-project
org_name
String
User Input
GitHub organization name
my-organization
repo_branch_name
String
User Input
Feature branch name for initial development
feature/initial-setup
token
Secret
Fixed
GitHub personal access token for API operations
account.github-token
webhook_url
String
Fixed
Webhook URL for CI/CD integration
https://app.harness.io/gateway/...
repo_desc (Optional)
String
User Input
Repository description
A new project created via IDP
Pipeline Variables Input Types:
In Harness pipelines, every variable is assigned one of the following types, based on how its value is provided or resolved during execution:
Fixed - These values are hardcoded in the pipeline definition and remain constant for every run.
Runtime Input - These values are left as
<+input>and are filled in by the developer through the IDP Workflow form when the onboarding process is triggered.Expression - These values are computed dynamically during pipeline execution using expressions. They may reference other variables, pipeline context, or system values.
These variables will be referenced in your pipeline steps using the <+pipeline.variables.variable_name> syntax, and they ensure flexibility while keeping your pipeline reusable across multiple onboarding requests. Go to Variables & Expressions to learn more.
Runtime Inputs in our Workflow:
In our GitHub Repository Onboarding workflow, most of the pipeline variables are configured as runtime inputs (<+input>) because they originate from user inputs collected through the IDP workflow form.
When a developer executes the workflow, they provide specific details like repository name, organization name, branch name, and webhook URL through the user interface. These user-provided values are then passed to the pipeline as runtime inputs, making each pipeline execution unique and tailored to the developer's requirements. Only the GitHub token remains as a fixed secret value since it is pre-configured in Harness for security purposes.
Configuring Pipeline Variables:
Follow these steps to add the required pipeline variables:
Navigate to the Variables tab in your pipeline editor.
For each variable, set the Type and Description
In the Value field, enter
<+input>for user-provided valuesFor secrets, reference the secret name (e.g.,
account.github-token)Mark variables as Required if they are essential for pipeline execution
Step 1: Create repository
Purpose: This pipeline step creates a new GitHub repository in the specified organization with the provided repository name and description.
This step will do the following:
Create a new GitHub repository in the specified organization
Set up the repository with the provided name and description
Prerequisites for this Step:
GitHub connector configured in Harness with proper authentication. Go to GitHub Connector to learn more.
GitHub organization access with repository creation permissions.
Pipeline variables configured (
repo_name,org_name,repo_desc)
Step Creation Instructions:
In the IDP stage, click + Add Step.
Select CreateRepo from the step library. A CreateRepo step is used to create a repository with the preferred Git provider. Go to Create Repo Step to learn more about its configuration.
Configure the step:
Step Name:
Create RepositoryIdentifier:
create_repoDescription:
Creates a new GitHub repository with specified configuration
Configure Repository Settings:
Git Provider: Select
Third-party Git ProviderConnector Type: Select
GithubRepository Type: Select
public(orprivatebased on your requirements)Codebase Repo Connector: Choose your configured GitHub connector (e.g.,
account.your_github_connector)Organization:
<+pipeline.variables.org_name>(since this is a user input, simply paste the variable reference here to pass the Workflow input here as a variable)Repository:
<+pipeline.variables.repo_name>(since this is a user input, simply paste the variable reference here to pass the Workflow input here as a variable)Description:
<+pipeline.variables.repo_desc>Default Branch:
main
Advanced Settings (Optional):
Initialize with README: Leave unchecked (we will create our own)
Add .gitignore: Leave unchecked (we will create a custom one)
Add License: Leave unchecked
Step 2: Create feature branch
Purpose: This step creates a feature branch from the main branch for initial development work and it runs a custom script to do the following:
Retrieves the default branch information from the repository
Gets the SHA (commit hash) of the default branch
Creates a new feature branch pointing to the same commit
Provides a dedicated branch for initial file commits
Prerequisites for this Step:
Previous steps completed successfully
GitHub PAT token with
repopermissions. Go to GitHub PAT to learn more.Pipeline variables configured (
repo_branch_name,org_name,repo_name,token)
Step Creation Instructions:
Add the Step:
Click + Add Step after the
CreateRepostepSelect Run from the step library. A Run step is used here run scripts in a Pipeline. Go to Run Step Settings to learn more about the same.
Configure Basic Settings:
Step Name:
Create Feature BranchIdentifier:
create_branchDescription:
Creates a feature branch for initial development
Configure Container Settings: Container Registry and Image ensure that the build environment has the binaries necessary to execute the commands that you want to run in this step. Go to Container Registry and Image to learn more.
Container Registry: Select your container registry connector (e.g.,
account.harnessImage)Image:
node:18(providescurlandbashutilities)Shell:
Bash
Configure the Script: This script automates the creation of a feature branch from the
mainbranch using GitHub's REST API. Here is what it does:Validates inputs: Ensures all required variables (branch name, owner, repo, token) are provided
Retrieves default branch: Gets the repository's default branch name (main/master)
Gets branch SHA: Fetches the commit hash of the default branch
Creates new branch: Uses GitHub API to create a new branch pointing to the same commit
Provides feedback: Confirms successful branch creation
Step 3: Create directory
Purpose: This step creates a project directory structure for organizing files before pushing to the repository.
Creates a directory with the repository name
Provides a workspace for file generation
Prepares the structure for the
DirectPushstep
Prerequisites for this Step:
Previous steps completed successfully
Pipeline variables configured (
repo_name)
Step Creation Instructions:
Add the Step:
Click + Add Step after the Create Feature Branch step
Select Run from the step library
Configure Basic Settings:
Step Name:
Create DirectoryIdentifier:
create_directoryDescription:
Creates project directory structure
Configure Container Settings:
Container Registry: Select your container registry connector
Image:
node:18Shell:
Sh(simple shell commands)
Configure the Script:
Step 4: Generate Project files
Purpose: This step creates standard project files to provide a consistent starting point for all repositories, it runs a custom script to do the following:
Creates a comprehensive
README.mdwith project informationGenerates a
demo.yamlconfiguration file with project metadataCreates a thorough
.gitignorefile covering common development artifactsProvides a consistent starting point for all repositories
Prerequisites for this Step:
Directory created (Step 3 completed)
Pipeline variables configured (
repo_name,repo_branch_name)
Step Creation Instructions:
Add the Step:
Click + Add Step after the Create Directory step
Select Run from the step library
Configure Basic Settings:
Step Name:
Generate Project FilesIdentifier:
generate_filesDescription:
Creates standard project files (README, .gitignore, demo.yaml)
Configure Container Settings:
Container Registry: Select your container registry connector
Image:
node:18Shell:
Bash
Configure the Script:
This script creates three essential project files that provide a standardized foundation for every repository. Here is what each part does:
Script Overview:
Sets up error handling with
set -eto exit on any command failureCreates README.md with project documentation template
Generates demo.yaml with project metadata and feature flags
Creates comprehensive .gitignore covering multiple development environments
Provides feedback on successful file creation
Complete Script
Script Breakdown:
1. Environment Setup & Variables:
Sets bash error handling to exit on failures
Gets repository name from pipeline variables
2. README.md Creation:
Creates a comprehensive README with project name
Includes getting started instructions
3. Demo Configuration File:
Creates a YAML configuration file with project metadata
Includes timestamp of creation
Documents the automated features enabled
4. Comprehensive .gitignore:
Covers common dependency directories (Node.js, PHP, etc.)
Excludes build artifacts and compiled outputs
Ignores environment and configuration files
Excludes IDE-specific files and temporary files
Handles OS-specific files (macOS, Windows)
Covers logging and runtime data
Includes coverage reports and package manager files
5. Success Confirmation:
Step 5: Push files to repository
Purpose: This step commits and pushes the generated files to the feature branch in the GitHub repository.
Commits all generated files to the feature branch
Pushes the initial project structure to GitHub
Makes the files available in the repository for development
Preserves the
mainbranch for branch protection setup
Prerequisites for this Step:
Project files generated (Step 4 completed)
GitHub connector with
pushpermissions. Go to GitHub Connector to learn more.Pipeline variables configured (
org_name,repo_name,repo_branch_name)
Step Creation Instructions:
Add the Step:
Click + Add Step after the Generate Project Files step
Select DirectPush from the step library under "Git Repository Setup". This step will push all the generated files to the new feature branch. Go to Direct Push Step to learn more.
Configure Basic Settings:
Step Name:
Push Files to RepositoryIdentifier:
push_filesDescription:
Commits and pushes generated files to the feature branch
Configure Push Settings:
Connector Type: Select
GithubConnector Reference: Choose your GitHub connector
Organization:
<+pipeline.variables.org_name>Repository:
<+pipeline.variables.repo_name>Code Directory:
<+pipeline.variables.repo_name>(the directory containing the files)Branch:
<+pipeline.variables.repo_branch_name>(push tofeature branch, notmain)
Advanced Settings:
Force Push: Leave unchecked (false)
Commit Message: You can customize or leave default
Step 6: Configure branch protection
Purpose: This step applies security rules to the main branch to enforce code review and CI/CD requirements.
Protects the
mainbranch from direct pushesRequires status checks to pass before merging
Enforces pull request reviews (1 approval required)
Requires code owner reviews when applicable
Applies rules to all users including administrators
Prerequisites for this Step:
Repository created with files pushed (Steps 1-5 completed)
GitHub PAT token with
admin:repo_hookpermissionsPipeline variables configured (org_name, repo_name, token)
Step Creation Instructions:
Add the Step:
Click + Add Step after the Push Files step.
Select Run from the step library.
Configure Basic Settings:
Step Name:
Configure Branch ProtectionIdentifier:
branch_protectionDescription:
Sets up branch protection rules on the main branch
Configure Container Settings:
Container Registry: Select your container registry connector
Image:
node:18Shell:
Bash
Configure the Script: This script configures branch protection rules on the
mainbranch using GitHub's REST API to enforce security and code quality standards. Here is what each part does:
Script Overview:
Sets up error handling with
set -eto exit on any command failureRetrieves default branch information from the repository
Applies comprehensive protection rules via GitHub API
Validates successful configuration with HTTP status code checking
Complete Script:
Script Breakdown:
1. Environment Setup & Variables:
Sets bash error handling to exit on failures
Gets organization, repository name, and GitHub token from pipeline variables
2. Default Branch Detection:
Queries GitHub API to get repository information
Extracts the default branch name (usually
mainormaster)Confirms which branch will be protected
3. Branch Protection Configuration:
Required Status Checks: Enforces CI/CD pipeline success before merging
strict: true- Requires branches to be up-to-date before mergingcontexts: ["ci/build", "ci/test"]- Specific checks that must pass
Admin Enforcement: Rules apply to repository administrators too
Pull Request Reviews: Mandatory code review process
dismiss_stale_reviews: true- Invalidates approvals when new commits are addedrequire_code_owner_reviews: true- Requires approval from code ownersrequired_approving_review_count: 1- Minimum number of approvals needed
Restrictions: Set to
null(no user/team restrictions)
4. Response Validation & Feedback:
Extracts HTTP status code from the API response
Success (200): Confirms protection rules and lists applied settings
Failure (non-200): Shows error details and exits with error code
Step 7: Setup webhook
Purpose: This step configures a webhook to trigger CI/CD pipelines when code changes occur in the repository.
Verifies the repository is accessible and ready
Creates a webhook with comprehensive event coverage
Configures secure JSON payload delivery
Enables automatic CI/CD pipeline triggering
Prerequisites for this Step:
Repository fully configured (Steps 1-6 completed)
GitHub PAT token with
admin:repo_hookpermissionsPipeline variables configured (
org_name,repo_name,token,webhook_url)Valid webhook URL endpoint
Step Creation Instructions:
Add the Step:
Click + Add Step after the Configure Branch Protection step
Select Run from the step library
Configure Basic Settings:
Step Name:
Setup WebhookIdentifier:
setup_webhookDescription:
Configures webhook for CI/CD pipeline integration
Configure Container Settings:
Container Registry: Select your container registry connector
Image:
node:18Shell:
Bash
Configure the Script: This script configures a webhook to automatically trigger CI/CD pipelines when code changes occur in the repository. Here is what each part does:
Complete Script:
Script Breakdown:
1. Environment Setup & Variables:
Sets bash error handling to exit on failures
Gets all required variables from pipeline configuration
2. Repository Verification:
Queries GitHub API to confirm repository exists
Exits with error if repository is not found
Ensures webhook creation will not fail due to missing repository
3. Webhook Creation: Webhook Configuration Details:
Events Monitored:
push,pull_request,releaseContent Type: JSON for structured payload delivery
Security: SSL verification enabled (
insecure_ssl: "0")Status: Active webhook ready to trigger immediately
4. Response Validation & Feedback:
Success (201): Extracts webhook ID and confirms configuration
Failure (non-201): Provides specific troubleshooting guidance based on HTTP status code
Error Codes:
422: Invalid webhook URL or configuration404: Repository not found or insufficient permissions403: Token lacks required webhook permissions
Pipeline YAML configuration
Below is the complete pipeline YAML configuration that you can copy and paste directly into your Harness pipeline. Make sure to replace the placeholder values with your specific configuration details.
Pipeline completion
After completing all 7 steps, your pipeline will:
β Create Repository - New GitHub repo with proper configuration
β Create Feature Branch - Dedicated branch for initial development
β Create Directory - Organized workspace structure
β Generate Project Files - Standard README, .gitignore, and demo files
β Push Files - Initial project structure committed to feature branch
β Configure Branch Protection - Security rules enforced on main branch
β Setup Webhook - CI/CD integration enabled
Build the Harness IDP workflow
Now that we have created the pipeline, we need to create the IDP Workflow that will serve as the user interface. This workflow will collect input from developers and trigger our GitHub Repository Onboarding pipeline.
Before you begin
Before creating the workflow, ensure you have:
Pipeline Created: The GitHub Repository Onboarding pipeline must be created and saved.
Pipeline URL: Note down your pipeline URL for the workflow configuration.
Harness Account Access: Permissions to create workflows in your Harness project.
Project Details: Your account ID, organization ID, and project ID.
Workflow overview
π€ Inputs Required:
Repository Information: Organization name, repository name, and description
Branch Configuration: Feature branch name for initial development
Authentication: Secure token handling for all Workflow requests (automatically generated)
β‘ Actions Performed:
Pipeline Triggering: Initiates the GitHub Repository Onboarding pipeline with user inputs
Parameter Mapping: Converts form inputs to pipeline variables seamlessly
π Outputs Provided:
Execution Tracking: Direct link to monitor pipeline progress in real-time
Status Updates: Immediate feedback on workflow submission and pipeline initiation
Create the workflow
Step 1: Navigate to IDP workflows
Go to Harness IDP in your Harness account
Navigate to Workflows in the left sidebar
Click "Create Workflow" to start creating a new workflow
Step 2: Configure basic workflow settings
Workflow Name:
GitHub Repository OnboardingIdentifier:
GitHub_Repository_OnboardingDescription:
Self-service workflow for creating GitHub repositories with automated setupOwner: Set to your username or team

Step 3: Configure the workflow YAML
Use the following YAML configuration for your workflow:
Understand the workflow configuration
1οΈβ£ Input parameters section
The parameters section in the Workflow YAML defines the form fields that developers will fill out:
Required Fields:
orgName: GitHub organization where the repository will be created.
repoName: Name of the new repository.
branchName: Initial feature branch name.
Optional Fields:
repoDesc: Description for the repository
Authentication:
token: Harness API token for pipeline execution (automatically handled)
2οΈβ£ Steps section
The Workflow contains one main step:
trigger Step:
Purpose: Triggers the GitHub Repository Onboarding pipeline
Action:
trigger:harness-custom-pipelineInput Mapping: Maps input form parameters to pipeline variables
3οΈβ£ Output section
Pipeline Details Link:
Provides a direct link to view the pipeline execution
Workflow YAML configuration
Tutorial outcome
This tutorial creates a complete self-service GitHub repository onboarding system. Here is what happens when a developer uses your workflow:
π Developer experience (in a nutshell)
Developer Action: Fills out a simple form in Harness IDP with repository details
Workflow Execution: Submits the form and receives immediate confirmation
Pipeline Automation: 7-step pipeline executes automatically in the background
Repository Ready: Fully configured repository appears in your GitHub organization
Automated outcomes in your GitHub organization
When the pipeline completes successfully, the following will be automatically created and configured in your GitHub organization:
Repository Structure:
β New GitHub repository with the specified name and description
β Feature branch created for initial development work
β Standard project files generated and committed:
README.mdwith project documentation templatedemo.yamlwith project metadata and configuration.gitignorewith comprehensive exclusion rules
Security Configuration:
β Branch protection rules applied to the main branch:
Requires pull request reviews (1 approval minimum)
Dismisses stale reviews when new commits are added
Requires code owner reviews when applicable
Enforces status checks (ci/build, ci/test) before merging
Applies rules to administrators as well
CI/CD Integration:
β Webhook configured for automated pipeline triggering:
Monitors
push,pull_requestandreleaseeventsSends JSON payloads to your specified webhook URL
End result
Developers can now create production-ready repositories in minutes instead of hours, with all security and CI/CD configurations applied automatically. The platform team maintains control over standards while enabling developer self-service at scale.
Last updated
Was this helpful?