Skip to main content

Service Onboarding using Scripts on Catalog

Scripts to create new services, register new services​

  • The GitHub Catalog Discovery plugin registers one location per repository. This might not be a good idea when there are many (3000+ in this case) as any error in fetching one catalog-yaml would mark the whole location as failed and create trouble with the entity sync.

  • To solve this we would recommend you to use the following scripts which would register separate locations for all the matching catalog-info.yaml files and hence would be synchronised separately.

Pre-Requisites to Use the Script​

  • Harness API Key - Docs
  • GitHub Token - Docs
  • A Repository to store all the IDP Config YAML
  • Python 3 installed on your machine where you're trying to execute the code along with requests library.

Download the Script​

  • A pregenerated/created repo (let's call it chosen_repo) is to be cloned. After opening it on your code editor and from chosen_repo, below commands can be run to first download and then generate and register catalog-info.yaml files of all the repos in your org. The catalog-info.yaml files will be in - chosen_repo/services/{repos}/catalog-info.yaml where repos will be all the repo in your org.

  • Command to Download the Script

  • Script Source

curl -o idp-catalog-wizard-github.py https://raw.githubusercontent.com/harness-community/idp-samples/main/catalog-scripts/idp-catalog-wizard-github.py
python3 idp-catalog-wizard-github.py --create-yamls --org YOUR_GITHUB_ORG --token YOUR_GITHUB_TOKEN --repo-pattern "regex_pattern"

  • Case 1: Run command using --create-yamls args, then you'll have to manually push the files - "services/" ..... after which you can run command using --register-yamls args to register all the YAML.

  • Case 2: Run command using --run-all args, all actions will be performed - create, push and register in one go.

Registered Locations​

Register YAML files using X-API-Key and account name (all given args in command below are required)​

python3 idp-catalog-wizard-github.py --register-yamls --org org_name --x_api_key your_x_api_key --account your_account

Perform all actions: create YAML files, push changes, and register YAML files (all given args in command below are required)​

python3 idp-catalog-wizard-github.py --run-all --org example-org --token your_token --x_api_key your_x_api_key --account your_account

Registered Locations - For Monorepos​

  • Discover catalog-info.yaml matching the regex filter and register under the catalog provided in apiurl. This would separate locations for all the matching catalog-info.yaml files and hence would be synchronized separately.

  • To use the script you need to add the appropriate flags and run it.

Create Services​

  • Generates a monorepo with the following file structure, assigning random English names.
repo
   - antronasal-service
      - catalog-info.yaml
   - cespititous-service
      - catalog-info.yaml
   - ....
   - geomaly-service
        - catalog-info.yaml

Delete Services​

  • Will clean up the services already created.

Troubleshooting​

  • Ensure you are using the correct credentials with the appropriate scope when running the scripts.
  • If you are trying out the scripts with a personal account instead of an organization account, update the GitHub API endpoints as specified below.

Using Personal GitHub Account​

If you're testing the above scripts on your personal GitHub account, make sure to update the repository listing endpoint to use users instead of org.

Here's the difference between the endpoints:

# For organizations
/orgs/{org}/repos

# For personal accounts
/users/{username}/repos