> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/internal-developer-portal/use-idp/software-catalog/tutorials/catalog-ingestion-api-tutorial.md).
# Use Catalog Ingestion API
In this tutorial we will be using the catalog metadata ingestion APIs to add additional metadata for the software components in the catalog, display them in the component overview page using [additional info card](https://developer.harness.io/internal-developer-portal/use-idp/software-catalog/tutorials/pages/Wnh2PrpJGdrUuNZxSdo7#idp2.0--add-an-additional-info-card) and track the values ingested using [Scorecard](/internal-developer-portal/use-idp/scorecards/create-scorecards/data-sources.md#custom-data-sources).
{% embed url="" %}
### Before you begin
1. You must have components registered in your software catalog. If you do not have any components registered, follow this [tutorial](/internal-developer-portal/use-idp/software-catalog/manage-catalog.md) to register one. We recommend you to register [this software component](https://github.com/harness-community/idp-samples/blob/main/example-catalog-info/tutorial-catalog-ingestion.yaml) for this tutorial.
### Add a new metadata
#### Use cURL command
* Use the following command to add a new metadata `codeCoverageScore` to the `warehouse` component.
```curl
curl --location 'https://app.harness.io/gateway/v1/catalog/custom-properties/entity/' \
--header 'Content-Type: application/json' \
--header 'x-api-key: ' \
--data '{
"entity_ref": "warehouse",
"property": "metadata.codeCoverageScore",
"value": 33
}'
```
**OR**
#### Use the python script
* You can use the [python script](https://github.com/harness-community/idp-samples/blob/main/sample-python-script.py), to auto-update the `<+metadata.codeCoverageScore>`
**OR**
#### Use Harness pipeline
* We recommend the use of following Harness Pipeline to add a new metadata `codeCoverageScore` to the `warehouse` component. While using the following pipeline YAML make sure to add the `x-api-key` as pipeline variable of type secret and name `apikey`.
Harness Pipeline
```yaml
pipeline:
name: catalog-ingestion-api-tutorial
identifier: catalogingestionapitutorial
projectIdentifier:
orgIdentifier:
tags: {}
stages:
- stage:
name: demo-stage
identifier: demostage
description: ""
type: IDP
spec:
platform:
os: Linux
arch: Amd64
runtime:
type: Cloud
spec: {}
execution:
steps:
- step:
type: Run
name: Run_1
identifier: Run_1
spec:
shell: Python
command: |-
import requests
import random
# Define the API endpoint and headers
url = 'https://app.harness.io/gateway/v1/catalog/custom-properties/entity'
headers = {
'Content-Type': 'application/json',
'x-api-key': '<+pipeline.variables.apikey>' # Replace with your actual API key
}
# Fetch the product data from the API
product_url = 'https://dummyjson.com/products'
response = requests.get(product_url)
# Check if the response is successful
if response.status_code == 200:
data = response.json()
# Extract stock data
products = data.get("products", [])
total_stock = sum(product.get("stock", 0) for product in products)
# Introduce variability
random_max_possible_stock = random.randint(500, 2000) # Randomize max stock
random_factor = random.uniform(0.5, 1.5) # Random multiplier for variation
# Calculate base score and apply randomness
base_score = (total_stock / random_max_possible_stock) * 100
code_coverage_score = min(base_score * random_factor, 100) # Ensure < 100
print(f"Code Coverage Score: {code_coverage_score:.2f}")
# Prepare the data to update the code coverage score
data_payload = {
"entity_ref": "warehouse",
"property": "metadata.codeCoverageScore",
"value": round(code_coverage_score, 2) # Send as a number, not a string
}
# Make the POST request to update the value
update_response = requests.post(url, headers=headers, json=data_payload)
# Check the response from the update request
if update_response.status_code == 200:
print("Code coverage score updated successfully!")
print("Response:", update_response.json())
else:
print(f"Failed to update the code coverage score. HTTP Status Code: {update_response.status_code}")
print("Response:", update_response.text)
else:
print(f"Failed to fetch product data. HTTP Status Code: {response.status_code}")
tags: {}
variables:
- name: apikey
type: Secret
description: ""
required: false
value: x-api-key
```
### Schedule a cron to update the catalog metadata
* If you use the cURL command or the python script to auto-update the metadata on regular intervals you need to configure a cron job.
* In case you are using Harness Pipeline use a trigger to update the data ingested on regular intervals. Create a pipeline trigger of type **Cron** to schedule an hourly trigger.
### Code coverage scorecard {#code-coverage-scorecard}
In case you want to track the information you have ingested using Scorecards, follow the steps below.
1. Go to the **Scorecards** and under **Configure** and select **Checks** from the top nav.
2. Now select **Create Custom Check** and add a New Custom Check to check the code coverage score, select the data source as **Catalog Info YAML** and data point as **Evaluate Expression(JEXL)**.
3. Add the **jexl** to be evaluated as `<+metadata.codeCoverageScore>` and **operator** `Greater than or equal to` from the dropdown and add the **value** as `50`
4. Now you can create a scorecard using the Custom Check you have created above.
2. Now go to the **Warehouse** Software Component in the **Catalog**, and you will find the Code Coverage Scorecard under the Scorecards tab and on the overview page. Go to [Scorecards custom data sources](/internal-developer-portal/use-idp/scorecards/create-scorecards/data-sources.md#custom-data-sources) to read more.
### Additional info card {#additional-info-card}
In case you want to display the same information you have ingested on your Overview page as an additional card, follow the steps below.
1. Go to the **Layout Page** and under **Configure** and select **Catalog Entities**.
2. Now Select **Edit Layout** for component **Service** and add the following under Overview card.
```yaml
- component: EntityAdditionalInfoCard
specs:
props:
title: Additional Info Card
items:
- label: Code Coverage Score
value: <+metadata.additionalInfo.codeCoverageScore>
type: string
style:
bold: true
gridProps:
md: 6
```
2. Now go to the **Warehouse** Software Component in the **Catalog**, and you will find an additional info card populated with information we ingested using the API above. You can read more about [additional info card](https://developer.harness.io/internal-developer-portal/use-idp/software-catalog/tutorials/pages/Wnh2PrpJGdrUuNZxSdo7#idp2.0--add-an-additional-info-card)
> **Note:** For detailed instructions on adding an Additional Info Card to display custom information on your entity's Overview page, see the [Additional Info Card](/internal-developer-portal/admin-and-customization/layout-and-appearance/catalog.md#additional-info-card) section in the Catalog Layout documentation.
### Conclusion
Using the Catalog Metadata Ingestion API, you can source information into your catalog from internal systems such as cost trackers, service health checkers, security scans, or even from simple spreadsheets tracking personnel details and use them to just display the information to users as wells as use them as an input for workflows.
Check out [other examples of the Catalog Ingestion API](/internal-developer-portal/use-idp/software-catalog/integrate-tools/catalog-ingestion-api.md).