Skip to main content

Manage Azure costs by using CCM on Harness Self-Managed Enterprise Edition

This topic walks you through the steps required to set up CCM for Azure in a self-managed platform.

Figure: Azure CCM Self-Managed Enterprise Edition architecture diagram

You need to perform the following tasks to set up CCM for Azure. For Step 1, 2 and 3 Sign in to your Azure Portal:

  1. Setup a new Application via App Registration.
  2. Create a new Client secret.
  3. Setup a new Storage Account and a new Storage Container.
  4. Deploy workloads via Helm charts.

Setup a new Application via App Registration

  1. Give it Name like Harness CCM App.
  2. For Supported account types select: Accounts in any organizational directory (Any Microsoft Entra ID tenant - Multitenant).
  3. Click on Register. For more information, go to Quickstart: Register an App.

Create a new Client secret

  1. Once you click register, the App Overview Page would be open.
  2. From the overview page, Copy and Save following from it: Copy & Save Directory (tenant) IDtenantId Copy & Save Application (client) IDclientId.
  1. Now click on Certificates & secrets in the left panel.
  2. Go to Client secrets (0) tab.
  3. Click on New client secret.
  4. Enter Description Harness CCM Client Secret.
  5. Select Expires from the drop down select 730 days (24 months)(or the maximum allowed time).
  6. Click Add Button.
  1. From the new Client secret, Copy & Save ValueclientSecret. For more information, go to Add credentials.

Make a note of the following:

  • tenantId

  • clientId

  • clientSecret

Setup a new Storage Account and a new Storage Container

  1. Select a Subscription from the drop down.
  2. Select a Resource Group from the drop down.
  3. Enter Storage account name ccmbillingdatasmp. Save Storage account namestorageName
  4. Move to Advanced Tab.
  5. In Blob storage section, enable Allow cross-tenant replication
  6. Click on Review.
  1. Click on Create. For more information, go to Create a storage account.
  2. Once the storage account is created, Go to Containers in left panel.
  3. Click on + Container.
  4. Enter Name as billingdatacontainer. Save Name as → containerName
  5. Click on Create. For more information, go to Create a container.
  1. Go to Shared access signature in left panel.
  2. Check all Allowed resource types which are Service, Container and Object.
  3. Add 10 years to End in Start and expiry date/time.
  4. Click on Generate SAS and connection string.
  5. Save SAS tokensasToken starting from sv=, ignore ? in beginning. For more information, go to Create your SAS tokens.

Make a note of the following:

  • storageName

  • containerName

  • sasToken

Deploy workloads via Helm charts

  1. Clone the chart repository.
git clone
cd main/src/harness
  1. Upgrade charts if you're already using Harness Self-managed Enterprise Edition services. Perform the following steps to update the override files:
    1. Retrieve the current override values provided during the installation or upgrade of the Helm charts.

      helm get values <chart-name> -n <namespace> > override.yaml
    2. After obtaining the override file, you can make necessary modifications as mentioned below.

Override file changes for a connected environment
enabled: true
enabled: true
ng: <SMP NG License with CCM>
enabled: true


Air-gapped environment for Azure in SMP is not supported as of now

  1. After making the necessary updates to the override file, you can proceed with the Helm chart upgrade.
helm upgrade <chart-name> <chart-directory> -n <namespace> -f override.yaml 

Handling Kubernetes secrets

When installing or upgrading the Helm charts, Kubernetes secrets with default values are created within the cluster. These generated secrets should be updated with the values mentioned above. Before updating the secrets, you need to convert the secret into base64 encoded format. For example, if your HARNESS_CE_AZURE_CLIENTID value is "clientId", it would be stored as Y2xpZW50SWQ== after encoding.

The following are the secrets specific to CCM services:

  • batch-processing
kubectl edit secret batch-processing -n <namespace>
  • cloud-info-secret-mount [config-file]
kubectl edit secret cloud-info-secret-mount -n <namespace>
config-file: <Config file> [In "Config file" provided below: Replace <clientId>, <tenantId> and <clientSecret>]
  • nextgen-ce
kubectl edit secret nextgen-ce -n <namespace>
Config file
environment = "production"
debug = false
shutdownTimeout = "5s"

enabled = false
address = ""
token = ""
secretPath = ""

format = "json"
level = "info"

enabled = false
address = ":9090"

enabled = false

# Configure either collectorEndpoint or agentEndpoint.
# When both are configured collectorEndpoint will take precedence and the exporter will report directly to the collector.
collectorEndpoint = "http://localhost:14268/api/traces?format=jaeger.thrift"
agentEndpoint = "localhost:6831"
# username = ""
# password = ""

address = ":8000"
basePath = "/"

enabled = true
interval = "24h"

enabled = false

# See available regions in the documentation:
# region = "us-east-1"

# Static credentials
# accessKey = ""
# secretKey = ""

# Shared credentials
# sharedCredentialsFile = ""
# profile = ""

# IAM Role ARN to assume
# assumeRoleARN = ""

# http address of a Prometheus instance that has AWS spot price metrics via banzaicloud/spot-price-exporter.
# If empty, the cloudinfo app will use current spot prices queried directly from the AWS API.
prometheusAddress = ""

# advanced configuration: change the query used to query spot price info from Prometheus.
prometheusQuery = "avg_over_time(aws_spot_current_price{region=\"%s\", product_description=\"Linux/UNIX\"}[1w])"

# Amazon pricing API credentials (optional)
# Falls back to the primary credentials.

# See available regions in the documentation:
# region = "us-east-1"

# Static credentials
# accessKey = ""
# secretKey = ""

# Shared credentials
# sharedCredentialsFile = ""
# profile = ""

# IAM Role ARN to assume
# assumeRoleARN = ""

enabled = false

# base64 encoded credentials in json format (base64 encoded content of the credential file)
# credentials = ""

# credentialsFile = ""

# project = ""

enabled = false

# region = ""
# accessKey = ""
# secretKey = ""

enabled = false

# tenancy = ""
# user = ""
# region = ""
# fingerprint = ""
# privateKey = ""
# privateKeyPassphrase = ""

# configFilePath = ""
# profile = ""

enabled = true

# subscriptionId = ""

# Client credentials
clientId = "<clientId>"
clientSecret = "<clientSecret>"
tenantId = "<tenantId>"

enabled = false

enabled = false

# accessToken = ""

enabled = true
address = ":8001"

serviceConfigLocation = "./configs"
serviceConfigName = "services"
format = "yaml"

enabled = false
host = "localhost"
port = 6379

enabled = false
hosts = "localhost"
port = 9042
keyspace = "cloudinfo"
table = "products"

expiration = 0
cleanupInterval = 0

The following are some secrets from platform-service that you need to update:

  • smtp-secret - Required to support budget alerts email.
kubectl edit secret smtp-secret -n <namespace> 

To increase TimescaleDB to 100Gi, run: kubectl edit pvc wal-volume-harness-timescaledb-0 -n <namespace>. Features like Recommendations and Anomalies within CCM services use it.

Next steps