Skip to main content

Set up CCM for GCP

Harness Cloud Cost Management (CCM) monitors the cloud costs of your GCP products, projects, SKUs, and location. As a first step, you need to connect Harness to your GCP account to get insights into your cloud infrastructure, and GCP services, Compute Engine, Cloud Storage, BigQuery, etc. CCM offers a wide range of features to track and control costs associated with your cloud resources.

info

After enabling CCM, it takes about 24 hours for the data to be available for viewing and analysis.

Before you begin​

  • The same connector cannot be used in NextGen and FirstGen. For information on creating a GCP connector in the FirstGen see Set Up Cost Visibility for GCP.
  • Review Required permissions and roles to create an IAM role at the organization level
  • Ensure that you have the following permissions to enable and configure the export of Google Cloud billing data to a BigQuery dataset:
    • Billing Account Administrator role for the target Cloud Billing account
    • BigQuery User role for the Cloud project that contains the BigQuery dataset that will be used to store the Cloud Billing data

Connect Harness to Google Cloud Platform (GCP) Account​

Connect Harness to your GCP account to gain access your GCP services, Compute Engine, Cloud Storage, BigQuery, etc. Harness CCM gives you cost insights that are derived from the billing export. For deep Kubernetes visibility and rightsizing recommendations based on the historical utilization and usage metrics, set up Kubernetes connectors. See Set Up Cloud Cost Management for Kubernetes.

info

Time periods in the GCP Cloud Billing report use the Pacific Time Zone (PST) and observe daylight saving time shifts. However, Harness CCM explorer uses the UTC time zone. You may notice some cloud cost differences between Harness CCM explorer and the GCP Cloud Billing report due to the time zone difference.

  1. Create a new Kubernetes connector using one of the two options below:
  1. Go to Account Resources > Connectors.
  2. Select + New Connector.
  3. Under Cloud Costs, select GCP.
  1. Perform the following tasks in the GCP Connector wizard.

Overview​

  1. In Overview, in Connector Name, enter a name that describes this account.
  2. In Specify Project ID, enter the project ID and select Continue. For more information on how to get a project ID, go to Create a BigQuery dataset.

GCP Billing Export​

Cloud Billing export to BigQuery enables you to export detailed Google Cloud billing data (such as usage and cost estimate data) automatically throughout the day to a BigQuery dataset that you specify.

info

Make sure that you have enabled Detailed Usage Cost on the Billing Export page on your GCP console. For more information, see Enable Cloud Billing export to the BigQuery dataset.

  1. In GCP Billing Export, select Launch GCP console.

  2. In the GCP Explorer window, in the pinned projects section, select your project ID to open the project. If you see an overflow menu (:) next to your project ID, select the menu and select Open.

  3. Select Create dataset. For more information, go to Create a BigQuery dataset.

    Creating a dataset in the GCP console.

  4. Enter a Dataset Name. You need to enter Dataset Name in Harness.

  5. Select a Data location.

  6. Set the Default table expiration to Never.

  7. Set the Encryption option to Google-managed key.

  8. To save, select CREATE DATASET.

  9. Enter the Dataset Name in Harness.

    Entering the dataset name.

  10. Next, you need to enter the table name in Harness. From the GCP console, copy the table name where the billing export is available. In your BigQuery dataset, the table is named gcp_billing_export_v1_*.

  11. Once the billing account is selected, open the navigation menu>Billing export>BigQuery export. In BigQuery Export section, user needs to setup "Detailed usage cost". When setting up your dataset, it is recommended to choose the United States (US) as the dataset location.

  12. From the Project list, choose the project that contains your BigQuery dataset. You’ll be requested to establish a BigQuery dataset if you don’t already have one.

  13. To create a dataset, if you don’t have one

    • In the Google Cloud Console, go to the Google BigQuery Page.
    • In the Explorer Panel, choose the Project for which you wish to build a dataset.
    • Select +Create Dataset from the drop-down menu.
    • Certain fields, such as Dataset ID, data location, and data expiry, will be required. You will be able to perform billing exports after you have your dataset ready.
  1. Enter the Table Name in Harness.
  2. Select Continue.

Choose Requirements​

Select the Cloud Cost Management features that you would like to use on your GCP account.

CCM offers the following features:

FeaturesCapabilities
Cost Visibility (Required)This feature is available by default. Make sure you have set up the GCP Billing Export. Provides the following capabilities:
  • Insights into GCP costs by projects, products, etc.
  • Root cost analysis using cost perspectives
  • Cost anomaly detection
  • Governance using budgets and forecasts
  • Alert users using Email and Slack notification
GCP Inventory Management (Optional)This feature provides visibility into your GCE VMs and unused disks and snapshots. The insights provided by inventory management can be used by Finance teams to understand resource utilization across the board.
GCP optimization using AutoStopping rules (Required for AutoStopping Rules)This feature allows you to enable Intelligent Cloud AutoStopping for your GCP cloud resources. For more information, see Create AutoStopping Rules for GCP.
  • Orchestrate GCE VMs based on idleness
  • Set dependencies between VMs
  • Granular savings visibility
  • Simple one-time setup
Cloud Governance (Optional)This feature allows you to optimize your cloud spend and avoid unnecessary costs by rightsizing resources and decommissioning unused instances. For more information, see Asset governance.
  • Asset Management (Instance, Disk, SQL-instance, Image)
  • Automated Actions

Make your selection and select Continue.

Grant Permissions​

Cloud Billing Export to BigQuery helps you export detailed Google Cloud billing data (such as usage and cost estimate data) to a BigQuery dataset that you specify. The export happens throughout the day automatically. 

  1. In Grant permissions, select Open BigQuery Page.

  2. Log into the GCP console and go to the BigQuery page.

  3. Select your project in the left panel.

  4. Select your dataset. For more information on creating a dataset, see Creating datasets.

  5. Select the more actions icon (three vertical dots) against the dataset, and then select Share.

  6. In Dataset permissions, in Add Principals, enter the Harness service account as a member.
    Copy the service account detail from Harness. The service account is generated dynamically for your account.

  7. In Select a role, select BigQuery Data Viewer, and then select Add.

  8. Select Done.
    When you are done, the following screen is displayed:

    Entering the dataset name.
    info

    To enable AutoStopping rules, you need to add more permissions.

    For more information, go to Create a GCP Connector for AutoStopping Rules.

  9. Select Continue in Harness.

Cloud asset governance rules​

info

Now, Harness CCM supports CAG for GCP.

Regions for Asset Governance Rules for GCP are not available in drop-down menu but can be configured manually in the YAML.

Connection Test​

The connection is validated and verified in this step. After successfully testing the connection, select Finish.

Your connector is now listed in the Connectors.

Next Steps​