Skip to main content

Optimize Cloud Costs for Kubernetes

Background on Cloud Costs

As engineers, we are natural optimizers. Responding to users in more expeditious and efficient ways are core to optimizations. There are always tradeoffs between architecture and costs. With the rise of public cloud consumption with on the surface “nearly infinite” resources, this unlocks the next generation of distributed architecture. Though this is not without cost. From an engineering perspective, cost/billing data from the public cloud vendors depending on your organization might not be available to you or can take multiple billing cycles to be disseminated.

Resources on the public cloud are certainly not free. Public cloud vendors not only bill for the core infrastructure but also depending on the services leveraged could be billing dimensions for the control/management planes and non-obvious dimensions such as network/data transmissions. When capacity planning for infrastructure, a natural inclination is to err on the side of caution and provision extra capacity in case of a spike.

Optimizing costs focuses on right sizing resource usage to match workload actuals. Coupled with finding more emphermial infrastructure e.g spot instances can reduce costs. Monitoring solutions typically monitor for when usage exceeds a certain threshold, but the inverse if usage is under utilized, traditional monitoring solutions might not alert on that. Harness Cloud Cost Management, or CCM, can help unlock insights based on usage in how to optimize costs. This example will connect Kubernetes workload(s) on a public cloud vendor to Harness CCM to start getting cost insights and recommendations.

Get Started with Optimizing Your Kubernetes Cloud Costs

By connecting your public cloud Kubernetes cluster to Harness, you can start to get recommendations on right-sizing the workloads that are running. This example is assuming there are some workloads running in the Kubernetes cluster.

Overview

To get actual costing data from your public cloud vendor, it is recommended to connect your public cloud billing API [usage report, billing export, etc] to Harness CCM. This is not a requirement. Harness CCM will poll usage data from the Kubernetes Metric Server which comes installed by default with GKE and AKS. If using EKS, installing the Kubernetes Metric Server is needed.

Prepare Your Kubernetes Cluster for Optimization Recommendations

If you have not already, make sure to sign up for a Harness CCM Account. Once signed up, to start receiving optimizations for your cluster, you will need to install a Harness Delegate, e.g the Harness worker node, into your cluster.

Install Delegate

You will also need to wire in a Kubernetes Delegate if you have not done so already.

Install Delegate

Install Delegate

What is a Delegate?

Harness Delegate is a lightweight worker process that is installed on your infrastructure and communicates only via outbound HTTP/HTTPS to the Harness Platform. This enables the Harness Platform to leverage the delegate for executing the CI/CD and other tasks on your behalf, without any of your secrets leaving your network.

You can install the Harness Delegate on either Docker or Kubernetes.

Install Delegate

Create New Delegate Token

Login to the Harness Platform and go to Account Settings -> Account Resources -> Delegates. Click on the Tokens tab. Click +New Token and give your token a name `firstdeltoken`. When you click Apply, a new token is generated for you. Click on the copy button to copy and store the token in a temporary file for now. You will provide this token as an input parameter in the next delegation installation step. The delegate will use this token to authenticate with the Harness Platform.

Get Your Harness Account ID

Along with the delegate token, you will also need to provde your Harness accountId as an input parameter to the delegate installation. This accountId is present in every Harness URL. For example, in the following URL

https://app.harness.io/ng/#/account/6_vVHzo9Qeu9fXvj-AcQCb/settings/overview

6_vVHzo9Qeu9fXvj-AcQCb is the accountId.

Now you are ready to install the delegate on either Docker or Kubernetes.

Prerequisite

Ensure that you have the Docker runtime installed on your host. If not, use one of the following options to install Docker:

Install on Docker

Now you can install the delegate using the following command.

docker run -d --name="firstdockerdel" --cpus="0.5" --memory="2g" \
-e DELEGATE_NAME=firstdockerdel \
-e NEXT_GEN=true \
-e DELEGATE_TYPE=DOCKER \
-e ACCOUNT_ID=PUT_YOUR_HARNESS_ACCOUNTID_HERE \
-e DELEGATE_TOKEN=PUT_YOUR_DELEGATE_TOKEN_HERE \
-e MANAGER_HOST_AND_PORT=PUT_YOUR_MANAGER_HOST_AND_PORT_HERE \
harness/delegate:22.11.77436

PUT_YOUR_MANAGER_HOST_AND_PORT_HERE should be replaced by the Harness Manager Endpoint noted below. For Harness SaaS accounts, you can find your Harness Cluster Location in the Account Overview page under Account Settings section of the left navigation. For Harness CDCE, the endpoint varies based on the Docker vs. Helm installation options.

Harness Cluster LocationHarness Manager Endpoint on Harness Cluster
SaaS prod-1https://app.harness.io
SaaS prod-2https://app.harness.io/gratis
SaaS prod-3https://app3.harness.io
CDCE Dockerhttp://<HARNESS_HOST> if Docker Delegate is remote to CDCE or http://host.docker.internal if Docker Delegate is on same host as CDCE
CDCE Helmhttp://<HARNESS_HOST>:7143 where HARNESS_HOST is the public IP of the Kubernetes node where CDCE Helm is running

Verify Docker Delegate Connectivity

Click Continue and in a few moments after the health checks pass, your Docker Delegate will be available for you to leverage. Click Done and can verify your new Delegate is on the list.

Delegate Available

You can now route communication to external systems in Harness connectors and pipelines by simply selecting this delegate via a delegate selector.

Pointing Harness CCM to Your Cluster

With the Harness Delegate running, the next step is to create a Cloud Integration which will represent your Kubernetes cluster.

Harness Platform -> Cloud Costs -> Setup -> Cloud Integration + New Cluster/Cloud Account

Select Kubernetes.

New Cloud Integration

Once Kubernetes is selected, now we can configure the Kubernetes Cluster connector.

Name: my_kubernetes_cluster

Click Continue then in the Details section, leverage the Harness Delegate to connect to the Kubernetes cluster.

Use Harness Delegate

Click Continue, then select the Harness Delegate that is running inside your cluster.

Use Specific Harness Delegate

Enabling Cloud Costs

Click Save and Continue and a connection test will occur and validate connectivity. With the Kubernetes cluster wired in, you can enable the wirings for the reporting.

Under the “Features Enabled” column, click Enable Cloud Costs.

Enable CCM

In the click-through, validation will occur and then click Finish. If additional permissions are needed, the wizard will guide you through enabling them.

Enabling CCM

Once you click Finish, Harness will start to analyze data around workload usage.

View Costs

Note, that initial recommendation and costing data can take up to 24 hours to start being reported/calculated.

After a few hours, cost information will start to be imported. Since in this example we did not connect a public cloud billing bucket, list pricing will be used in cost calculation.

Harness -> Cloud Costs -> Setup -> Cloud Integration -> my_kubernetes_cluster -> View Costs

Initial Costs

After about a day, more usage data will be captured and recommendations will start to appear.

First Costs

Digging into the savings recommendation, Harness CCM will provide recommendations off of the usage data captured.

Looking at Your First Cost Optimizations

Digging into the recommendations list, looking at a recommendation that corresponds to the Kubernetes cluster, can see that the cluster is sized much larger than the resources that are being consumed.

Harness -> Cloud Costs -> Recommendations -> Your Resource

First Rec

Per this recommendation, resizing the worker node machine size is a prudent move. Taking a look at the back work for the recommendation, can take a look at the actual usage back in the previous perspective. Actions to be taken to reduce costs would be to right size the nodes and if not already, can place resource requests and limits onto the Kubernetes workloads based on what Harness CCM is reporting. This is just the start of the capabilities with Harness CCM, which can also provide additional rules and capabilities around auto-stopping of workloads and give additional infrastructure and workload recommendations. If additional changes are needed, can leverage the Harness Platform to experiment and make those changes.