Skip to main content

Infrastructure requirements for KOTS

Installation of Harness Self-Managed Enterprise Edition in an existing Kubernetes cluster requires the following infrastructure.

Production environment

Self-Managed Enterprise Edition NextGen is installed as an application on an existing Self-Managed Enterprise Edition FirstGen installation.

The following tables list the resource requirements for the installation of Self-Managed Enterprise Edition in the production environment.

Self-Managed Enterprise Edition FirstGen

MicroservicePodsCPU / PodMemory / PodTotal CPUTotal Memory
Manager22448
Verification21326
Machine Learning Engine18282
UI20.250.250.50.5
MongoDB3481224
Proxy10.50.50.50.5
Ingress20.250.250.50.5
TimescaleDB328624
KOTS Admin Pods   48
Total37.573.5

The compute resources listed for the KOTS admin pods support a full stack. In an existing cluster, the requirements for KOTS are usually lower.

Self-Managed Enterprise Edition NextGen

MicroservicePodsCPU / PodMemory / PodTotal CPUTotal Memory
Log Minio114Gi14Gi
Log service113Gi13Gi
SCM10.10.5Gi0.10.5Gi
Gateway20.53Gi16Gi
NextGen UI20.20.2Gi0.40.4Gi
Platform service213Gi26Gi
Test Intelligence213Gi26Gi
Access Control213Gi26Gi
CI Manager213Gi26Gi
NextGen Manager226Gi412Gi
Pipeline216Gi212Gi
Total1917.561.9Gi

Development environment

The following table lists the requirements for the installation of Self-Managed Enterprise Edition in the development environment.

MicroservicePodsCPU / PodMemory / PodTotal CPUTotal Memory
Manager12424
Verification11313
Machine Learning Engine13232
UI10.250.250.250.25
MongoDB324612
Proxy10.50.50.50.5
Ingress10.250.250.250.25
TimescaleDB12828
Kots Admin Pods   48
Total1938

Harness recommends the following minimum requirements for nodes.

  • 8 cores vCPU
  • 12 GB memory

Storage requirements

Your Kubernetes cluster must attach a Kubernetes StorageClass resource. You provide the name of the StorageClass during the installation process.

A typical installation of Self-Managed Enterprise Edition uses a total of 1000 GB of storage in the following distribution:

ComponentPodsStorage per podTotal
MongoDB3200 GB600 GB
Timescale DB3120 GB360 GB
Redisn/an/a40 GB

A Proof of Concept (PoC) installation of Self-Managed Enterprise Edition requires 200 GB of storage in the following distribution:

ComponentPodsStorage per podTotal
MongoDB350 GB150 GB
Timescale DB120 GB20 GB
Redisn/an/a30 GB

Allow list and outbound access requirements

Add the following URLs to your allow list:

URLUsage
kots.ioKOTS pulls the latest versions of the kubectl plugin and KOTS admin console (kotsadm).
app.replicated.comKOTS admin console connects to check for the releases that your license allows.
proxy.replicated.comAllows you to proxy your registry to pull your private images.

Provide outbound access to the following URLs:

  • proxy.replicated.com​
  • replicated.app
  • k8s.kurl.sh​
  • app.replicated.com

Outbound access is required for connected install only. Outbound access is not required to install in Airgap mode. If your cluster does not have direct outbound connectivity and requires a proxy for outbound connections, see the following for information on how to create a proxy on the node machines: https://docs.docker.com/network/proxy.

Cluster and network architecture

The following diagram describes the cluster and network architecture for a Self-Managed Enterprise Edition Kubernetes Cluster installation.

Namespace requirements

The examples in this documentation use the harness namespace.

If your installation will operate in a different namespace, you must update the Harness spec samples you use to apply the namespace you specified.

Load balancer

The installation of Harness Self-Managed Enterprise Edition requires a load balancer. You enter the URL of the load balancer into the KOTS admin console when Self-Managed Enterprise Edition is installed.

After Harness Self-Managed Enterprise Edition is installed, the load balancer is used to access the Harness Manager UI with a web browser.

For information on how to create the load balancer, see Self-Managed Enterprise Edition - Kubernetes Cluster: Setup Guide.

gRPC and load balancer settings

The configuration of gRPC depends on load balancer support for HTTP2.

Load balancer support for HTTP2 over port 443

If your load balancer supports HTTP2 over port 443, you configure gRPC when you install Self-Managed Enterprise Edition NextGen. gRPC is configured in the GRPC Target and GRPC Authority fields.

The following table describes the GRPC Target and GRPC Authority fields.

ValueDescription
GRPC TargetThe hostname of the load balancer. This is the URL of the load balancer.
GRPC AuthorityAppend the hostname to the following string: manager-grpc-<hostname>. For example, manager-grpc-35.202.197.230.

No load balancer support for HTTP2 over port 443

If your load balancer does not support HTTP2 over port 443, use one of the following configuration options:

  • Load balancer supports multiple SSL ports. Add port 9879 in the application load balancer and target port 9879 or node port 32510 on the Ingress controller.

    ValueDescription
    GRPC TargetThe hostname of the load balancer.
    GRPC AuthorityThe hostname of the load balancer.
  • Load balancer does not support multiple SSL ports. Create a new load balancer and target port 9879 or node port 32510 on the Ingress controller:

    ValueDescription
    GRPC TargetThe hostname of the new load balancer.
    GRPC AuthorityThe hostname of the new load balancer.

Trusted certificate requirement for Harness Self-Managed Enterprise Edition

You can use secure or unencrypted connections to Harness Manager. This option depends on the URL scheme you apply during installation, when you configure the Load Balancer URL field. You can use https:// or http://.

For secure connections from your integrations to Harness Manager, you must use a public trusted certificate. This includes your integration with Harness Delegate as well as to Github Webhooks and so on. Harness does not support self-signed certificates for connections to Harness Manager.

For connections from Harness Manager outbound to an integration, you can use a self-signed certificate. In this case, you must import the self-signed certificate into Harness Delegate's JRE keystore manually or by using a Harness Delegate Profile.

Terminate at Harness

You have the option to terminate at the Harness ingress instead of the load balancer. If you configured the Harness ingress controller, you can add a TLS secret to the harness namespace.

The following instruction adds a TLS secret based on a public certificate with the name harness-cert:

kubectl create secret tls harness-cert --cert=path/to/cert/file --key=path/to/key/file