> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/harness-platform/3.0/harness-platform-resources/references/network-and-security/cloud-private-connectivity/kubernetes.md).

# Deploy appliances on Kubernetes

Deploy and scale subnet routers, App Connectors, or combined appliances with one official-image YAML per group

{% hint style="info" %}
**Note**

Cloud Private Connectivity requires enablement by Harness.
{% endhint %}

Kubernetes is the recommended customer-appliance deployment. Deploy a subnet router, App Connector, or combined appliance with one YAML file per group. The template uses the official Tailscale image and automatically stores each replica's identity in a different Kubernetes Secret. You do not need a Helm chart, operator, PVC, or customer-side Harness delegate for these appliances.

***

## Before you begin

Before you deploy private connectivity appliances on Kubernetes, ensure you have the following

* **Harness provisioning**: Complete the [account verification and configuration](https://developer.harness.io/harness-platform/3.0/harness-platform-resources/references/network-and-security/cloud-private-connectivity/configure.md#verify-and-configure-the-account) and the response should be `READY`. Obtain a valid customer appliance key from the credential API. Adding replicas never creates another account network.
* **Kubernetes access**: Use Linux workers that allow privileged networking containers. Ask your cluster administrator for permission to create the template's namespace-scoped resources and enable Secret encryption at rest. Go to [Kubernetes Secret security](https://kubernetes.io/docs/concepts/security/secrets-good-practices/) for details on storage and access controls.
* **Dedicated namespace**: Use an appliance-only namespace. The template's Role can get/create/update/patch all Secrets there, including other appliance groups' auth and state. Use separate namespaces for groups with different trust boundaries.
* **Network access**: Each pod must reach its private applications, DNS resolver, Kubernetes API, and approved Tailscale coordination/relay endpoints. Private applications need no public inbound address, but a fully disconnected network is not supported. Go to [network requirements](https://developer.harness.io/harness-platform/3.0/harness-platform-resources/references/network-and-security/cloud-private-connectivity.md#network-connectivity-requirements) for details.
* **Local tools**: Use Bash, `kubectl`, and `base64`. Keep the raw appliance key in a mode-`0600` file without a trailing newline. Never copy machine-state Secrets into another pod or use a Harness-managed cloud delegate credential or delegate token as the appliance key.

The template pins the official Tailscale image and follows [Tailscale's Kubernetes guidance](https://tailscale.com/docs/solutions/connect-kubernetes-pods-to-tailnet-using-sidecar). Ensure your worker architecture and cluster policies support the image and privileged containers. Customer appliance requirements are separate from the [supported hosted build images](https://developer.harness.io/harness-platform/3.0/harness-platform-resources/references/network-and-security/cloud-private-connectivity.md#supported-hosted-build-images).

***

## Deploy an appliance group

{% stepper %}
{% step %}

### Prepare one private YAML file

Download the following file and save it as `customer-tailscale-appliance-secret-state.yaml`:

{% file src="/files/iqqWcexOBjb76yxnArLw" %}
customer-tailscale-appliance-secret-state.yaml
{% endfile %}

Use a unique group name for each role, site, or route set. Replace all occurrences of `tailscale-appliance` together so resource references stay consistent:

{% code overflow="wrap" %}

```bash
set -euo pipefail
set +x
umask 077
PC_CONTEXT='<customer-kubernetes-context>'
PC_NAMESPACE='harness-private-connectivity'
PC_GROUP='site1-both'
PC_AUTH_FILE='/absolute/path/to/appliance.authkey'
PC_DEPLOY_DIR=$(mktemp -d)
PC_MANIFEST="$PC_DEPLOY_DIR/$PC_GROUP.yaml"
sed "s/tailscale-appliance/$PC_GROUP/g" customer-tailscale-appliance-secret-state.yaml > "$PC_MANIFEST"
```

{% endcode %}

Use a lowercase group name with letters, digits, and hyphens, starting and ending with a letter or digit, at most 50 characters. Make it unique across your connected clusters. Start with `spec.replicas: 1`.

The six documents have these purposes:

| Resource                          | Purpose                                                                                            |
| --------------------------------- | -------------------------------------------------------------------------------------------------- |
| Secret                            | Supplies the reusable enrollment key from a mounted file.                                          |
| ServiceAccount, Role, RoleBinding | Allow automatic identity-state storage within the dedicated namespace.                             |
| Headless Service                  | Provides the StatefulSet's service identity; it does not expose the private applications publicly. |
| StatefulSet                       | Runs stable pod ordinals with independent state, readiness checks, and network forwarding.         |
| {% endstep %}                     |                                                                                                    |

{% step %}

### Select a role

Change the main container's `env` entries before the first deployment. The example CIDRs must be permitted by the account configuration and reachable from these pods. With account routes `["*"]`, use actual CIDRs here; never set `TS_ROUTES` or `--advertise-routes` to `*`.

Exit nodes are not supported. Do not add `--advertise-exit-node`, and never set `TS_ROUTES` or `--advertise-routes` to `0.0.0.0/0` or `::/0`.

{% tabs %}
{% tab title="Subnet Router" %}
Use explicit private CIDRs. Set the existing `TS_ROUTES` entry and replace `TS_EXTRA_ARGS` with:

{% code overflow="wrap" %}

```yaml
- name: TS_ROUTES
  value: "10.77.0.0/24"
- name: TS_EXTRA_ARGS
  value: "--advertise-connector=false --accept-routes=false"
```

{% endcode %}

For several CIDRs, use a comma-separated value. A private DNS name behind a configured subnet needs a resolver and route, not necessarily an App Connector.
{% endtab %}

{% tab title="App Connector" %}
Use the account's `domains` configuration. Replace `TS_EXTRA_ARGS` with:

{% code overflow="wrap" %}

```yaml
- name: TS_EXTRA_ARGS
  value: "--advertise-connector --accept-routes=false"
```

{% endcode %}

Remove the downloaded template's existing `TS_ROUTES` entry; do not leave it with an empty value. For app-only appliances, keep `advertiseRoutes: ["*"]` or supply approved ranges containing the application IPs; `[]` permits no application routes. The account allowlist does not create static advertisements on this pod. Custom split DNS may remain empty; the pod's own DNS must resolve the configured domains.
{% endtab %}

{% tab title="Combined" %}
Use both account fields, `advertiseRoutes` and `domains`. Keep static routes in `TS_EXTRA_ARGS`:

{% code overflow="wrap" %}

```yaml
- name: TS_EXTRA_ARGS
  value: "--advertise-connector --advertise-routes=10.77.0.0/24 --accept-routes=false"
```

{% endcode %}

Remove the downloaded template's existing `TS_ROUTES` entry. Reapplying that setting can replace dynamically learned App Connector routes. For multiple static CIDRs, use a comma-separated list in `--advertise-routes`.
{% endtab %}
{% endtabs %}

Keep `TS_ACCEPT_DNS=false`, `--accept-routes=false`, `TS_AUTH_ONCE=true`, and the per-pod `TS_KUBE_SECRET` unchanged. Harness supplies the enrollment identity; do not add customer-selected tags. Domain lists stay in Harness, not in the YAML. Separately named App Connector groups share the account's domain configuration; names are not a per-group domain-policy selector.
{% endstep %}

{% step %}

### Check the pod's DNS

Keep the default cluster DNS when it resolves the private application names. Otherwise, ask your network owner for a resolver that answers both private and public names, and add this under `spec.template.spec`:

```yaml
dnsPolicy: None
dnsConfig:
  nameservers: ["10.77.0.53"]
  options:
    - name: ndots
      value: "1"
```

Replace the example address. Do not mix unrelated private/public resolvers and assume that Kubernetes performs split DNS. The template uses the Kubernetes-injected API service address with normal TLS verification, so a private pod resolver does not need to resolve the Kubernetes API hostname. Account split DNS configures the clients; it does not configure this pod's resolver.
{% endstep %}

{% step %}

### Fill the enrollment secret

The first YAML document creates the auth Secret. Base64-encode the newline-free key locally:

{% code overflow="wrap" %}

```bash
base64 < "$PC_AUTH_FILE" | tr -d '\r\n' > "$PC_DEPLOY_DIR/appliance-key.base64"
```

{% endcode %}

Open that file privately in your editor and replace the YAML's `REPLACE_WITH_BASE64_CUSTOMER_AUTH_KEY` placeholder. The resulting document has this shape:

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: site1-both-auth
type: Opaque
data:
  TS_AUTHKEY: "<base64-of-your-customer-key>"
```

**Base64 is encoding, not encryption.** Treat the completed YAML as a credential: no Git, tickets, screenshots, online encoders, or recorded terminals. Do not put the encoded value directly in the container's environment variable; Kubernetes decodes the Secret and the pod reads the mounted file.

For an externally managed Secret, remove only the first document and provide `site1-both-auth` with data key `TS_AUTHKEY`. Use one owner for that Secret. The remaining YAML is unchanged.
{% endstep %}

{% step %}

### Apply and verify

Create a new namespace once. If it already exists, inspect it and its ownership instead of recreating it:

{% code overflow="wrap" %}

```bash
kubectl --context="$PC_CONTEXT" create namespace "$PC_NAMESPACE"
```

{% endcode %}

Validate the file before applying it:

{% code overflow="wrap" %}

```bash
kubectl --context="$PC_CONTEXT" -n "$PC_NAMESPACE" apply --server-side \
  --field-manager=harness-pc-customer --dry-run=server -f "$PC_MANIFEST"
```

{% endcode %}

Stop if validation fails. Otherwise, apply the file:

{% code overflow="wrap" %}

```bash
kubectl --context="$PC_CONTEXT" -n "$PC_NAMESPACE" apply --server-side \
  --field-manager=harness-pc-customer -f "$PC_MANIFEST"
```

{% endcode %}

Wait for readiness, then check the pod and Tailscale identity:

{% code overflow="wrap" %}

```bash
kubectl --context="$PC_CONTEXT" -n "$PC_NAMESPACE" \
  rollout status "statefulset/$PC_GROUP" --timeout=180s
kubectl --context="$PC_CONTEXT" -n "$PC_NAMESPACE" get pods -l "app=$PC_GROUP"
kubectl --context="$PC_CONTEXT" -n "$PC_NAMESPACE" \
  exec "$PC_GROUP-0" -c tailscale -- tailscale status
```

{% endcode %}

Server-side apply avoids a credential-bearing client-side `last-applied-configuration` annotation; it does not encrypt the key. Keep the same field manager for later updates. Do not use `--force-conflicts` to take over another installer's resources.

When the pods are `Ready` and have distinct device identities, [enable and verify Harness Cloud access](https://developer.harness.io/harness-platform/3.0/harness-platform-resources/references/network-and-security/cloud-private-connectivity/configure.md#enable-and-verify-harness-cloud-access) to check access from Harness.

Use the configured domain name for App Connector access. Go to [Troubleshooting](https://developer.harness.io/harness-platform/3.0/harness-platform-resources/references/network-and-security/cloud-private-connectivity/reference.md#troubleshooting) if the initial request fails.
{% endstep %}
{% endstepper %}

***

## Scale and maintain appliances

### Increase replicas or add another group

For identical configuration, edit the same file from `replicas: 1` to `replicas: 3`, then run the same apply and rollout commands. New pods get `-1` and `-2` identities and automatically created state Secrets. No state-name list, RBAC edits, or new setup call is required.

Check enrollment readiness before scaling:

| State                                                                     | Required action                                                                               |
| ------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
| Reusable enrollment key is still valid                                    | No new key is needed. Scale and verify each new pod.                                          |
| Key expired or was revoked; new pods have no state                        | Mint a new customer key, update the auth Secret in the file, and apply before expansion.      |
| Previously enrolled ordinals return with retained, still-authorized state | They can reuse that identity even after the enrollment key expires. Do not clear their state. |

With the template's default ordered startup, an unready `-1` can prevent `-2` from starting. Check the first failing pod. Do not delete healthy state to bypass enrollment failure. Quotas, worker capacity, outbound access, and destination reachability must also permit expansion.

For different roles, sites, or routes, prepare another file with a new group name, for example `site1-sr`, `site1-ac`, or `site2-both`. These groups can share the dedicated namespace if they share a trust boundary. Each group may use the valid reusable key, but must never copy another group's identity state. Every App Connector must have the DNS and reachability needed for the account domains it serves.

Harness manages load balancing and failover across eligible replicas serving the same routes or applications. No additional load-balancing flags are needed in this YAML. Each client uses a selected replica, so adding replicas does not combine bandwidth for one transfer or guarantee equal utilization. Configure pod placement if replicas must run on different workers or zones. Never force-delete a pod on an unreachable worker until the old process is confirmed stopped.

### Renew the key and update configuration

Use these operations for day-2 changes:

| Change                           | Action                                                                                                                                                                        |
| -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Enrollment key approaches expiry | Call `POST /credential`, replace only the auth Secret's base64 value, and apply the current file.                                                                             |
| Restart or scale down/up         | Retain all per-pod state Secrets; do not logout or request another account setup.                                                                                             |
| Change role or static routes     | PUT the changed Harness fields as needed, deploy a separately named replacement group, verify it, then withdraw the old group’s routes before scaling it to zero (see below). |
| Change App Connector domains     | PUT the `domains` field and verify the names after the synchronous `READY` response; do not add domain flags to pods.                                                         |
| Change DNS or image              | Keep public/private DNS working and use the same pinned image in both containers. Verify a replacement appliance before replacing the remaining replicas.                     |

Enrollment keys expire after 90 days and cannot be made non-expiring through this API. Renew before adding new identities after expiry; already-authorized devices keep working. Go to [renew an appliance key](https://developer.harness.io/harness-platform/3.0/harness-platform-resources/references/network-and-security/cloud-private-connectivity/reference.md#renew-an-appliance-key) for details on the request and response extraction.

Track the returned `expiresAt`. Use the protected API procedure to extract `.data.authKey`, encode it, and update the same file so future scaling cannot restore an older key. Secret-volume updates are asynchronous; verify new pods become Ready. Existing healthy pods need no restart just for auth-key renewal. If an initial bootstrap process does not recover after the updated file is projected, replace only that unready pod normally and retain its state Secret. Do not force deletion.

The template does not automatically renew keys. Use your approved external secret automation if required; do not place a Harness API key inside the appliance.

Account policy/DNS updates do not restart or re-enroll appliances or the Harness-managed cloud delegate. They also do not change the static CIDRs configured on a customer router.

`TS_AUTH_ONCE=true` preserves enrolled state, including learned routes. For role changes, use the replacement-group procedure above rather than editing initial enrollment flags. Preserve other groups' required routes, domains, DNS, and shared namespace.

### Retire a group without leaving stale routes

A stopped appliance can leave approved routes on clients. A broader route on a replacement does not take over a more-specific route from the stopped appliance. After verifying the replacement, run this on each old replica while it is still running:

{% code overflow="wrap" %}

```bash
kubectl --context='<kubernetes-context>' --namespace=harness-private-connectivity exec \
  '<old-workload>-0' -c tailscale -- \
  tailscale set --advertise-connector=false --advertise-routes=
```

{% endcode %}

Then scale the old StatefulSet to zero and verify from a fresh Harness Cloud VM. Keep its state Secrets. This is for intentional retirement or replacement, not ordinary restarts or temporary scale-down. `TS_AUTH_ONCE` preserves these disabled settings: restarting a retired group alone does not reactivate its role.

For redundancy, use appliances serving the same prefixes. Overlapping broader/narrower prefixes are not equivalent backups. Go to [Tailscale high availability](https://tailscale.com/docs/how-to/set-up-high-availability) for details.

***

## Troubleshooting

Use the first failing boundary to diagnose the problem:

<details>

<summary>A new Private Connectivity replica cannot enroll</summary>

Check the first unready ordinal, key expiry or revocation, the mounted raw-key file, and outbound access. Update the auth Secret with a newly minted customer key when required. Keep existing state and healthy pods intact.

</details>

<details>

<summary>A Private Connectivity appliance cannot persist Kubernetes state</summary>

Check the namespaced ServiceAccount, RoleBinding, Secret get/create/update/patch permissions, and Kubernetes API reachability. Do not clear or copy state Secrets. Event-permission warnings alone are not state-write failures.

</details>

<details>

<summary>An appliance is Ready but the private application is unreachable</summary>

Check pod DNS, approved routes, forwarding, target firewall, TLS, and application authorization. A discovered IP outside `advertiseRoutes` is not permitted; expand the approved list only if the customer intends to allow it. Use a configured domain first for App Connector discovery, then verify the actual operation from the Harness Cloud VM.

</details>

***

## Next steps

* [Customer API reference](https://developer.harness.io/harness-platform/3.0/harness-platform-resources/references/network-and-security/cloud-private-connectivity/reference.md): Configuration updates, credential renewal, and troubleshooting.
* [Deploy an appliance on an Ubuntu VM](https://developer.harness.io/harness-platform/3.0/harness-platform-resources/references/network-and-security/cloud-private-connectivity/virtual-machine.md): Deploy an appliance when Kubernetes is unavailable.
