> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/continuous-delivery/troubleshooting-and-resources/armory/general/configure-persistent-storage-to-capture-heapdumps-for-spinnaker-services.md).

# Configure persistent storage to capture heapdumps for Spinnaker Services

### Introduction <a href="#introduction" id="introduction"></a>

There might be a need to capture `heapdump` of Java-based Spinnaker Services to troubleshoot items such as memory issues. Pods and containers are stateless and the storage on the containers are generally ephemeral which means the ***dump generated is lost when the service goes down***. This article explains the steps involved to configure heapdump and non-ephemeral storage on Spinnaker Services to retain these dumps. Clouddriver is taken as a reference. However, the same can be applied to other Java-based Spinnaker Services such as Orca, Gate, Front50 etc.

### Prerequisites <a href="#prerequisites" id="prerequisites"></a>

N/A

### Instructions <a href="#instructions" id="instructions"></a>

There are couple steps involved to configure the heapdump to be stored on a non-ephemeral storage.

#### Create a storage and mount them on the container for which heapdumps are required <a href="#create-a-storage-and-mount-them-on-the-container-for-which-heapdumps-are-required" id="create-a-storage-and-mount-them-on-the-container-for-which-heapdumps-are-required"></a>

Volumes can be mounted on to the container in two ways as explained below Generate a Kustomize patch and have the volumes and volume mounts created for the container. This involves adding the patch to the deployment of the required Spinnaker Service as shown below

```
apiVersion: spinnaker.armory.io/v1alpha2
kind: SpinnakerService
metadata:
  name: spinnaker
spec:
  kustomize:
    clouddriver:
      deployment:
        patchesStrategicMerge:
          - |
            spec:
              template:
                spec:
                  containers:
                    - name: clouddriver
                      volumeMounts:
                        - name: spinnaker-home
                          mountPath: /path/to/heapdump
                  volumes:
                    - name: spinnaker-home
                      emptyDir: {}​
```

Configure the volumes under the Kubernetes section of `spec.spinnakerConfig.service-settings` of the Spinnaker Service that the volume needs to be mounted in:

```
apiVersion: spinnaker.armory.io/v1alpha2
kind: SpinnakerService
metadata:
  name: spinnaker
spec:
  spinnakerConfig:
    service-settings:
      clouddriver: 
       kubernetes:
          volumes:
           - id: heap-dump
             type: emptyDir
             mountPath: /path/to/heapdump​
```

* After applying one of the above mentioned configurations, the volume can be found mounted in the Clouddriver pod(s).

#### **Configure JAVA\_OPS parameters to generate Heapdump when JVM runs out of memory** <a href="#configure-javaops-parameters-to-generate-heapdump-when-jvm-runs-out-of-memory" id="configure-javaops-parameters-to-generate-heapdump-when-jvm-runs-out-of-memory"></a>

To generate the heapdump for the Spinnaker Service when it runs out of memory, add the below `JAVA_OPTS` parameters for the intended service under `spec.spinnakerConfig.service-settings` of Spinnaker manifest.

```
apiVersion: spinnaker.armory.io/v1alpha2
kind: SpinnakerService
metadata:
  name: spinnaker
spec:
  spinnakerConfig:
    service-settings:
      clouddriver: 
       env:
          JAVA_OPTS: -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/path/to/heapdump
```

This shall generate the heapdump under `/path/to/heapdump` when Clouddriver runs out of memory.