etcd

Published: 11 Sep. 2026
Last updated: 11 Sep. 2026

Overview

etcd is a distributed key-value store that provides a reliable way to store data that needs to be accessed by a distributed application. One such application is Kubernetes where etcd is used to store all Kubernetes resource manifests. This page documents how to backup etcd data using CloudCasa. But note that in most cases, you do not need to backup etcd data directly. All CloudCasa backups include the manifests of the selected resources and they can be restored if needed, without directly interacting with etcd. But having backup of etcd data may be needed in some DR scenarios where a cluster or node is being rebuilt.

The instructions in this page would apply to any Kubernetes distribution but for OpenShift, see OpenShift State for a more specific set of steps.

etcd backup

To backup etcd data, we will use the command etcdctl snapshot save to dump the data to a PVC and then backup that PVC. This command can be run using App Hooks but the process requires a running pod where above command is available. One option is to run a pod in some namespace dedicated for this purpose. Such a pod should mount a PVC with enough capacity to hold etcd data. Starting with 30 GB size is a good idea.

The following steps provide detailed information on how such a dedicated pod can be run and how to configure the etcd backup.

Create a namespace called “cloudcasa-etcd-backup”
In that namespace, Create a “secret” containing three pieces of information that are required to connect to etcd:
- etcd-cafile
- etcd-certfile
- etcd-keyfile
Since kube-apiserver also connects to etcd, one can find this information by looking at the command line arguments passed to kube-apiserver process. The options to look for are “etcd-cafile”, “etcd-certfile”, and “etcd-keyfile”. In addition, end point of the etcd server is needed as well. Here is a sample output from one kube-apiserver pod:
```
$ kubectl get pod -n kube-system kube-apiserver-client2 -o yaml | grep etcd
    - --etcd-cafile=/var/lib/minikube/certs/etcd/ca.crt
    - --etcd-certfile=/var/lib/minikube/certs/apiserver-etcd-client.crt
    - --etcd-keyfile=/var/lib/minikube/certs/apiserver-etcd-client.key
    - --etcd-servers=https://127.0.0.1:2379
```
Once these three files are found, copy the files with the following names:
- etcd-cafile => ca.crt
- etcd-certfile => client.crt
- etcd-keyfile => client.key
and create the secret as follows:
```
$ kubectl -n cloudcasa-etcd-backup create secret generic etcd-creds \
  --from-file=ca.crt --from-file=client.crt --from-file=client.key
```
Note that once the secret is created, copies of the files can be deleted. They are not used any more.

Create a pod which is able to run etcdctl commands and with a PVC mounted. The pod also mounts the secret created above. As already mentioned, this pod will be used to run etcdctl command to dump snapshot file. Here is a sample spec for such a pod. Note that some values such as PVC size and exact etcd image can be changed as needed.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: etcd-snapshot-data
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 30Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: etcd-apphook-runner
spec:
  containers:
    - name: etcd
      image: docker.io/bitnamilegacy/etcd:3.6.4-debian-12-r3
      command:
        - sleep
      args:
        - infinity
      volumeMounts:
        - name: data
          mountPath: /data
        - name: etcd-creds
          mountPath: /etc/etcd
          readOnly: true
  volumes:
    - name: data
      persistentVolumeClaim:
        claimName: etcd-snapshot-data
    - name: etcd-creds
      secret:
        secretName: etcd-creds

Assuming the pod spec is in a file called “etcd-backup.yaml”, run the following commands to start the pod and apply a label.

$ kubectl apply etcd-backup.yaml -n cloudcasa-etcd-backup
$ kubectl label pod etcd-apphook-runner -n cloudcasa-etcd-backup app=etcd-backup

Verify that etcd snapshot can be created by exec’ing into the pod and running the following command:
```
etcdctl --endpoints=<ENDPOINT> --cacert=/etc/etcd/ca.crt --cert=/etc/etcd/client.crt --key=/etc/etcd/client.key snapshot save /data/snapshot.db
```
ENDPOINT is the end point of etcd server. It can be found by checking the kube-apiserver command args as described above. Note that if etcd server is running as a pod in the same cluster, the end point seen in the kube-apiserver command line args could be a local address such as “https://127.0.0.1:2379” which may or may not be accessible from this helper pod. An alternate way of finding the end point in this case is to find the “advertised URLs” of the etcd server pod. Here is an example command:
```
kubectl -n kube-system get pod <ETCD-POD-NAME> -o yaml | grep -i advertise
```
If etcd snapshot file is successfully created, move to the next step. Otherwise, troubleshoot and fix the problem. In particular, make sure that end point and credentials are valid.

Create App Hook in Cloudcasa UI:

Pod selector: app = etcd-backup
Container: etcd
Command: etcdctl --endpoints=<ENDPOINT> --cacert=/etc/etcd/ca.crt --cert=/etc/etcd/client.crt --key=/etc/etcd/client.key snapshot save /data/snapshot.db

Example App Hook command:

Command: etcdctl --endpoints=https://192.168.49.2:2379 --cacert=/etc/etcd/ca.crt --cert=/etc/etcd/client.crt --key=/etc/etcd/client.key snapshot save /data/snapshot.db

Note that the above command creates a file with the same name “snapshot.db” every time it runs, thus overwriting previously created file. This is not a problem since the previous copy should have been backed up by the time the app hook runs next time. How ever, you can prevent overwriting the previous file by appending a timestamp to the file name, like so:

/bin/sh -c "etcdctl --endpoints=<ENDPOINT> --cacert=/etc/etcd/ca.crt --cert=/etc/etcd/client.crt --key=/etc/etcd/client.key snapshot save /data/snapshot-$(date +%Y-%m-%d_%H-%M-%S).db"

But in this case, you need to make sure that old files are deleted regularly so that PVC doesn’t run out of space.

Define a backup of the namespace “cloudcasa-etcd-backup” by configuring the app hook created above.

Restoring etcd snapshot file

There are multiple options to restore etcd snapshot files that are backed up using the above mechanism. The files will be in a PVC called “etcd-snapshot-data” in each recovery point.

Note

Restoring etcd snapshot file as described in this section doesn’t automatically update currently running etcd data in the cluster. You will need to take further steps such as etcd restore. The exact steps depend on how you installed the cluster in the first place.

Download files

Using CloudCasa’s file download feature, you can browse the etcd snapshot files in the backup PVC (in namespace “cloudcasa-etcd-backup”) and download them to your local machine. After the files are downloaded, you can copy them over to any host as needed.

File restore to a PVC

Using CloudCasa’s file restore feature, you can browse the etcd snapshot files in the backup PVC (in namespace “cloudcasa-etcd-backup”) and restore them to any target PVC (in any cluster). After the files are restored to a PVC, you can copy them over to any host as needed.

PVC restore

Instead of just restoring files, you can restore entire PVC containing etcd backup files. After restoring the PVC, you may need to mount it to a pod so that you can copy the required files (using kubectl cp for example).

A slight variation of this method is to restore entire namespace “cloudcasa-etcd-backup” to an alternate namespace. This way, you automatically get a pod and the PVC so that you don’t need to run a pod separately. Once the files are copied over, the namespace can be deleted.