Error “/var/lib/kubelet/pods//volumes: no such file or directory”

Summary

You may sometimes see this type of error in the job activity logs:

“Failed to copy PVC <PVC-NAME> in namespace <NAMESPACE>. Error is stat /var/lib/kubelet/pods//volumes: no such file or directory”

Details

This error is caused by attempting to use the backup method “Read data from underlying host volume” on PVCs which are not attached to any pod. When this backup method is used, the data mover tries to read the data from a host directory that is mounted under the “pod” directory. If the PVC is not attached to a pod, there will not be a host directory for it, so the read fails with this error.

For this reason, unattached PVCs cannot be backed up using the “Read data from underlying host volume” method. The other LIVE method “Read data from PVC” should work, as should the snapshot method.

If it is not possible or desirable to change the backup method at the storage class level (e.g. because the method works for all other PVCs), you can configure a different backup method just for this PVC by setting the label cloudcasa.io/backup-method. Possible values for this label are as follows:

SNAPSHOT

Take snapshot of PVC and then create a temp PVC from the snapshot, to be used for backup.

LIVE_FROM_PVC

Mount the PVC to the mover pod and read it.

LIVE_FROM_HOST_POD_VOL

Directly read from the underlying host volume. Avoid this for unattached PVCs

LIVE

Pick one of the above two “live” options automatically.

SKIP

Do not backup PVC data. Note that PVC/PV resource manifests will still be backed up.