KubeVirt VM Backup & Restore

Introduction

CloudCasa provides backup, restore, and migration services for virtual machines (VMs) running on Kubernetes clusters under KubeVirt or KubeVirt-based systems such as SUSE Virtualization (formerly Harvester) and Red Hat OpenShift Virtualization. CloudCasa agent compatibility has been verified with KubeVirt version v1.0.0 and above, and CDI version v1.57.0 and above. CloudCasa has also been tested with SUSE Harvester versions 1.3.1 & 1.3.2, and Red Hat OpenShift Virtualization 4.16.10.

VM Backup

Backing up KubeVirt VMs requires no more configuration than backing up any other Kubernetes cluster. Detection of VMs is automatic. The entire cluster can be backed up, which is generally recommended, or VMs can be selected individually or by using namespaces and/or labels.

If you wish, individual VMs can easily be selected for backup by using the VMs/PVCs tab In the Selections step of the Cluster Backup wizard. Just click Select resources. The resource browser has multiple views available. Choose VMs to select from a table view of VMs, sortable by VM name or Namespace name. The Search box will filter by VM name.

Individual VMs can also be selected for backup using the resource browser in the Select resources tab of the cluster backup wizard.

When VMs are selected using either of these methods, all resources associated with them are automatically included in the backup.

The Guest Agent

The QEMU Guest Agent (GA), running inside the VM, enables the execution of “freeze” and “unfreeze” hooks within the VM’s operating system. These hooks are used to prepare the filesystem by momentarily halting I/O operations, ensuring that the volume state is stable at the time of the backup. The GA will be used automatically if it is configured on a VM. If the GA is not installed the VM will still be backed up, but without executing the freeze operations. We recommend installing the GA on your VMs to help guarantee backup consistency.

VM Backup Status

During and after backup, the VMs tab in the in the Activity details pane for the backup job will show the status of VM backups.

VM Restore

CloudCasa restores of KubeVirt VMs are performed the same as restores of any other Kubernetes resources. You can restore an entire cluster or one or more namespaces containing VMs. If you wish to select individual VMs for restore, this is most easily done using the Resource Browser. Use the Specific resources tab in the Select Resources step of the cluster restore wizard. Selecting a VM in this way will automatically include all resource associated with it (e.g. PVCs, DataVolumes, ConfigMaps, Secrets), making restore of individual VMs simple.

You can also select specific VM-related resources for restore, although this is not the generally recommended method. For KubeVirt VMs, the list of resources could include, but is not limited to:

  • VirtualMachine (VM): The primary VM definition in KubeVirt.

  • VirtualMachineInstance (VMI): Represents the running instance of a VM.

  • DataVolume: DataVolumes are used to define persistent storage for VMs in KubeVirt, managed by CDI.

  • PersistentVolumeClaim (PVC): PVCs linked to the DataVolumes and used as storage for VM data.

  • Pods: Associated with the VM

  • Secrets: Secrets linked to the VM, such as SSH keys or credentials.

  • ConfigMaps: Configurations that the VM might require on boot or runtime.

Several options related to VM restore are available in the VM options section of the Restore transforms step in the cluster restore wizard. These are:

  • Clear MAC address(es) - Specifies that MAC addresses for restored VMs should be cleared and new addresses should be assigned. This defaults to on. Disable it to preserve the saved MAC addresses for restored VMs. Do this only if you are sure that it will not result in duplicate MAC addresses on your network.

  • Generate new firmware UUID - Specifies that new firmware UUIDs should be generated for restored VMs (see note below). This defaults to off.

  • Run Strategy - This field controls whether and how restored VMs are started after restore. You can select any valid Run Strategy option (see note below) or select “Use strategy from backup” to retain the run strategy for each VM from the backup. If you are uncertain, we recommend selecting “Halted” and booting VMs manually after restore.

A note on Firmware UUIDs

In virtualized environments, each Virtual Machine (VM) is assigned a unique firmware UUID (Universally Unique Identifier) that serves as a distinct identifier for the VM’s hardware components. This UUID is utilized by various software systems, including licensing servers, system management tools, and certain applications, to uniquely recognize and manage individual VMs. In KubeVirt VMs, the firmware UUID is specified within the VirtualMachineInstance (VMI) spec.

When restoring a VM using CloudCasa, if the original firmware UUID is retained, it can lead to conflicts, especially if the original VM is still active. Such conflicts may manifest in issues like:

  • Licensing Problems: Software licenses that are tied to specific hardware identifiers might malfunction or become invalid.

  • Management System Confusion: System management tools could misidentify VMs, leading to operational inconsistencies.

  • Application Errors: Applications that rely on unique hardware identifiers might experience errors or unexpected behavior.

To mitigate these issues, CloudCasa has introduced a new feature that allows the generation of a new firmware UUID during the VM restore process. This enhancement ensures that each restored VM is recognized as a distinct entity, preventing potential conflicts with existing VMs.

By generating a new firmware UUID upon restoration, CloudCasa ensures that each VM maintains a unique identity within the infrastructure, thereby avoiding conflicts and ensuring seamless integration with licensing systems, management tools, and applications.

Run Strategy Settings

The following Run Strategy settings are supported:

  • Use strategy from backup - Retains the run strategy from the backup for each VM.

  • Always: The system is will keep the VM in a running state.

  • RerunOnFailure: Similar to Always, except that the VM is only restarted if it terminated in an uncontrolled way or due to an infrastructure reason.

  • Once: The VM will run once and not be restarted upon completion regardless of the completion status.

  • Manual: The system will not automatically turn the VM on or off. Instead, the user manually controls the VM state.

  • Halted: The system ensures that no VM is in a running state.

Note: Older “spec.running: true” and “spec.running: false” settings are converted to “RunStrategy: Always” and “RunStrategy: Halted”.

See also

See Run Strategies in the Kubevirt User Guide for more information.

Prerequisites

CDI and KubeVirt must be installed prior to restoring a VM backup. These components provide the essential CRDs (Custom Resource Definitions) and controllers required for managing VM and data volume objects. If CDI and KubeVirt are not pre-installed, the restore process will fail because the necessary Kubernetes objects and functionality (like DataVolumes) will be missing. It is highly recommended to have both CDI and KubeVirt deployed before running VM restores, however CDI is not required if the virtual machines themselves are not consuming any virtual disks.

Note that this is generally not a concern if you are restoring to the same cluster, if the entire cluster is being restored, or if you are restoring to a cluster that is pre-configured for running VMs such as OpenShift Virtualization or SUSE Virtualization.

More about CDI

Container Data Importer (CDI) is a KubeVirt subcomponent specifically designed to handle data import, export, and cloning of Persistent Volume Claims (PVCs). CDI is primarily used to import external disk images into Kubernetes as PVCs. This is essential when initially provisioning VMs from existing disk images, such as those in the QCOW2 or RAW format. In the context of backup and restore, CDI ensures that any PVCs associated with VMs (such as data volumes) are properly managed and restored. For instance, when the agent performs a restore operation, CDI helps to rehydrate or re-import the VM disk images into Kubernetes PVCs from the backed-up data.

VM File-level Restore

Since VMs typically use block type PVs that are opaque to the Kubernetes cluster itself, the normal CloudCasa file-level restore facility will not work with them. But fear not! File-level restore is still available. You can retrieve individual files from a VM’s filesystems by using the Recovery Point File Browser. Go to Clusters/Recovery Points and choose the Files action for the desired recovery point. This will open the file and resource browser. Choose the VM you are interested in, and click the Browse icon next to it to open the VM file browser. The VM file browser will actually mount an image of the VM filesystem, so you will need to choose a proxy cluster and select the PVC(s) you are interested in browsing. The Proxy Cluster can be any cluster that supports running VMs and has access to the backup storage containing the recovery point. It defaults to the source cluster.

Note that you may need to wait for several minutes for file data to be retrieved. You can browse the VM’s filesystems and select files to be retrieved. These files can be downloaded directly to your workstation or to another system as a zip file. Click Download and choose “Download Now” or “Create Download Link”. Download links will expire in 1 hour, by default.