KubeVirt VM Backup & Restore

Introduction

CloudCasa provides backup, restore, and migration services for virtual machines (VMs) running on Kubernetes clusters under KubeVirt or KubeVirt-based systems such as SUSE Harvester and Red Hat OpenShift Virtualization. CloudCasa agent compatibility has been verified with KubeVirt version v1.0.0 and above, and CDI version v1.57.0 and above. CloudCasa has also been tested with SUSE Harvester versions 1.3.1 & 1.3.2, and Red Hat OpenShift Virtualization 4.16.10.

VM Backup

Backing up KubeVirt VMs requires no more configuration than backing up any other Kubernetes cluster. Detection of VMs is automatic. The entire cluster can be backed up, which is generally recommended, or VMs can be selected by using namespaces and/or labels. Individual VMs can also be selected for backup using the resource browser in the Select resources tab of the cluster backup wizard. When VMs are selected in this way, all resources associated with them are automatically included in the backup.

During and after backup, the VMs tab in the in the Activity details pane for the backup job will show the status of VM backups.

The Guest Agent

The QEMU Guest Agent (GA), running inside the VM, enables the execution of “freeze” and “unfreeze” hooks within the VM’s operating system. These hooks are used to prepare the filesystem by momentarily halting I/O operations, ensuring that the volume state is stable at the time of the backup. The GA will be used automatically if it is configured on a VM. If the GA is not installed the VM will still be backed up, but without executing the freeze operations. We recommend installing the GA on your VMs to help guarantee backup consistency.

VM Restore

CloudCasa restores of KubeVirt VMs are performed the same as restores of any other Kubernetes resources. You can restore an entire cluster or one or more namespaces containing VMs. If you wish to select individual VMs for restore, this is most easily done using the Resource Browser. Use the Specific resources tab in the Select Resources step of the cluster restore wizard. Selecting a VM in this way will automatically include all resource associated with it (e.g. PVCs, DataVolumes, ConfigMaps, Secrets), making restore of individual VMs simple.

You can also select specific VM-related resources for restore, although this is not the generally recommended method. For KubeVirt VMs, the list of resources could include, but is not limited to:

  • VirtualMachine (VM): The primary VM definition in KubeVirt.

  • VirtualMachineInstance (VMI): Represents the running instance of a VM.

  • DataVolume: DataVolumes are used to define persistent storage for VMs in KubeVirt, managed by CDI.

  • PersistentVolumeClaim (PVC): PVCs linked to the DataVolumes and used as storage for VM data.

  • Pods: Associated with the VM

  • Secrets: Secrets linked to the VM, such as SSH keys or credentials.

  • ConfigMaps: Configurations that the VM might require on boot or runtime.

Several options related to VM restore are available in the VM options section of the Restore transforms step in the cluster restore wizard. These are:

  • Clear MAC address(es) - Specifies that MAC addresses for restored VMs should be cleared and new addresses should be assigned. This defaults to on. Disable it to preserve the saved MAC addresses for restored VMs. Do this only if you are sure that it will not result in duplicate MAC addresses on your network.

  • Generate new firmware UUID - Specifies that new firmware UUIDs should be generated for restored VMs (see note below). This defaults to off.

  • Run Strategy - This field controls whether and how restored VMs are started after restore. You can select any valid Run Strategy option or select “Use strategy from backup” to retain the run strategy for each VM from the backup. If you are uncertain, we recommend selecting “Halted” and booting VMs manually after restore.

A note on Firmware UUIDs

In virtualized environments, each Virtual Machine (VM) is assigned a unique firmware UUID (Universally Unique Identifier) that serves as a distinct identifier for the VM’s hardware components. This UUID is utilized by various software systems, including licensing servers, system management tools, and certain applications, to uniquely recognize and manage individual VMs. In KubeVirt VMs, the firmware UUID is specified within the VirtualMachineInstance (VMI) spec.

When restoring a VM using CloudCasa, if the original firmware UUID is retained, it can lead to conflicts, especially if the original VM is still active. Such conflicts may manifest in issues like:

  • Licensing Problems: Software licenses that are tied to specific hardware identifiers might malfunction or become invalid.

  • Management System Confusion: System management tools could misidentify VMs, leading to operational inconsistencies.

  • Application Errors: Applications that rely on unique hardware identifiers might experience errors or unexpected behavior.

To mitigate these issues, CloudCasa has introduced a new feature that allows the generation of a new firmware UUID during the VM restore process. This enhancement ensures that each restored VM is recognized as a distinct entity, preventing potential conflicts with existing VMs.

By generating a new firmware UUID upon restoration, CloudCasa ensures that each VM maintains a unique identity within the infrastructure, thereby avoiding conflicts and ensuring seamless integration with licensing systems, management tools, and applications.

Prerequisites

CDI and KubeVirt must be installed prior to restoring a VM backup. These components provide the essential CRDs (Custom Resource Definitions) and controllers required for managing VM and data volume objects. If CDI and KubeVirt are not pre-installed, the restore process will fail because the necessary Kubernetes objects and functionality (like DataVolumes) will be missing. It is highly recommended to have both CDI and KubeVirt deployed before running VM restores, however CDI is not required if the virtual machines themselves are not consuming any virtual disks.

Note that this is generally not a concern if you are restoring to the same cluster, if the entire cluster is being restored, or if you are restoring to a cluster that is pre-configured for running VMs such as OpenShift Virtualization or SUSE Virtualization.

More about CDI

Container Data Importer (CDI) is a KubeVirt subcomponent specifically designed to handle data import, export, and cloning of Persistent Volume Claims (PVCs). CDI is primarily used to import external disk images into Kubernetes as PVCs. This is essential when initially provisioning VMs from existing disk images, such as those in the QCOW2 or RAW format. In the context of backup and restore, CDI ensures that any PVCs associated with VMs (such as data volumes) are properly managed and restored. For instance, when the agent performs a restore operation, CDI helps to rehydrate or re-import the VM disk images into Kubernetes PVCs from the backed-up data.