Virtual Machine Migration

VM migration in Private Cloud Director refers to the process of moving a running or stopped virtual machine (VM) from one compute host (hypervisor) to another within the same Private Cloud Director environment. This is a fundamental capability for maintaining system availability, performing infrastructure maintenance, and balancing workloads in data centers.

Common Use Cases

Following are the common scenarios in which migration is initiated in Private Cloud Director.

• Maintenance mode: When you invoke Maintenance Mode on host, VMs are live migrated to other compatible hosts in the virtualized cluster.
• Load Balancing: Private Cloud Director Dynamic Resource Rebalancing (DRR) uses live migration to distribute VMs more evenly across hosts in a virtualized cluster to optimize resource utilization and prevent individual hosts from becoming overloaded.
• Host Failure (VM Evacuation): Private Cloud Director Virtual Machine High Availability (VM HA) service uses VM evacuation to recover VMs from a failing or failed host.

Types of VM Migration

Private Cloud Director supports following types of VM migrations:

• Cold migration
• Live migration
• VM evacuation

Let's dive into the specifics of each.

Cold Migration

Cold migration is a method of moving a virtual machine from one host to another, where the VM is shut down (powered off)or suspended during the process. The VM disks and configuration files are transferred to the destination host as part of the cold migration process.

Downtime: Cold migration essentially requires VM downtime, as the VM must be powered off or suspended before it can be cold migrated.

Cold Migration Prerequisites

• VM should be powered off or suspended.
• Cold migration uses the management network for copying over virtual machine disk information. Make sure that the management network interface has sufficient network bandwidth and capacity to support disk copy operation.
• All hosts within the virtualized cluster must meet CPU Model Pre-requisites.
• Source and destination hosts must run the same operating system version. For more information, refer to the Operating System Compatibility Requirements.

Using CLI

1. Authenticate pcdctl CLI
2. Run the command below to perform the cold migration. The command below will cold migrate the VM to a suitable host within the virtualized cluster.

$ pcdctl server migrate <VM_UUID>

3. To migrate VM to a specific host use the below command:

$ pcdctl server migrate --host <HOST_UUID> <VM_UUID>

Using UI

• Log in to the UI and select Virtual Machine menu on the left side nav bar.
• Select the desired VM then click on the "other" option on the action bar and select the migrate action.
• Select the target hypervisor host and hit the "Migrate VM" button.

Live Migration

Live migration is a method of moving a virtual machine from one host to another, where the VM remains running throughout the migration process.

The live migration process copies a virtual machine's memory from the source to the destination host while the VM is running. Any memory pages that get modified or "dirtied" on the VM during this time are copied over again. Finally, the VM enters a brief pause period during which its remaining memory and CPU state are copied over to the destination host, and finally, the VM is resumed on the destination host.

Downtime: The VM will experience a brief downtime (typically milliseconds to seconds) during the pause period before resuming on the destination host.

Live migration can be classified further by the way it treats virtual machine storage:

• Live Migration for a VM using Ephemeral Shared Storage: The virtual machine has an ephemeral root disk that is located on Ephemeral Shared Storage.

  • Live migration enables the testing and validation of whether the source and target hosts are utilizing the same underlying shared storage.
  • When validated, live migration is performed without copying over VM disk.

• Live Migration of a VM using Ephemeral Local Storage: The Virtual machine has an ephemeral root disk that is located on Ephemeral Local Storage.

  • Live migration will copy over the entire virtual machine root disk from the source host to the destination host.

• Live Migration of a VM using Volumes only: The Virtual machine is using block storage volumes, rather than ephemeral disk.

  • Live migration will migrate the VM without performing a storage copy. The same volumes will continue to be attached to the VM once it's migrated to the target host.

Live Migration Prerequisites

• Live migration uses the management network to copy virtual machine memory and, optionally, disk information. Make sure that the management network interface has sufficient network bandwidth and capacity to support VM memory migration traffic.

  • Support for adding a separate network interface for live migration is coming in the future.

• When using live migration in a production setup, specifically for features like Dynamic Resource Rebalancing (DRR), it is recommended to configure all virtual machines to use shared storage only for the best performance of the migration operation.

• When using Ephemeral Local Storage, live migration requires sufficient network bandwidth and capacity to support virtual machine disk copy.

  • This is not required when using Ephemeral Shared Storage at the virtualized cluster level or using Block Storage Volumes exclusively for all virtual machines.

• All hosts within the virtualized cluster must meet CPU Model Pre-requisites for live migration to work. Read more about CPU Models.

• Source and destination hosts must run the same operating system version. For more information, refer to the Operating System Compatibility Requirements.

Using CLI

1. Authenticate pcdctl CLI
2. Run the command below to perform the live migration.

pcdctl server migrate --live-migration <VMUUID> --host <HOST_UUID>

Using UI

1. Log in to the PCD UI and select Virtual Machine.
2. On Virtual Machine search for a specific VM and select the VM.
3. Click other tab and then select the Migrate.

This will list down the eligible target hypervisors for Live migration.

4. Select the target hypervisor and click Migrate VM.

VM Evacuation

VM Evacuation is specifically designed for disaster recovery. If a host fails or becomes unresponsive, VM evacuation attempts to restart a VM that was running on the failed host onto a healthy host.

VM Evacuation Prerequisites

• VM evacuation can only be performed for VMs using Ephemeral Shared Storage or Block Storage Volumes. If the VM is using Ephemeral Local Storage, evacuation can not work as the virtual machine disk can not be copied over from the source host to a target host.
• All hosts within the virtualized cluster must meet CPU Model Pre-requisites for VM evacuation to work. Read more about CPU Models.
• Source and destination hosts must run the same operating system version. For more information, refer to the Operating System Compatibility Requirements.

Using CLI

1. Authenticate pcdctl CLI
2. Run the command below to perform the live migration.

$ pcdctl server evacuate --host <FAILED_HOST UUID>

Operating System Compatibility Requirements

• Hosts running Ubuntu 22.04 can only migrate VMs to other Ubuntu 22.04 hosts.
• Hosts running Ubuntu 24.04 can only migrate VMs to other Ubuntu 24.04 hosts.
• Cross-version migrations (Ubuntu 22.04 ↔ Ubuntu 24.04) are not supported.

Known Limitation: While initial migration from Ubuntu 22.04 to Ubuntu 24.04 may appear to succeed, subsequent migrations back to Ubuntu 22.04 may fail. This is due to incompatibilities between KVM versions on different operating systems.

Recommended Approach for OS Upgrades

When upgrading your cluster from Ubuntu 22.04 to Ubuntu 24.04 consider the following:

• Plan for one-way migration only (22.04 → 24.04).

• Complete all hosts upgrades before resuming normal migration operations.

• Do not attempt to migrate VMs back to older OS versions.