Live Migration Fails With Error: "Migration pre-check error: CPU doesn't have compatibility. internal error: Unknown CPU feature ssbd.
Problem
Attempts to perform a live migration of an instance fails with the following error.
Environment
- Platform9 Managed OpenStack - All Versions
- Red Hat Enterprise Linux
Cause
The instance being migrated may require a feature not available on the destination host. In this case, ssbd refers to "Speculative Store Bypass Disable", a Spectre vulnerability mitigation technique available in certain Red Hat kernels and presented as a CPU feature or capability on patched kernels.
When a virtual machine instance is created on a compute node with a kernel that is patched against a side-channel attack using speculative store bypass, subsequent migrations or resizes to other compute nodes may fail if that host has not been patched.
Required features for a virtual machine instance can be found using the virsh dumpxml [domain] command shown here.
Resolution
- Verify that all hosts have the
ssbdCPU flag present.
If patched, the output from cat /proc/cpuinfo will reflect ssbd as a CPU flag.
If patched, the output from virsh capabilities will also reflect the ssbd feature.
Unpatched hosts will not reflect the feature.
Unpatched hosts may reflect two different states, depending on the kernel version.
- If necessary, upgrade the kernel to a patched version.
The following kernels have been observed as providing Speculative Store Bypass mitigation.
Unpatched kernels may include the following.
Patched kernels may be provided during RHEL upgrades or installed manually. Upgrading from Red Hat Enterprise Linux 7.4 to Red Hat Enterprise Linux 7.5 should provide a patched kernel and allow migrations and resizes that previously failed due to this issue to complete properly.
Additional Information
For more information on how Red Hat addresses kernel side-channel attacks using Speculative Store Bypass, please refer to Kernel Side-Channel Attack using Speculative Store Bypass - CVE-2018-3639.
If a kernel upgrade is not possible, or if the error is unrelated to the ssbd CPU feature, an instance's XML file can be modified manually to allow for a successful migration between hosts with different capabilities. Please refer to Live Migration Fails With Error: "Unacceptable CPU Info: CPU Doesn't Have Compatibility."