Maintenance Mode

Private Cloud Director maintenance mode enables administrators to perform routine maintenance operations on their fleet of hosts in the data center in a safe and orchestrated manner. Maintenance mode allows you to safely migrate virtual machines from a host and prevent new virtual machines from being scheduled during maintenance operations.

What Happens in Maintenance Mode?

When maintenance mode is enabled for a host:

• All running VMs on the host are automatically live-migrated to another compute host in the cluster.

  • You have the option to choose the host to which the VMs will get migrated

• The host is marked as unschedulable, ensuring no new VMs are placed on it.

• Once all VM migrations are complete, the host remains online but isolated from scheduling operations, allowing administrators to perform software updates, hardware maintenance, or diagnostics.

Enabling Maintenance Mode

To enable maintenance mode on a host:

1. Navigate to Infrastructure → Cluster Hosts in the Private Cloud Director UI.
2. Select the target host from the list (optional). If no target host is selected, maintenance mode will select an appropriate target host from the cluster.
3. Click Other → Enable Maintenance Mode.

As noted in point 2 above, you may choose to:

• Allow Private Cloud Director to automatically determine the best destination for each VM based on cluster capacity and placement policies.
• Manually specify a destination host for all VM migrations.

Click Migrate VMs and Enable Maintenance Mode to initiate the process.

After host maintenance operations are completed, you can disable maintenance mode to allow the host to rejoin the scheduler.

Disabling Maintenance Mode

Before disabling maintenance mode, ensure that any maintenance tasks on the host have been completed successfully and that the host is healthy and ready to handle new workloads.

1. Navigate to Infrastructure → Cluster Hosts.

2. Locate the host that is currently in Maintenance Mode.

   1. Hosts in maintenance mode are typically marked with a "Maintenance Mode" label or status indicator.

3. Select the host, then click on Other → Disable Maintenance Mode

The host will be marked as schedulable again and Private Cloud Director will resume placing new virtual machines on this host.

Maintenance Mode Interoperation with Other Services

This section describes how maintenance mode interoperates with other services configured for your cluster.

Host Aggregates

Maintenance mode will honor Host Aggregates and migrate VMs to another host from the same host aggregate.

Limitations:

• All hosts from the Host Aggregate must belong to a single cluster and not span multiple clusters.
• If the host aggregate has a single host that has gone down, maintenance mode will not find a suitable target to migrate the VMs, and maintenance mode will fail.

DRR

A host in maintenance mode will have scheduling VMs disabled while maintenance mode is active. Consequently, DRR will exclude any hosts that are in maintenance mode in a given cluster, and only balance workloads on the remaining active hosts.

VMs with Hard Affinity or Anti-Affinity Rules

• Hard affinity: Maintenance mode will fail if one or more VMs with hard affinity rules are in a powered-on state.
• Soft affinity: Maintenance mode will attempt to keep all VMs in the affinity group together by migrating the first VM to a randomly selected host in the cluster and then migrating the subsequent VMs to the same host until it reaches capacity.
• Hard anti-affinity: Maintenance mode will honor the anti-affinity policy by placing the VMs in the anti-affinity group on separate hosts. If no appropriate target host is found to migrate a VM, maintenance mode will fail.
• Soft anti-affinity: Maintenance mode will attempt to find target hosts that satisfy the anti-affinity policy for each VM to be migrated in a powered-on state, ensuring that all VMs part of the soft anti-affinity policy are placed on separate hosts. If maintenance mode cannot find a suitable target host, maintenance mode will migrate the VM(s) to a target host that violates the soft affinity policy, and the policy violation will be displayed on the UI.

VM States

Maintenance mode will migrate VMs that are in an active state (powered on).

Maintenance mode will not migrate VMs with the following statuses:

• Shutdown (powered off)
• Suspended
• Error
• Unknown

Maintenance mode will not proceed if a VM exists in the following states:

• Pending resize confirmation: The resize operation needs to be confirmed for the VM to be eligible for migration by maintenance mode.
• Rescued: The VM must be un-rescued for the VM to be eligible for migration by maintenance mode.

VMs with Special Properties

1. Virtual TPM-enabled VMs: Maintenance mode does not have the ability to live migrate VMs with Virtual TPM enabled today. This ability is coming soon.
2. Zero flavor VMs: Maintenance mode will migrate VMs created with zero flavor in a powered-on state.
3. VMs with hot-added CPU or memory: Maintenance mode will migrate VMs that have hot-added CPU or memory resources.