VM Creation Fails Due to AIO (Asynchronous I/O) limit on Hypervisor

Problem

VM creation failed due to the following error on the hypervisor:

Unable to use native AIO: failed to create linux AIO context: Resource temporarily unavailable

Environment

• Private Cloud Director Virtualization - v2025.4 and Higher

• Private Cloud Director Kubernetes – v2025.4 and Higher

• Self-Hosted Private Cloud Director Virtualization - v2025.4 and Higher

• Self-Hosted Private Cloud Director Kubernetes - v2025.4 and Higher

• Component: Compute Service

Cause

The asynchronous I/O (AIO) limit on the hypervisor is set too low, preventing libvirt/QEMU from allocating sufficient I/O handles for new VM disk operations.

Diagnostics

1. Check current AIO limit:

$ sysctl fs.aio-max-nr
fs.aio-max-nr = 65536  # example output

1. Review libvirt logs:

$ journalctl -u libvirtd | grep aio

#example error:
Unable to use native AIO: failed to create linux AIO context: Resource temporarily unavailable

Resolution

Increase the AIO limit:

• To increase the limit one time (this change will not persist a node reboot):

# 2097152 should be a safe value to start with and can be monitored against aio-nr for a longer term
$ sudo sysctl -w fs.aio-max-nr=2097152

• To increase the limit permanently (this change will persist a node reboot):

# 2097152 should be a safe value to start with and can be monitored against aio-nr for a longer term
$ vi /etc/sysctl.conf
fs.aio-max-nr = 2097152

Validation

• Confirm updated value:

$ sysctl fs.aio-max-nr
fs.aio-max-nr = 2097152 # Expected output

• Re-attempt VM creation and verify it is active and Running.

Additional Information

• The AIO (Asynchronous I/O) limit is controlled by the Linux kernel parameter fs.aio-max-nr, defines the maximum number of asynchronous I/O requests that can exist system-wide at any given time.

• If the current number of I/O handles /proc/sys/fs/aio-nr comes too close to /proc/sys/fs/aio-max-nr their is high risk of encountering this failure. The parameter values can fluctuate based on several factors, including the number of LUNs, the number of VMs, whether multipath is enabled, and other relevant configurations. These variations directly impact the number of asynchronous I/O operations.

• QEMU/libvirt uses Linux AIO to perform disk I/O operations efficiently for VMs.