VM Creation Failure Due to Image Role Host and Stale OVN NetNS Issue

Problem

VM creation was failing at the volume-from-image stage due to the image role host being in a failed state. The failure was linked to a stale OVN metadata network namespace, which caused the pf9-neutron-ovn-metadata-agent to crash, impacting Glance functionality.

Environment

• Private Cloud Director Virtualization - v2025.4 and Higher

• Private Cloud Director Kubernetes – v2025.4 and Higher

• Self-Hosted Private Cloud Director Virtualization - v2025.4 and Higher

• Self-Hosted Private Cloud Director Kubernetes - v2025.4 and Higher

• Component: Networking and Image Service

Cause

The root cause was traced to the Image role host being in a failed state. Additionally, a stale/corrupted OVN metadata namespace caused the pf9-neutron-ovn-metadata-agent service to fail, which contributed to network namespace issues affecting Image service and possibly volume creation.

Diagnostics

• Image service backend was not visible in the openstack volume backend pool list command output.

• pf9-neutron-ovn-metadata-agent was in a failed state with the following critical log in /var/log/pf9/pf9-neutron-ovn-metadata-agent.log:

[TimeStamp] CRITICAL neutron [-] Unhandled error: OSError: [Errno 22] failed to open netns

• Stale/invalid NETNS entry found under /var/run/netns:

Affected Host:

$ ip netns list
ovnmeta-[ovnnetns-id-1] (id: 0)
ovnmeta-[ovnnetns-id-2]

$ ls -l /var/run/netns
total 0
-r--r--r-- 1 root root 0 [Time Stamp] ovnmeta-[ovnnetns-id-1]
---------- 1 root root 0 [Time Stamp] ovnmeta-[ovnnetns-id-2]

• The invalid namespace had no permissions and was inaccessible..

Resolution

• Identified that pf9-neutron-ovn-metadata-agent was in a failed state.

• Found a stale NETNS entry with invalid permissions:

---------- 1 root root 0 Jun [Time Stamp] ovnmeta-[ovnnetns-id-2]

• Deleted the stale netns:

$ sudo ip netns delete ovnmeta-<OVS_NETNS_ID_2>

• Restarted the metadata agent:

$ sudo systemctl restart pf9-neutron-ovn-metadata-agent

• Restarted the glance-api service to ensure backend availability:

$ sudo systemctl restart pf9-glance-api

Validation

• Post-remediation, pf9-neutron-ovn-metadata-agent started successfully without errors.

• Image service backend was registered properly.

• VM creation using image-backed volume was tested and succeeded.