VMs Unable to Retrieve Metadata From Cloud-Init
Problem
Virtual machines in a specific subnet fail to reach metadata service on 169.254.169.254. As a result, cloud-init did not apply user-data configurations such as password injection.
Environment
- Private Cloud Director Virtualization - v2025.4 and Higher
- Private Cloud Director Kubernetes – v2025.4 and Higher
- Self-Hosted Private Cloud Director Virtualization - v2025.4 and Higher
- Self-Hosted Private Cloud Director Kubernetes - v2025.4 and Higher
- Component:
- Networking ( Neutron/OVN)
Cause
The root cause of the issue is missing distributed Neutron port used for metadata routing. This port, with IP [IP-Address] and device owner network:distributed, is critical for OVN to route metadata requests from VMs to the metadata proxy service.
Diagnostics
- VMs failed to reach
169.254.169.254viapingorcurl. - Gateway IP was also unreachable from within the instance.
ip netns lscommand on impacted hosts showed no namespaces.- Found no distributed Neutron port on the host.
pf9-neutron-ovn-metadata-agentservice was confirmed to be listening on port8775.
Resolution
- Manually create the missing distributed Neutron port using the command:
$ openstack port create --network <NETWORK_ID> \ --fixed-ip subnet=<SUBNET_ID>,ip-address=<IP_ADDRESS>\ --device-owner network:distributed \ metadata-proxy-port-<IP_ADDRESS> +-------------------------+-----------------------------------------------------+| Field | Value | +-------------------------+-----------------------------------------------------+| admin_state_up | UP | | allowed_address_pairs | | | binding_host_id | | | binding_profile | | | binding_vif_details | | | binding_vif_type | unbound | | binding_vnic_type | normal | | data_plane_status | None | | device_id | | | device_owner | network:distributed || device_profile | None | | extra_dhcp_opts | || . | || fixed_ips | ip_address='[IP-Address]', subnet_id='[subnet-id]' | |. | || hardware_offload_type | None || security_group_ids | || status | DOWN |+-------------------------+-----------------------------------------------------+The [IP-Address] is any free IP Address available in the DHCP pool of the subnet.
- Restarted
pf9-neutron-ovn-metadata-agenton all impacted compute nodes.
$ sudo systemctl restart pf9-neutron-ovn-metadata-agent- Verified that
ip netns lsshowed namespaces post-restart.
$ ip netns listovnmeta-[ovnnetns-id-1] (id: 0)- Re-tested metadata access from inside VMs using
curl. - Spawned a new VM with Ubuntu image and confirmed that cloud-init correctly applied the configured password and user-data.
Cirros OS image do not have cloud init service for handling the user data injection. Use image with cloud init service ex: Ubuntu or Rocky Linux for validation
Validation
- From within the VM, below curl command returns the expected user-data content :
$ curl http://169.254.169.254/openstack/latest/meta_data.json{"uuid":"[UUID]", availability_zone":"[AZ]", "hostname":"[hostname]", "name": "[name]","launch_index": 0, "random_seed": "[random-id]", "devices":[], "dedicated_cpus":[]}$ curl http://169.254.169.254/openstack/latest/user_data#cloud-configpassword: [Password] chpasswd: { expire: False }ssh_pwauth: Truemanage_etc_hosts: true runcmd:- ['sh' , '-c', 'echo "Hello World" › /tmp/helloworld.txt' ]- Cloud-init logs inside the Ubuntu VM (/var/log/cloud-init.log) confirmed successful metadata retrieval and password configuration.
- VM SSH access was successful using credentials configured via user-data.
Additional Information
- In OVN-based OpenStack environments, the distributed metadata port must be explicitly present for each subnet requiring metadata access.
Was this page helpful?