Nova-Compute (ostackhost) Fails to Create Resource Provider: "Conflicting resource provider name: already exists."

Problem

• Deauthorizing and reauthorizing a host with the same hostname sometimes leaves a stale resource provider entry in the Nova __Placement DB.

• The following error is observed in the /var/log/pf9/ostackhost.log log.

None

ERROR nova.scheduler.client.report [...] Failed to create resource provider record in placement API for UUID 19b8176c-8228-43f2-ab7c-8bf1757b5d16. Got 409: {"errors": [{"status": 409, "request_id": "req-fe92d80b-23ac-48d4-96ef-8813e0d3adfe", "detail": "There was a conflict when trying to complete your request.<br> Conflicting resource provider name: host-kvm3 already exists. ", "title": "Conflict"}]}.

Environment

• Platform9 Managed OpenStack - v4.0 and Higher

• Nova-Compute

Cause

The Resource Manager service (running on and exposed from the Platform9 management plane) fails to clean up the resource provider entry in Nova for a host in the two situations below.

1. If the deauthorize operation did not succeed or if the host was forcefully deauthorized by performing a DB operation.

2. If there are any old instances still associated with this host or if there are any stale allocations.

Resolution

Issue 1:

1. Find the older host in the OpenStack Compute service list by its UUID.

None

$ openstack compute service list | grep c00563c1-372a-49f6-bb94-fddf7dfc74b2
| 273 | nova-compute   | c00563c1-372a-49f6-bb94-fddf7dfc74b2       | nova     | enabled  | down  | 2023-09-22T13:40:58.000000 |

1. Delete the stale service by ID.

None

$ openstack compute service delete 273

1. Verify that a new Compute service entry is created with the new host UUID.

None

$  openstack compute service list
| 274 | nova-compute   | 19b8176c-8228-43f2-ab7c-8bf1757b5d16      | nova     | enabled  | up    | 2023-10-02T15:51:43.000000 |

1. Verify that the resource provider got created for the new host. Check the /var/log/pf9/ostackhost.log file to ensure that the resource provider was created successfully.

None

INFO nova.scheduler.client.report [...] Created resource provider record via placement API for resource provider with UUID 19b8176c-8228-43f2-ab7c-8bf1757b5d16 and name host-kvm3.

Issue 2:

1. Check for stale allocations.

None

$ nova-manage placement audit --verbose

1. Get the Host name associated with the Resource Provider having stale allocations.

None

$ TOKEN=$(openstack token issue -f value -c id)
$ curl -X GET https://<FQDN>/placement/resource_providers/<Resource_Provider_ID> -H "X-Auth-Token: $TOKEN" -H "Content-Type: application/json"
{"uuid": "f39ef00a-ced6-4e72-8a10-94f8ec19342d", "name": "host-kvm3", "generation": 2, "links": [{"rel": "self", "href": "/resource_providers/f39ef00a-ced6-4e72-8a10-94f8ec19342d"}, {"rel": "inventories", "href": "/resource_providers/f39ef00a-ced6-4e72-8a10-94f8ec19342d/inventories"}, {"rel": "usages", "href": "/resource_providers/f39ef00a-ced6-4e72-8a10-94f8ec19342d/usages"}]}

1. Delete stale allocations.

None

$ nova-manage placement audit --verbose --delete

1. Delete Resource Provider.

None

$ TOKEN=$(openstack token issue -f value -c id)
$ curl -X DELETE https://<FQDN>/placement/resource_providers/<Resource_Provider_ID> -H "X-Auth-Token: $TOKEN" -H "Content-Type: application/json"

1. Verify that the resource provider got created for the new host. Check the /var/log/pf9/ostackhost.log file to ensure that the resource provider was created successfully.

None

INFO nova.scheduler.client.report [...] Created resource provider record via placement API for resource provider with UUID 19b8176c-8228-43f2-ab7c-8bf1757b5d16 and name host-kvm3.

Additional Information

• Documentation