Instances Lose External Connectivity After Detaching Floating IP

Problem

After removing a floating IP from an instance, external connectivity ceases.

Environment

  • Platform9 Managed OpenStack - v4.5 - v5.2

  • Neutron

Cause

The neutron-l3-agent service fails to update the routing rule corresponding to the instance's fixed IP within the router namespace on the L3 agent node(s) responsible for hosting the virtual router (in a DVR configuration, this would be the instance hypervisor). Subsequently, this causes a loss of external connectivity for the instance.

Resolution

  1. Identify the fixed IP, network, and host associated with the affected instance.

Screen_Shot_2020-01-02_at_3.38.20_PM.png
Screen_Shot_2020-01-02_at_3.38.20_PM.png
  1. Identify the router associated with the network.

Screen_Shot_2020-01-02_at_3.39.56_PM.png
Screen_Shot_2020-01-02_at_3.39.56_PM.png
  1. Record the UUID of the network and router identified in the previous two steps.

Info

Note: In the UI, this may be done by navigating to the "Networks" tab, selecting either Networks or Routers and then subsequently clicking on the name of the resource. An OpenStack UUID column will be visible in the resource overview.

  1. In the qrouter namespace (corresponding to the UUID of the router) on the instance's host, inspect the routing rules.

  1. Inspect the route table (of which the ID may be identified based on the prior step) corresponding to the instance's fixed IP.

The default route with the rfp interface in the rule indicates all instance traffic is still going to the floating IP namespace.

  1. Delete the stale routing rule.

Additional Information

Last updated