Multiple External Neutron Networks

Starting with release 2.3, Platform9 OpenStack now supports multiple external networks. An external network maps to a physical network in your data center, and provides VMs external connectivity via SNAT, as well as floating (elastic) IPs. VMs cannot be directly attached to an external network. A Neutron router can only connect to one external network. That router will provide the NAT and L3 forwarding between tenant networks, and the connected external network:

• Using SNAT, VMs can communicate from internal tenant networks to the external network and Internet (assuming external network permits Internet access in your infrastructure). One IP address from the external network pool will be consumed for this purpose.
• Assign floating IPs to each VM to have them publicly addressable from the Internet or external network. Each Floating IP will consume one address from the external network.

As with provider networks, an external network may be either Flat or VLAN-based. VLAN-based external networks may share the same trunk port and OVS bridge as traffic for your VLAN-based tenant networks. External networks may span multiple flat, access ports and multiple VLAN-based trunk ports within the same OpenStack deployment.

First, some terminology out of the way:

• An access port represents a single “flat” physical network or VLAN, and will carry untagged traffic.
• A trunk port logically groups together multiple VLANs. An 802.1Q “QTag” header will be inserted into the Ethernet frame for all VLAN traffic. All untagged traffic is implicitly assigned a default, native VLAN per your data center’s switch configuration.

Should I create a Flat or VLAN-based network in Neutron? Neutron and OVS will either send VLAN tagged or untagged traffic, so the choice depends upon your networking infrastructure. Choose:

• Flat: If your network hosts connect to your switches via a dedicated access port, or the external network is the untagged, native VLAN of a trunk port
• VLAN: If your external network is located on a specific VLAN and your hosts are connected via a trunk port. The benefit of this is you do not need a separate NIC on your hosts for each external network.

Warning: Sending untagged traffic for a network intended to be VLAN-based, or tagged traffic on an access port may lead to unintended packet flows or packets being dropped in your switches – please verify your networking infrastructure and topology first.

Each physical network corresponds to a trunk or access port (an individual NIC, or a pair of NICs bonded together) on the host. An Open vSwitch bridge must be created for each physical network.

When configuring Platform9 OpenStack’s Networking Config, each physical network must be given a Label as a name, and that label mapped to a particular OVS bridge on the host during host authorization. Please note that the external network need not be present on every host, only those requiring external connectivity. In a Neutron DVR setup, this includes every hypervisor. In a legacy/standalone Neutron configuration, only dedicated network nodes, not hypervisors, need to have access to the external network.

Examples: Let us walk through a few example network topologies:

Example 1: Legacy topology with 1 external network – flat

In this example, we have a dedicated access port (eth2) connecting to one flat external network. A trunk port configured as a bonded interface (eth0 and eth1) is used to carry VLAN-based tenant network traffic. The external network and its associated OVS bridge is only present on the network host. In total, there are two OVS bridges; br-ext & br-data. We will need to define these labels for these bridges in the Networks –> Networking Config page of the UI, before authorizing the host. Let’s name our labels “external” and “vlannet”. The labels do not need to match the actual OVS bridge names – they are simply handy names to represent the physical networks within OpenStack.

No VLAN range is specified for the “external” label, as it is not used for tenant networks. A range is specified for “vlannet”, but this is only used for auto-assigned tenant networks. An admin user may still create VLAN-based provider and external networks on arbitrary VLANs (see next example).

When authorizing the hosts, both labels will be mapped to OVS bridges on the network node. On hypervisors, only the “vlannet” label needs to be mapped since they do not require direct external connectivity.

Network node:

Hypervisor:

Example 2 – Legacy with multiple external networks – 1 Flat and 1 VLAN

This is similar to our previous example, except we have added an additional VLAN-based external network to our trunk port. Perhaps the original external network ran out of IP addresses – it is easier to provision a new VLAN on an existing port, than add new NICs to all hosts (not to mention more cost effective). What would the Networking Config and host authorization look like? Identical!

We still have the same number of OVS bridges mapped to the same group of ports. Traffic for the second external network is tagged with a VLAN ID of 10, and sent out the trunk port via the br-vlan OVS bridge. While the 10.10.0.0/16 external network is present on all hypervisors, all north-south traffic to this network will still pass thru the network node, then bridged to the hypervisor via the VM’s tenant network.

When creating the external networks, we must select the correct corresponding Network Label and enter the VLAN ID:

Flat Network 10.8.0.0/16:

VLAN network 10.10.0.0/16 on VLAN 10:

Example 3 – DVR with multiple VLAN external networks

DVR deployments are recommended when there is large amounts of east-west L3 traffic, or a lot of north-south traffic to VMs using floating IPs. All north-south floating IP traffic is handled directly on the hypervisors themselves – there are no dedicated network nodes that routed traffic must pass thru. For VMs without Floating IPs, all traffic must pass through a standalone SNAT router – however these will be scheduled across a set of SNAT-capable hypervisors.

As such, every hypervisor needs to have access to the external network(s):

In this example, we have 3 external networks.

Each are VLAN-based (VLANs 8, 9, 10), and share the same trunk port used for VLAN-based tenant network traffic. This reduces complexity as only one NIC, or a pair of NICs in a bond, is needed. This configuration requires only one physical network label & one OVS bridge, br-vlan, to be present on all hosts.

Also, we are only using VXLAN based tenant networks in this example. Therefore, we must enable VLAN for our external networks, but disable it for tenant network creation. The VLAN range field will subsequently disappear in Networking Config:

During host authorization, there will only be one label-OVS bridge to map. You may configure multiple hypervisors as eligible to host SNAT routers. Platform9 recommends having at least 2 SNAT capable hypervisors to handle failover, and spread north-south traffic load:

Finally, all three external networks would be created as VLAN-based with the tags 8, 9, and 10, and the same Physical Network “vlannet” selected. This would be identical to the configuration of the second external network in Example #2.