L3/routing OpenStack

Routing two external networks

Welcome back! Unlike other posts in this blog, used to advertise my own achievements in OpenStack (really?, how you dare), this post will help you to understand how can you connect one virtual machine to two external networks. Of course, avoiding the direct approach of directly creating two ports on each network. No, let’s do this a bit more difficult: let’s connect the virtual machine to a private network and then route the traffic to the external networks.

There are two different approaches to achieve this: (1) using two private networks connected each one to one router and each router to an external network; or (2) using one single network connected to two routers and each router connected to an external network.

Initial deployment.

The first elements we need to have configured in our OpenStack deployment is the two external networks. The provider network, provider type, tag and CIDR will depend on the underlying network connected, that means the external network that will be connected to the OpenStack routers. Unless any other private network, the external networks are directly connected to an existing network. This is why this configuration is done by the administrator.

In our case, I’m going to create two VLAN networks connected to the provider network “datacenter1”:

openstack network create --provider-physical-network datacenter1 --provider-network-type vlan --provider-segment 10 --external external1
openstack network create --provider-physical-network datacenter1 --provider-network-type vlan --provider-segment 20 --external external2
openstack subnet create --subnet-range --network external1 sext1
openstack subnet create --subnet-range --network external2 sext2

The assigned CIDRs are and

Using two private networks.

In this one we need to create two private networks:

openstack network create private1
openstack network create private2
openstack subnet create --subnet-range --network private1 sprivate1
openstack subnet create --subnet-range --network private2 sprivate2

It is worth mentioning that when a subnet is created, a gateway IP address is assigned. This IP can be explicitly defined with the parameter --gateway in the subnet creation command. If not, the Neutron server will use the first IP address of the subnet CIDR. For example:

$ openstack subnet create --subnet-range --network private1 sprivate1 
| Field                | Value                                |
| allocation_pools     |          |                   
| cidr                 |                      |                   
| created_at           | 2023-04-28T18:26:12Z                 |                   
| description          |                                      |                   
| dns_nameservers      |                                      |                   
| dns_publish_fixed_ip | None                                 |                   
| enable_dhcp          | True                                 |                   
| gateway_ip           |                         |                   
| host_routes          |                                      |                   
| id                   | e6ca8c46-5163-4487-955f-02e14e112a93 |                   
| ip_version           | 4                                    |                   
| ipv6_address_mode    | None                                 |                   
| ipv6_ra_mode         | None                                 |                   
| name                 | sprivate1                            |                   
| network_id           | dfe77d90-fc4b-4239-8981-e964d973e2fe |                   
| project_id           | fcb0a1d3353946c7a2d04373405cc0e8     |                   
| revision_number      | 0                                    |                   
| segment_id           | None                                 |                   
| service_types        |                                      |                   
| subnetpool_id        | None                                 |                   
| tags                 |                                      |                   
| updated_at           | 2023-04-28T18:26:12Z                 |                   

Now the routers, including the external gateway assignation (that means link the external network to the router) and add the private network (actually the subnet).

openstack router create --external-gateway external1 router1
openstack router create --external-gateway external2 router2
openstack router add subnet router1 sprivate1
openstack router add subnet router2 sprivate2

And finally, the virtual machine creation:

openstack server create --image my_image --flavor my_flavor --network private1 --network private2 server1

That will create a virtual machine with two ports, one in each private network. If we print now the route table of the virtual machine, we’ll see how each subnet, via DHCP, introduced it’s own default gateway entry:

$ ip route show
default via dev ens4 proto dhcp src metric 100
default via dev ens3 proto dhcp src metric 100 via dev ens4 proto dhcp src metric 100 via dev ens3 proto dhcp src metric 100 dev ens4 proto kernel scope link src dev ens3 proto kernel scope link src

This routing table has a problem: Linux will resolve only one default route. We need to remove one (for example that related to router2) and manually add a new route to Now we’ll be able to access to “external2” using “router2”:

ip route del default via
ip route add via dev ens3

Using one private network connected to two routers.

The previous proposal has a big drawback: we need to access to the virtual machine and manually change the routing table. We can avoid that using one single network connected to “router1” and “router2”. But first let’s step back to the router creation. To link a router with a private subnet (a subnet that belongs to a non-gateway network), the command used was openstack router add subnet <router> <subnet>. This is where the subnet gateway IP address enter into play. When a subnet is “added” to a router, Neutron is creating a port on this subnet and adding this port as a router interface. The IP address assigned to this port is the subnet gateway IP address.

But when you try to do the same procedure again with other router and the same subnet, that happens:

$ openstack router add subnet router1 sprivate1
$ openstack router add subnet router2 sprivate1
ConflictException: 409: Client Error for url:, IP address already allocated in subnet 32639ae8-ad76-4da8-8bc2-0c08f02ea735

That means Neutron can only create one router interface per subnet by default, assigning the gateway IP. This is because of the routing table previously described. The default router using the subnet gateway will be injected inside the virtual machine.

In order to create another router interface, we need to manually create the port and assign this port to the router:

openstack port create --network private1 port_interface_router
openstack router add port router2 port_interface_router

Cool, that’s all! No, actually there is one missing bit here: how can we tell the virtual machine to route the traffic to “private2” network? Using the subnet host-route. This is a subnet parameter that allows to define a route that will be injected inside the routing table, via DHCP. All we need to know is:

  • The external network CIDR. In this case, the second external network CIDR is
  • The router private interface. That means the “port_interface_router” IP address, that will be used as a gateway for the “external2” network. In our case this IP address (randomly assigned by Neutron) is
openstack subnet set --host-route destination=,gateway= sprivate1

This second proposal doesn’t require to manually modify the routing table of the virtual machine. All external traffic will go through “router1”, because it has the default gateway IP interface; except for the traffic going to that will be routed by “router2”.

As usual, I hope this short guide helps you to understand OpenStack a bit more.

Leave a Reply

Your email address will not be published. Required fields are marked *