Welcome back! Unlike other posts in this blog, used to advertise my own achievements in OpenStack (really?, how you dare), this post will help you to understand how can you connect one virtual machine to two external networks. Of course, avoiding the direct approach of directly creating two ports on each network. No, let’s do this a bit more difficult: let’s connect the virtual machine to a private network and then route the traffic to the external networks.
There are two different approaches to achieve this: (1) using two private networks connected each one to one router and each router to an external network; or (2) using one single network connected to two routers and each router connected to an external network.
Initial deployment.
The first elements we need to have configured in our OpenStack deployment is the two external networks. The provider network, provider type, tag and CIDR will depend on the underlying network connected, that means the external network that will be connected to the OpenStack routers. Unless any other private network, the external networks are directly connected to an existing network. This is why this configuration is done by the administrator.
In our case, I’m going to create two VLAN networks connected to the provider network “datacenter1”:
openstack network create --provider-physical-network datacenter1 --provider-network-type vlan --provider-segment 10 --external external1 openstack network create --provider-physical-network datacenter1 --provider-network-type vlan --provider-segment 20 --external external2 openstack subnet create --subnet-range 10.10.0.0/24 --network external1 sext1 openstack subnet create --subnet-range 10.20.0.0/24 --network external2 sext2
The assigned CIDRs are 10.10.0.0/24 and 10.20.0.0/24.
Using two private networks.
In this one we need to create two private networks:
openstack network create private1 openstack network create private2 openstack subnet create --subnet-range 192.168.10.0/24 --network private1 sprivate1 openstack subnet create --subnet-range 192.168.20.0/24 --network private2 sprivate2
It is worth mentioning that when a subnet is created, a gateway IP address is assigned. This IP can be explicitly defined with the parameter --gateway
in the subnet creation command. If not, the Neutron server will use the first IP address of the subnet CIDR. For example:
$ openstack subnet create --subnet-range 192.168.10.0/24 --network private1 sprivate1 +----------------------+--------------------------------------+ | Field | Value | +----------------------+--------------------------------------+ | allocation_pools | 192.168.10.2-192.168.10.254 | | cidr | 192.168.10.0/24 | | created_at | 2023-04-28T18:26:12Z | | description | | | dns_nameservers | | | dns_publish_fixed_ip | None | | enable_dhcp | True | | gateway_ip | 192.168.10.1 | | host_routes | | | id | e6ca8c46-5163-4487-955f-02e14e112a93 | | ip_version | 4 | | ipv6_address_mode | None | | ipv6_ra_mode | None | | name | sprivate1 | | network_id | dfe77d90-fc4b-4239-8981-e964d973e2fe | | project_id | fcb0a1d3353946c7a2d04373405cc0e8 | | revision_number | 0 | | segment_id | None | | service_types | | | subnetpool_id | None | | tags | | | updated_at | 2023-04-28T18:26:12Z | +----------------------+--------------------------------------+
Now the routers, including the external gateway assignation (that means link the external network to the router) and add the private network (actually the subnet).
openstack router create --external-gateway external1 router1 openstack router create --external-gateway external2 router2 openstack router add subnet router1 sprivate1 openstack router add subnet router2 sprivate2
And finally, the virtual machine creation:
openstack server create --image my_image --flavor my_flavor --network private1 --network private2 server1
That will create a virtual machine with two ports, one in each private network. If we print now the route table of the virtual machine, we’ll see how each subnet, via DHCP, introduced it’s own default gateway entry:
$ ip route show default via 192.168.10.1 dev ens4 proto dhcp src 192.168.10.164 metric 100 default via 192.168.20.1 dev ens3 proto dhcp src 192.168.20.6 metric 100 169.254.169.254 via 192.168.10.2 dev ens4 proto dhcp src 192.168.10.164 metric 100 169.254.169.254 via 192.168.20.2 dev ens3 proto dhcp src 192.168.20.6 metric 100 192.168.10.0/24 dev ens4 proto kernel scope link src 192.168.10.164 192.168.20.0/24 dev ens3 proto kernel scope link src 192.168.20.6
This routing table has a problem: Linux will resolve only one default route. We need to remove one (for example that related to router2) and manually add a new route to 10.20.0.0/24. Now we’ll be able to access to “external2” using “router2”:
ip route del default via 192.168.20.1 ip route add 10.20.0.0/24 via 192.168.20.1 dev ens3
Using one private network connected to two routers.
The previous proposal has a big drawback: we need to access to the virtual machine and manually change the routing table. We can avoid that using one single network connected to “router1” and “router2”. But first let’s step back to the router creation. To link a router with a private subnet (a subnet that belongs to a non-gateway network), the command used was openstack router add subnet <router> <subnet>
. This is where the subnet gateway IP address enter into play. When a subnet is “added” to a router, Neutron is creating a port on this subnet and adding this port as a router interface. The IP address assigned to this port is the subnet gateway IP address.
But when you try to do the same procedure again with other router and the same subnet, that happens:
$ openstack router add subnet router1 sprivate1 $ openstack router add subnet router2 sprivate1 ConflictException: 409: Client Error for url: http://192.168.10.40:9696/networking/v2.0/routers/ad37a008-3360-4a5e-a570-99ae6c70f963/add_router_interface, IP address 192.168.10.1 already allocated in subnet 32639ae8-ad76-4da8-8bc2-0c08f02ea735
That means Neutron can only create one router interface per subnet by default, assigning the gateway IP. This is because of the routing table previously described. The default router using the subnet gateway will be injected inside the virtual machine.
In order to create another router interface, we need to manually create the port and assign this port to the router:
openstack port create --network private1 port_interface_router openstack router add port router2 port_interface_router
Cool, that’s all! No, actually there is one missing bit here: how can we tell the virtual machine to route the traffic to “private2” network? Using the subnet host-route
. This is a subnet parameter that allows to define a route that will be injected inside the routing table, via DHCP. All we need to know is:
- The external network CIDR. In this case, the second external network CIDR is 10.20.0.0/24.
- The router private interface. That means the “port_interface_router” IP address, that will be used as a gateway for the “external2” network. In our case this IP address (randomly assigned by Neutron) is 192.168.10.25.
openstack subnet set --host-route destination=10.20.0.0/24,gateway=192.168.10.25 sprivate1
This second proposal doesn’t require to manually modify the routing table of the virtual machine. All external traffic will go through “router1”, because it has the default gateway IP interface; except for the traffic going to 10.20.0.0/24 that will be routed by “router2”.
As usual, I hope this short guide helps you to understand OpenStack a bit more.