Today’s topic is related to the OVN backend driver in OpenStack Neutron. This is the last in-tree ML2 driver added, initially deployed as an external plugin. From the ovn.org website, “OVN (Open Virtual Network) is a series of daemons for the Open vSwitch that translate virtual network configurations into OpenFlow”. OVN is based on Open vSwitch: the network backend is provided by OVS while OVN is the orchestrator across multiple nodes. The integration with Neutron is done as a ML2 mechanism driver, same as ML2/OVS, ML2/SR-IOV or ML2/Linux Bridge. But it is a bit different from the mentioned ones.
Recently I have come across a possible misunderstanding about the OVN mechanism driver agents. OVN is more than a simple networking backend; it is a cloud management system (CMS) by itself. The ports, networks, routers, ACLs, etc., are stored in the OVN Northbound database. Those abstract concepts are converted to physical resources, represented in the Southbound database. Each OVS service running in the compute nodes is handled by an OVN controller, running locally, that is connected to this Southbound database. You can read more (and better described) about the integration of OVN in Neutron in the RHOSP documentation.
This is a high level architecture schema from the ovn.org website:
CMS
|
|
+-----------|-----------+
| | |
| OVN/CMS Plugin |
| | |
| | |
| OVN Northbound DB |
| | |
| | |
| ovn-northd |
| | |
+-----------|-----------+
|
|
+-------------------+
| OVN Southbound DB |
+-------------------+
|
|
+------------------+------------------+
| | |
HV 1 | | HV n |
+---------------|---------------+ . +---------------|---------------+
| | | . | | |
| ovn-controller | . | ovn-controller |
| | | | . | | | |
| | | | | | | |
| ovs-vswitchd ovsdb-server | | ovs-vswitchd ovsdb-server |
| | | |
+-------------------------------+ +-------------------------------+
This is, in two paragraphs, what OVN is about (now I want my Spanish teacher from primary school to read that and reconsider her statement about me saying that “I didn’t have the ability to summarize”).
Back to OpenStack, how does OVN integrates with Neutron? Easily: Neutron has access to the Northbound and Southbound databases. The OVN ML2 mechanism driver converts the Neutron resources (ports, networks, subnets, floating IPs, etc.) into the OVN ones. The Neutron server is in charge of keeping both databases, the Neutron and the OVN ones, always synchronized.
And where are the agents? When using ML2/OVS or ML2/SR-IOV, Neutron deploys a L2 agent in each host. This L2 agent. This is not the case for OVN because this is executed by the CMS system. We don’t need to deploy any agent; that means the RPC channel is unnecessary (but this, fellow reader, is another spicy story).
$ openstack network agent list +--------------------------------------+--------------------+-------+-------------------+-------+-------+---------------------------+ | ID | Agent Type | Host | Availability Zone | Alive | State | Binary | +--------------------------------------+--------------------+-------+-------------------+-------+-------+---------------------------+ | 658b107b-f43a-46b0-8bcb-e11083634cfd | L3 agent | dev20 | nova | XXX | UP | neutron-l3-agent | | 901d91ee-6984-4ec8-aa03-b02be608c394 | Metadata agent | dev20 | None | :-) | UP | neutron-metadata-agent | | d596219f-65ed-4e93-8250-ff74ae86fc3a | DHCP agent | dev20 | nova | :-) | UP | neutron-dhcp-agent | | e53583ba-df5f-4576-a48d-522df59bf482 | Open vSwitch agent | dev20 | None | :-) | UP | neutron-openvswitch-agent | +--------------------------------------+--------------------+-------+-------------------+-------+-------+---------------------------+
OVN controller agent and OVN metadata agent.
The what? What did you say one paragraph before? Right: we call them agents, we use the agent’s API, the CLI commands, but those “agents” are not handled by the ML2 mechanism driver.
The OVN mechanism driver inherits from MechanismDriver driver, unless other drivers in Neutron that inherit from AgentMechanismDriverBase. The second base class provides integration with the ML2 plugin L2 agents. In other words, the related agents are not spawned by the ML2/OVN mechanism driver.
The OVN controller is “is the local controller daemon for OVN, the Open Virtual Network. It connects up to the OVN Southbound database over the OVSDB protocol, and down to the Open vSwitch database over the OVSDB protocol and to ovs-vswitchd via OpenFlow”. This daemon is deployed during the OVN installation and is a dependant piece of software for the OVN CMS infrastructure. Neutron does not need to handle nor control it.
The OVN metadata agent is a Neutron service, deployed along with the Neutron installation. But this service is strictly not needed by the ML2/OVN mechanism driver. This agent configures the local host and spawns the needed services (for example HAProxy) to act as a proxy between the local virtual machines and the Nova metadata service.
Because both local services are very important for the ML2/OVN mechanism driver, both are exposed as ML2/OVN agents in the Neutron API.
How Neutron stores the OVN agents information.
Neutron database has a specific database table called “agents“. This table stores the status of each spawned agent in every node. A user can add and remove the agents using the CLI.
And… no, this is not how is done in ML2/OVN (let’s have some fun!). The OVN agents (controller and metadata) are stored locally in each API instance, in a agent cache, that is a singleton instance. This information is not backed in the Neutron database; when the API is restarted, it populates the local cache again. This information is stored in the OVN Southband database, in the “Chassis_Private” registers, and read via events:
ChassisAgentDownEvent: sets the agent flag “set_down” to True; once the agent timeout expires, if the flag “set_down” is enabled, then the agent is set to down.ChassisAgentDeleteEvent: this event removes the agent from the local cache.ChassisAgentWriteEvent: this event creates a new register in the local cache.ChassisAgentTypeChangeEvent: the agents can mutate; this event change the agent type (actually changes the class used to model the stored agent).ChassisMetadataAgentWriteEvent: this event refresh the status of the OVN metadata agent. This is similar to the heartbeat sent by the OVS agent or the SR-IOV agent, via RPC, to inform to the Neutron server that the agent is alive and running.
This is a CLI output when listing the agents:
$ openstack network agent list +--------------------------------------+------------------------------+--------+-------------------+-------+-------+----------------------------+ | ID | Agent Type | Host | Availability Zone | Alive | State | Binary | +--------------------------------------+------------------------------+--------+-------------------+-------+-------+----------------------------+ | dcc0a85b-9b20-4e93-9f9f-c50b3c516067 | DHCP agent | u20ovn | nova | XXX | UP | neutron-dhcp-agent | | 62e29a01-a0ac-55c9-b4ec-e223d5c90853 | OVN Metadata agent | u20ovn | | :-) | UP | neutron-ovn-metadata-agent | | a55c8d85-2071-4452-92cb-95d15c29bde7 | OVN Controller Gateway agent | u20ovn | | :-) | UP | ovn-controller | +--------------------------------------+------------------------------+--------+-------------------+-------+-------+----------------------------+
In next posts I’ll talk about the future of the OVN agents, the efforts to change the local cache implementation in order to use the Neutron database, and a new OVN agent that is already implemented “to bring them all and in the darkness bind them”.
2 replies on “ML2/OVN mechanism driver and the “secret agents””
Great article! You truly are great at summarizing!
[…] in ML2/OVN the bandwidth reporting differs a bit. Do you remember that in ML2/OVN there are no agents? How the resource providers are created then? It is the Neutron server who reads the OVN Southband […]