In the previous blog, we covered the advantages of having a VxLAN control plane. The IS-IS control plane provides basic functionality; VTEP discovery and MAC address synchronization. However, this is a proprietary protocol that doesn’t interoperate. So what is the way forward? It is simpler than you think, we can utilize an existing RFC and extend this with new services: RFC 4760, better known as MP-BGP.
MP-BGP is an extension to the BGP protocol which allows for the exchange of address families between BGP routers (better known as Premises Equipment or PE). These address families can be Layer 2 or Layer 3 based.
Instead of establishing a connection for each exchange, MP-BGP uses a single session (the peer relationship) to exchange multiple address families in a separated fashion. MP-BGP has been around for a long time and has a proven track record of delivering VPN services. If you take VxLAN into consideration, MP-BGP could be used to exchange MAC address information between VTEP’s.
MP-BGP is used to provide control plane services for MPLS Layer 2 and Layer 3 VPN’s. I can hear you think now: “Hold on, you say Layer 2 and Layer 3 VPN’s. If these services have been available for years, why not utilize those instead of developing a completely new set of protocols.”
The MP-BGP functionality as described in RFC 4760 has some limitations. One of these limitations is the lack of multi-homing support. Multi-homing allows for load balancing of flows between tunnel endpoints. In other words, each flow can use a different path (if there are multiple paths available between source and destination) which greatly increases availability and performance.
Another limitation of MP-BGP is that the learning of MAC and IP addresses is still based on flooding rather than exchanging information between tunnel endpoints. This means that with the original MP-BGP functionality BUM traffic is still flooded.
MP-BGP provides a robust foundation for a new set of services that overcome the limitations in RFC4760. These services are described in RFC 7432, better known as EVPN, and deliver the following:
- A scalable Layer 2 and Layer 3 control plane
- Multi-homing services (all active forwarding)
- Operation in conjunction with legacy MPLS services (MPLS over GRE)
- Endpoints can act as proxy for ARP discovery, this limits the amount of ARP traffic on VPN’s
One thing to keep in mind is that the main purpose of the control plane is to distribute MAC and IP address information and with EVPN this can be achieved. Yes, you are reading correctly, also IP address information, let me explain.
The EVPN control plane for exchanging MAC address information works roughly the same as the IS-IS control plane. Once the VTEP’s know about their location, the IS-IS protocol is used to exchange the MAC address information that is tied to the local VTEP’s to the other VTEP’s in the domain.
Now, imagine a situation where a host has to reach a router because it has to communicate with a destination that is not on the same subnet. Typically in a situation where you have VxLAN VTEP’s and a tunnel you would need another (external) device that performs the Layer 3 operations, as shown in the diagram below.
This means that if MAC A on VTEP1 wants to communicate outside its subnet, the Layer 3 traffic has to go to VTEP3. Of course, this is not desirable because this introduces suboptimal routing. It is preferable to have a local breakout on each location for Layer 3 traffic.
A possible solution for this is to use VRRP as shown in the diagram below.
When filtering the VRRP hello packets between the routers, both routers will become Master. This means that hosts in a given subnet will use the local router as the default gateway (communication with the router takes place based on MAC address and local ARP resolution). The main drawback here is that you can only have a pair of routers per VRRP instance (a maximum of two sites), and this only works on a per VLAN basis. Another challenge for using VRRP is that it involves manual configuration.
Now, with EVPN, Layer 3 services can be achieved in a much simpler way by using either the centralized EVPN gateway or distributed EVPN gateway function. With the centralized EVPN gateway, a single VTEP is used to provide the Layer 3 forwarding. For example, in a situation with multiple VTEP’s connecting to each other, one of these VTEP’s will be responsible for routing. With a distributed EVPN gateway, each local VTEP will be responsible for the Layer 3 forwarding. Let’s have a look at the centralized EVPN gateway functionality.
The setup of a centralized gateway consists of the creation of a VSI interface on the VTEP that will deal with the Layer 3 traffic.
If MAC A wants to communicate outside of the Layer 2 domain, it sends an ARP request to obtain the MAC address of VSI interface on VTEP3. This VTEP responds to that ARP request, and then MAC A sends the Layer 3 traffic to the centralized EVPN gateway (VTEP3) through a VxLAN tunnel. Finally, VTEP3 removes the VxLAN header and forwards the traffic at Layer 3. Returning traffic on the Centralized EVPN gateway will be forwarded to MAC A based on the ARP entry for this host.
With the centralized EVPN gateway function we still need to have sufficient bandwidth and resources at the central location, however, a complex VRRP configuration is not required anymore.
That’s it for now. In the next blog, I will be covering the distributed EVPN gateway functionality which will give you much more flexibility and efficiency.