E-LAN (VPLS)
E-LAN is a simulated MP2MP (multipoint-to-multipoint) service. The service provider infrastructure acts as one large layer 2 switch, providing a simulated broadcast domain.
Why is it MP2MP and not P2MP? This is because no single port is a root. Every port can send to every other port in MP2MP (”any to any”). If there was a single root, or hub, that can send to all spokes, that is P2MP.
The technology we will focus on which enables this is called VPLS (Virtual Private LAN Service).
In many ways, VPLS is very similar to AToM, in that it is a L2VPN service which uses two labels: a transport label which represents the egress PE, and a service label which represents the VC ID.
Another similarity to AToM is that VPLS uses targeted LDP. But because there are many egress PEs now, and not just one (as in xconnect), there is a single LDP session with every other PE. This means there is a full mesh of LDP neighborships for all PEs participating in the LDP. (VPLS can also use BGP for autodiscovery and service label signaling. We will explore this in a future article.)
The main difference between VPLS and AToM, or “xconnect,” is that the routers must preform bridging functions to simulate a LAN. This involves:
Learning MAC addresses, and aging them out based on a timer
Forwarding frames based on the destination MAC address
Flooding BUM traffic (broadcast, unknown unicast, and multicast)
Loop prevention
With AToM, the router simply transports every frame it receives to the remote egress PE. No bridging/switching functions are needed.
So how does the router know all the other PEs in the VPLS domain in order to form targeted LDP sessions with them? There are two ways: manual configuration of every neighbor PE, or autodiscovery of neighbor PEs using MP-BGP. In this article we will use manual configuration for simplicity in order to focus on the VPLS technology itself, and then add autodiscovery via MP-BGP later.
Lab
Here is our topology:
This service should appear as a giant L2 switch to the customer:
The service provider routers are pre-configured with loopback IPs, LDP, and OSPF.
Here is the VPLS configuration:
PE1
PE2
PE3
vfi stands for virtual forwarding instance.
The bridge-domain is needed to group together physical ports and virtual circuits (which are all defined in the referenced VFI). The bridge-domain creates the switching functionality, allowing the PE to learn MAC addresses.
You may have noticed that on PE3 I used bridge-domain 2 instead of bridge-domain 1. This is to demonstrate that the bridge-domain number is locally signficant. It is not a VLAN ID.
To expand on this concept, imagine this switch configuration:
On the switch, “vlan 100” creates the vlan, and the ports gi0/0 and gi0/1 belong to the vlan. If frames from gi0/0 or gi0/1 are transported over the trunk on gi0/48, an 802.1q tag of 100 is added. So the “100” number both is used to define a domain of bridging, and also defines the specific tag to insert on traffic over a trunk port.
However on the router running VPLS, “bridge-domain 2” does not define a tag to impose on frames. The number 2 is simply used to be able to create multiple, separate bridge-domains and differentiate them from each other.
Some people refer to this concept as a separation of the VLAN tag and flooding domain. The tag on frames from the CE can be anything - double tagged, single tagged, or not tagged at all. You can pop tags, push tags and translate tags. All of this is done on the service instance. The flooding domain is defined by the bridge-domain and has nothing to do with what tag the frames happen to have.
Moving on, the service instance number is also locally signficant, and does not need to match (even though it does in my example).
The only thing that needs to be consistent on all PEs in the VPLS domain is the vpn id. This is similar to the pw-id or xconnect ID that we saw with the psuedowire service.
Flooding Behaviour
With the psuedowire/xconnect service, there was no consideration for flooding behaviour. No matter what the layer 2 payload was, the PE simply encapsulated the frame and transported it to the other end.
However with VPLS, the service provider network has to act as a switch. BUM traffic must be flooded without causing loops, and unicast traffic must be delivered to the correct PE without flooding.
If the core routers were switches, the topology would look like this:
You might wonder, why is each PE directly connected now in this diagram, when there was P4 in the middle and each PE was connected to P4 in the first diagram?
Think of the IP/MPLS network as an underlay, and the VCs forming an overlay.
By manually specifying each neighbor PE in the l2vpn vfi definition, it creates a pseudowire, or VC (virtual circuit) for each neighbor. If PE1 knows that MAC X is behind PE2, it should encapsulate the traffic with an MPLS header for PE2’s loopback. If PE1 knows that MAC Y is behind PE3, it should encapsulate the traffic with a header for PE3’s loopback. In this way, you should think of the overlay topolgy as the diagram above, with each PE directly connected to each other.
So if the CE routers were switches, how would they prevent broadcast storms? You would likely have STP running and one port would be Blocked, as such:
But it would be pretty ugly to run STP over MPLS pseudowires, right? Instead there is a split-horizon rule, which states the following: If a BUM frame is received on a VC, then the router must not forward the frame out any other VC.
Here is an example. The dotted lines are VCs (virtual circuits) and the solid lines are ACs (access circuits).
CE1 originates a broadcast frame
PE1 receives it on an AC (attachment circuit) and therefore forwards it out all ports, including all VCs
PE2 and PE3 receive the frame on a VC, so they only forward out all ACs. They cannot forward a copy out any other VCs.
As part of this process, PE2 and PE3 learn the MAC address aaaa.aaaa.0001 via the pseudowire to PE1, if they have not learned the MAC address already.
This split-horizon behaviour is on by default. To turn it off you would configure the l2vpn vfi like this:
This is actually a technique for creating a P2MP service, known as E-Tree. We will get to this later, but for now, know that you want split-horizon turned on when using VPLS, and it is on by default.
VC setup
Let’s examine pcaps of the LDP mapping messages to see how each PE learns the other neighbor PEs labels to use for each VC for this VPLS service.
R1’s mapping message:
R2’s mapping message:
R3’s mapping message:
To my eye, I cannot see any difference between these mapping messages and the mapping messages for the xconnects. This means that the control plane is exactly the same for xconnects as for VPLS. Essentially, the control plane for VPLS is a full mesh of pseudowires. The difference is the data plane, and how the router inspects the layer two traffic and makes a forwarding/switching decision.
Verification
The show command with verbose output that we used for xconnects also works well for VPLS. This shows the label stack, stats on packet totals, LDP status, and more.
Let’s prove that MTU on the AC must match on all PEs. With an xconnect, the MTU of the interface itself is signaled. But with VPLS, you might have multiple ACs on the PE that belong to the same VPLS domain. So where is the MTU configured?
The answer is that the MTU is configured under the VFI, and is by default 1500.
Let’s change it to 1600 on PE3 to see what happens.
Now on PE1, we see the VC as down.
Let’s put the MTU back to its default of 1500.
CE Configuration
Let’s configure all CEs in the same subnet and enable OSPF. Remember to leave the OSPF network type as the default (broadcast).
All routers see each other as OSPF neighbors
“Routers as switches”
Now that our CEs are functional, lets explore how the PE routers preform switching functions.
First we explore the CAM table on PE1:
Here we see three MAC addresses have been learned. As you may have noticed, the outgoing port for a MAC address can either be a physical port or a pseudoport.
The first MAC address is known via Gi2 on EFP100, which is service instance 100. (EFP = Ethernet Flow Point, which is what is created by the service instance).
The two other MAC addresess are known via the VFI named CUSTOMER_VPLS. But what is 404012 and 404011? These must be the two VCs between PE1 ↔ PE2, and PE1 ↔PE3. But which is which?
Using the somewhat obtuse command below, we can see which pseudoport belongs to which neighbor:
From this information, we can see that the MAC ending in 4581 is known via 3.3.3.3 (PE3), and the MAC ending in 7133 is known via 2.2.2.2 (PE2).
Fixing MTU
The customer sends you a ticket stating that they cannot send packets larger than 1474 bytes. They include this output:
Everything in the lab is at the default MTU of 1500. Why can the customer only send 1474 byte sized packets?
This is because 1474 + 14 (ethernet header) + 4 (control word) + 4 (MPLS top label) + 4 (MPLS bottom label) = 1500
Let’s set the MTU in the MPLS core to the minimum value which will allow the customer to use 1500 byte packets but nothing larger than 1500. Let’s leave the control word on, unlike last time with the xconnect, where we turned the control word off.
What value should we use on our core interfaces?
1500 + 14 (ethernet) + 4 (control word) + 4 (transport label)+ 4 (service label) = 1526
Now 1500 byte size pings work:
Futher Reading
Luc De Ghein, MPLS Fundamentals, Ch. 11 VPLS
Last updated
