E-LAN (EVPN Single-Homed)

In this article we will use EVPN to create an E-LAN service. EVPN will replace VPLS that we configured in the first E-LAN article.

PEs will continue to learn MAC addresses in the data plane on the access circuits (ACs) but they will now learn remote MACs in the control plane via EVPN. This will reduce flooding of unknown unicast addresses, and prevents the need for PEs to age-out MACs located on other remote PEs.

We will keep this simple and use single-homing, so the ESI will be all zeros for every interface. This will let us focus on how EVPN and VPLS differ.

With EVPN E-LAN, we will see the type 2 route type, which in my opinion is the defining route type for EVPN overall. This route type advertises a single MAC address and optionally the IP associated with that MAC. When a PE learns the MAC of a CE that is connected to it, a type 2 route is created.

Route type 3 is also used in order to transport BUM traffic. An ingress PE needs to know which PEs to send BUM traffic to. Each PE advertises the fact that it participates in the EVI by using the type 3 route. This is similar to the auto-discovery function of Kompella circuits that we configured in the VPLS with BGP auto-discovery article.

To summarize, while EVPN VPWS used type 1 and type 4 (when multi-homing), E-LAN uses type 2 and type 3. If you were to use multi-homing with EVPN E-LAN you would see four total route types.

Lab

We will re-use the existing lab from the VPLS-based E-LAN using Martini circuits.

Here are the startup configs if you are building the lab from scratch:

PE1#
hostname PE1
!
interface Loopback0
 ip address 1.1.1.1 255.255.255.255
!
interface GigabitEthernet1
 ip address 10.1.4.1 255.255.255.0
 ip ospf network point-to-point
 mpls ip
!
interface GigabitEthernet2
 no ip address
 service instance 100 ethernet
  encapsulation default
!
router ospf 1
 network 0.0.0.0 255.255.255.255 area 0

PE2#
hostname PE2
!
interface Loopback0
 ip address 2.2.2.2 255.255.255.255
!
interface GigabitEthernet1
 ip address 10.2.4.2 255.255.255.0
 ip ospf network point-to-point
 mpls ip
!
interface GigabitEthernet2
 no ip address
 service instance 100 ethernet
  encapsulation default
!
router ospf 1
 network 0.0.0.0 255.255.255.255 area 0

PE3#
hostname PE3
!
interface Loopback0
 ip address 3.3.3.3 255.255.255.255
!
interface GigabitEthernet1
 ip address 10.3.4.3 255.255.255.0
 ip ospf network point-to-point
 mpls ip
!
interface GigabitEthernet2
 no ip address
 service instance 100 ethernet
  encapsulation default
!
router ospf 1
 network 0.0.0.0 255.255.255.255 area 0

P4#
hostname P4
!
interface Loopback0
 ip address 4.4.4.4 255.255.255.255
!
interface GigabitEthernet1
 ip address 10.1.4.4 255.255.255.0
 ip ospf network point-to-point
 mpls ip
!
interface GigabitEthernet2
 ip address 10.2.4.4 255.255.255.0
 ip ospf network point-to-point
 mpls ip
!
interface GigabitEthernet3
 ip address 10.3.4.4 255.255.255.0
 ip ospf network point-to-point
 mpls ip
!
router ospf 1
 network 0.0.0.0 255.255.255.255 area 0

#CE1
hostname CE1
!
int lo0
 ip address 1.1.1.1 255.255.255.255
!
int Gi0/0
 ip address 10.1.1.1 255.255.255.0
!
router ospf 1
 network 0.0.0.0 255.255.255.255 area 0

#CE2
hostname CE2
!
int lo0
 ip address 2.2.2.2 255.255.255.255
!
int Gi0/0
 ip address 10.1.1.2 255.255.255.0
!
router ospf 1
 network 0.0.0.0 255.255.255.255 area 0

#CE3
hostname CE3
!
int lo0
 ip address 3.3.3.3 255.255.255.255
!
int Gi0/0
 ip address 10.1.1.3 255.255.255.0
!
router ospf 1
 network 0.0.0.0 255.255.255.255 area 0

First we will add BGP with the l2vpn/evpn address-family on all core routers. This closely resembles the EVPN VPWS configuration we did earlier, just with different neighbor IP addresses.

PE1,2,3#
router bgp 65000
 neighbor 4.4.4.4 remote-as 65000
 neighbor 4.4.4.4 update-source lo0
 address-family l2vpn evpn
  neighbor 4.4.4.4 activate

P4#
router bgp 65000
 neighbor 1.1.1.1 remote-as 65000
 neighbor 1.1.1.1 update-source lo0
 neighbor 2.2.2.2 remote-as 65000
 neighbor 2.2.2.2 update-source lo0
 neighbor 3.3.3.3 remote-as 65000
 neighbor 3.3.3.3 update-source lo0
 address-family l2vpn evpn
  neighbor 1.1.1.1 activate
  neighbor 1.1.1.1 route-reflector-client
  neighbor 2.2.2.2 activate
  neighbor 2.2.2.2 route-reflector-client
  neighbor 3.3.3.3 activate
  neighbor 3.3.3.3 route-reflector-client

Now we’ll remove the existing l2vpn vfi, that was left over from the Maritini circuit based VPLS configuration.

PE1,2,3#
bridge-domain 1
 no member vfi CUSTOMER_VPLS
no l2vpn vfi context CUSTOMER_VPLS

First let’s enable EVPN on only PE1, and then take a look at the routes it advertises.

PE1#
l2vpn evpn
 replication-type ingress
!
l2vpn evpn instance 1 vlan-based
!
bridge-domain 1
 member evpn-instance 1

Those simple 5 lines are all we need to do to enable EVPN E-LAN (with all defaults). In line 2 and 3, we define the replication-type for BUM traffic. (The only option is ingress replication). When this PE receives a BUM frame from a directly connected CE, it will replicate the frame and send it to all PEs participating in the EVI. The PE knows which PEs are participating via the type 3 routes each PE advertises. This is also referred to as HER (Head End Replication).

In line 5 we define the EVPN instance as instance 1, and then associate it with bridge-domain 1 in lines 7 and 8.

The EVPN instance is defined as vlan-based, which is the simpliest type. There are three options:

VLAN-based
- One EFP (one VLAN) is asssociated to a BD, and that BD is associated to an EVI
VLAN-bundle
- Multiple EFPs (multiple VLANs) shared a single BD, which is associated to an EVI
VLAN-aware
- Each EFP (VLAN) is associated with a single BD, but there can be multiple BDs per EVI

If you haven’t already from the previous lab, associate Gi2 with the bridge-domain

bridge-domain 1
 member Gi2 service-instance 100

We can now confirm the EVPN EVI is up:

PE1#show l2vpn evpn evi 1 detail
EVPN instance:       1 (VLAN Based)
  RD:                1.1.1.1:1 (auto)
  Import-RTs:        65000:1 
  Export-RTs:        65000:1 
  Per-EVI Label:     none
  State:             Established
  Replication Type:  Ingress (global)
  Encapsulation:     mpls
  Bridge Domain:     1
    Ethernet-Tag:    0
    BUM Label:       22
    Per-BD Label:    17
    State:           Established
    Access If:       
    Pseudoports:
      GigabitEthernet2 service instance 100
        Routes: 1 MAC, 0 MAC/IP

This output shows us that the RD and RTs have been autogenerated for us. We can see that the replication type is ingress, which we globally set in our configuration for all EVPN EVIs (lines 1 and 2). Encapsulation is mpls by default, and you can see the labels reserved for this EVI, and the state of the EVI (Established). The bottom of the output shows that there is a single MAC route. This is the MAC address of CE1 connected on Gi2. PE1 has already learned the MAC of CE1 due to it multicasting OSPF Hellos.

PE1#show bgp l2vpn evpn | begin Network
     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1.1.1.1:1
 *>   [2][1.1.1.1:1][0][48][5254000F1800][0][*]/20
                      ::                                 32768 ?
 *>   [3][1.1.1.1:1][0][32][1.1.1.1]/17
                      ::                                 32768 ?

We can see that a type 2 route for CE1’s MAC has been advertised, as well as a type 3 route that advertises that PE1 is participating in the EVI. The type 3 route is used by other PEs to populate the BUM flood list.

Let’s enable EVPN on PE2 and PE3 and see if OSPF adjacencies come up.

PE2,3#
l2vpn evpn
 replication-type ingress
!
l2vpn evpn instance 1 vlan-based
!
bridge-domain 1
 member evpn-instance 1
 member Gi2 service-instance 100

Now each PE has 4 routes received from the RR. This is because each PE is advertising a single type 2 route and a single type 3 route.

PE1#show bgp l2vpn evpn sum | begin Neighbor
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
4.4.4.4         4        65000      29      23       11    0    0 00:14:55        4

OSPF comes up on all CEs

CE1#show ip ospf nei

Neighbor ID     Pri   State           Dead Time   Address         Interface
2.2.2.2           1   FULL/DROTHER    00:00:38    10.1.1.2        GigabitEthernet0/0
3.3.3.3           1   FULL/DR         00:00:34    10.1.1.3        GigabitEthernet0/0

Let’s examine in the CAM table of PE1

PE1#show bridge-domain 1
Bridge-domain 1 (2 ports in all)
State: UP                    Mac learning: Enabled
Aging-Timer: 300 second(s)
Maximum address limit: 65536
    GigabitEthernet2 service instance 100
    EVPN Instance 1
   AED MAC address    Policy  Tag       Age  Pseudoport
   -   5254.000F.1800 forward dynamic_c 297  GigabitEthernet2.EFP100
   -   5254.0013.4581 forward static_r  0    OCE_PTR:0xe89d0860
   -   5254.0016.7133 forward static_r  0    OCE_PTR:0xe89d08a0

The remote CEs’ MACs are learned statically, not dymaically. The locally attached CE’s MAC is the only MAC being learned dynamically and therefore being aged out.

Flooding Process

What happens when CE1 sends a BUM frame?

PE1 does ingress replication, as we configured before. So PE1 replicates the frame for as many PEs as are participating in the EVI, and then sends each PE a copy. PE1 learns which PEs are participating in the EVI via the type 3 EVPN routes.

Remember earlier in the detailed EVI output, how there were two labels? Each PE reserves a BUM label and a per-BD label for the EVI. PE1 uses the PE’s advertised BUM label to send BUM traffic to the PE. Each PE advertised this label in the type 3 route. Remember, the type 3 route is all about BUM traffic replication. Each PE signals that it participates in the EVI and wants to receive BUM traffic, as well as the BUM label it has reserved for this EVI.

Let’s figure out what label stack PE1 will push on BUM traffic that it replicates and sends it to PE2 and PE3.

PE1#show ip cef 3.3.3.3        
3.3.3.3/32
  nexthop 10.1.4.4 GigabitEthernet1 label 18-(local:16)
PE1#show ip cef 2.2.2.2 
2.2.2.2/32
  nexthop 10.1.4.4 GigabitEthernet1 label 17-(local:21)


PE1#show bgp l2vpn evpn rd 2.2.2.2:1 detail
<snip>
BGP routing table entry for [3][2.2.2.2:1][0][32][2.2.2.2]/17, version 8
  Paths: (1 available, best #1, table EVPN-BGP-Table)
  Flag: 0x100
  Not advertised to any peer
  Refresh Epoch 2
  Local
    2.2.2.2 (metric 3) (via default) from 4.4.4.4 (4.4.4.4)
      Origin incomplete, metric 0, localpref 100, valid, internal, best
      Extended Community: RT:65000:1
      Originator: 2.2.2.2, Cluster list: 4.4.4.4
      PMSI Attribute: Flags:0x0, Tunnel type:IR, length 4, label:31 tunnel identifier: < Tunnel Endpoint: 2.2.2.2 >
      rx pathid: 0, tx pathid: 0x0
      Updated on Jul 9 2022 19:27:27 UTC


PE1#show bgp l2vpn evpn rd 3.3.3.3:1 detail  
<snip>
BGP routing table entry for [3][3.3.3.3:1][0][32][3.3.3.3]/17, version 4
  Paths: (1 available, best #1, table EVPN-BGP-Table)
  Flag: 0x100
  Not advertised to any peer
  Refresh Epoch 2
  Local
    3.3.3.3 (metric 3) (via default) from 4.4.4.4 (4.4.4.4)
      Origin incomplete, metric 0, localpref 100, valid, internal, best
      Extended Community: RT:65000:1
      Originator: 3.3.3.3, Cluster list: 4.4.4.4
      PMSI Attribute: Flags:0x0, Tunnel type:IR, length 4, label:31 tunnel identifier: < Tunnel Endpoint: 3.3.3.3 >
      rx pathid: 0, tx pathid: 0x0
      Updated on Jul 9 2022 19:26:54 UTC

In the output above, we can see that the outgoing transport label for PE3’s loopback is 18, and for PE2 is 17. From the type 3 EVPN route that each PE originated, we can see the BUM label. Each PE advertised a BUM label of 31 in their type 3 routes. This just happened to be the same.

Taking a pcap on Gi1, we can see the labels PE1 imposes on broadcast frames that are originated from CE1, and then replicated to PE2 and PE3. The frame towards PE3 has 18/31 and towards PE2 has 17/31

We can also see OSPF hellos from CE2 and CE3 being multicasted to PE1 with PE1’s BUM label, which is 22. We can see that value in multiple places - the EVPN EVI detailed output, the LFIB, and the BGP route that PE1 generated.

PE1#show l2vpn evpn evi 1 detail 
EVPN instance:       1 (VLAN Based)
  RD:                1.1.1.1:1 (auto)
  Import-RTs:        65000:1 
  Export-RTs:        65000:1 
  Per-EVI Label:     none
  State:             Established
  Replication Type:  Ingress (global)
  Encapsulation:     mpls
  Bridge Domain:     1
    Ethernet-Tag:    0
    BUM Label:       22
    Per-BD Label:    17

PE1#show mpls forwarding-table labels 22
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
22         No Label   evpn(mc:bd 1)    29087         none       point2point

PE1#show bgp l2vpn evpn route-type 3 0 1.1.1.1 
BGP routing table entry for [3][1.1.1.1:1][0][32][1.1.1.1]/17, version 2
Paths: (1 available, best #1, table evi_1)
  Advertised to update-groups:
     1         
  Refresh Epoch 1
  Local
    :: (via default) from 0.0.0.0 (1.1.1.1)
      Origin incomplete, localpref 100, weight 32768, valid, sourced, local, best
      Extended Community: RT:65000:1
      PMSI Attribute: Flags:0x0, Tunnel type:IR, length 4, label:21 tunnel identifier: 0000 0000
      rx pathid: 0, tx pathid: 0x0
      net: 0x7F952DEC5DE8, path: 0x7F9565ECA760, pathext: 0x7F9565EDAD08
      flags: net: 0x0, path: 0x4000028000003, pathext: 0x81
      attribute: 0x7F952DED5EF8, ref: 2
      Updated on Oct 6 2022 13:59:01 UTC

This capture was taken at Gi1 on PE1. P4 already preformed PHP, removing the transport label and leaving only the service label (22).

To explore further on your own, you could try shutting down the AC of one PE to see what happens when it withdraws the MAC. Do other PEs immediately remove the MAC from their bridge tables? What mechanism causes the PEs to remove the MAC?

EVPN Instance ID and ethernet tag ID

The instance ID is not actually signaled in the BGP Update. We configured instance 1 on all PEs, but this is not signaled in the NLRI. The instance ID only needs to match between PEs because it is used in the auto-generated RT value. The RT is <ASN>:<instance number>. Therefore you are free to change the EVPN instance number on each PE, but you would need to manually specify the RT values so that they match between all PEs.

The ethernet tag ID is a separate field which is used to signal the VLAN. In this lab we used a VLAN-based EVPN instance, in which a single VLAN maps to a single BD which maps to a single EVI. So the ethernet tag ID field is essentially unused. You can see the ethernet tag ID set to zero in all updates.

Also in VLAN-bundle, the ethernet tag ID is not used. There is still a one-to-one mapping of BD to EVI, but multiple EFPs can participate in a single BD. The downside to VLAN-bundle is that a single MAC address can only be present on a per-BD basis. If an EFP with VLAN 10 and an EFP with VLAN 20 learn the same MAC address, communication will be broken, because there is no differentiation between these vlans due to the fact that they both share a single BD.

The ethernet tag ID is used in VLAN-aware. This is because multiple BDs map to a single EVI, so you need to differentiate between each BD. When you add the vlan-aware EVI to the BD, it forces you to choose an ethernet tag ID.

PE1(config)#l2vpn evpn instance 105 vlan-aware 
PE1(config-evpn-evi)#bridge-domain 105
PE1(config-bdomain)#member evpn-instance 105 ?
  ethernet-tag  Ethernet Tag

Conclusion

EVPN is an elegant solution to providing VPLS services without targeted LDP. EVPN E-LAN uses type 2 and type 3 routes to signal MAC addresses and create BUM replication lists.

The benefit of EVPN is that a single PE only learns MAC addresses locally. Remote MAC addresses are signaled via the control plane, they are not learned via the data plane. You saw that the remote MAC addresses are present in the BD table as a static entry, which is created due to received type 2 routes.

We also saw the difference between vlan-based, vlan-bundle, and vlan-aware EVPN instances.

VLAN-based = one EFP → one BD → one EVI
VLAN-bundle = multiple EFPs → one BD → one EVI
VLAN-aware = multiple EFPs → multiple BDs (one to one basis of VLAN to BD) → one EVI
- Requires an ethernet tag ID

This concludes the series on EVPN. I hope that this has been useful!

Lab

Flooding Process

EVPN Instance ID and ethernet tag ID

Conclusion

Further Reading