BGP Load Sharing (Multipath)

By default, a router will never use multiple BGP routes to the same destination. A router always picks a single best BGP path. If many path attributes are the same between multiple paths, “random” criteria such as the lowest RID and then lowest neighbor IP address are used to pick a single path.

When enabling multipath, you have three options. You can enable multiple for iBGP routes, eBGP routes, or both iBGP/eBGP routes. If you enable mulitple for both iBGP/eBGP routes, this is called “eiBGP.” It considers both iBGP and eBGP paths to be equal. (Otherwise eBGP paths would always beat iBGP paths). However, you can also separately enable iBGP multipath and eBGP multipath, which will only consider like-paths for multipathing (iBGP and eBGP paths are not considered equal, only multiple iBGP paths can be multipath or multiple eBGP paths can be multipath).

When using multipath, it is important to understand that a single path is still selected as the “best path.” However, multiple paths can be installed into the RIB and used for load sharing. But the single “best path” is what will be advertised to other neighbors. The process of advertising only the best path to a neighbor has not changed. In order to change this, you would use the add path feature, which we are not looking at in this article.

In order for paths to be considered for multipath, the following must match:

• Weight

• Local pref

• AS-Path length

• AS-Path content

  • Paths must have the exact same AS path

• Origin

• MED

• IGP metric to the next-hop

  • This means that the next-hop for each path can be different as long as the metrics to the next-hops are the same

To enable multipath you use the following commands:

router bgp 65000
 maximum-path <number>        ! Sets the maximum eBGP paths that can be used
 maximum-path ibgp <number>   ! Sets the maximum iBGP paths that can be used
 maximum-path eibgp <number>  ! Considers iBGP and eBGP paths as equals

Examples

eBGP multipath

On R1, I have enabled eBGP multipath, but the paths have a differing neighboring AS

router bgp 65001
 neighbor 10.1.2.2 remote-as 65002
 neighbor 10.1.3.3 remote-as 65003
 maximum-paths 2

R1#show bgp ipv4 uni | be Network
     Network          Next Hop            Metric LocPrf Weight Path
 *    100.100.100.100/32
                      10.1.3.3                               0 65003 65004 i
 *>                   10.1.2.2                               0 65002 65004 i

• The path from 10.1.3.3 is not considered for multipathing because it is learned from a different AS (65003) compared to the current best path

If we change the topology and learn the prefix from two different neighbors from the same neighboring AS, the route is still not installed, because the AS path content is not exactly the same. Interestingly, Cisco guides will say that only the neighboring AS needs to be the same, but this does not appear to be the case. It seems this may have been the case on older IOS versions.

router bgp 65001
 neighbor 10.1.5.5 remote-as 65010
 neighbor 10.1.6.6 remote-as 65010
 maximum-paths 2

R1#show bgp ipv4 uni | be Network
     Network          Next Hop            Metric LocPrf Weight Path
 *    100.100.100.100/32
                      10.1.6.6                               0 65010  65004 i
 *>                   10.1.5.5                               0 65010 65002 65004 i

We can tell R1 to relax the AS-Path content check, and only ensure that the AS-Path length is identical:

router bgp 65001
 
 neighbor 10.1.5.5 remote-as 65010
 neighbor 10.1.6.6 remote-as 65010
 maximum-paths 2

R1#show bgp ipv4 uni | be Network 
     Network          Next Hop            Metric LocPrf Weight Path
 *m   100.100.100.100/32
                      10.1.6.6                               0 65010 65003 65004 i
 *>                   10.1.5.5                               0 65010 65002 65004 i

R1#show ip route bgp | beg Gateway
Gateway of last resort is not set

      100.0.0.0/32 is subnetted, 1 subnets
B        100.100.100.100 [20/0] via 10.1.6.6, 00:00:26
                         [20/0] via 10.1.5.5, 00:00:26

• The m means the path has been selected for multipath. The route from 10.1.5.5 is still the bestpath, and is the path that would be advertised to other BGP neighbors.

iBGP multipath

I’ll change the topology once more so we can verify iBGP multipath. R1’s two neighbors are now iBGP neighbors.

router bgp 65010
 neighbor 10.1.5.5 remote-as 65010
 neighbor 10.1.6.6 remote-as 65010
 maximum-paths ibgp 2

R1#show bgp ipv4 uni | be Network
     Network          Next Hop            Metric LocPrf Weight Path
 *>i  100.100.100.100/32
                      10.1.5.5                      100      0 65020 65002 65004 i
 * i                  10.1.6.6                      100      0 65020 65003 65004 i

The paths have the same neighboring AS but not the same exact AS Path. We can relax the AS-Path content check again to use both paths for multipath.

router bgp 65010
 bgp bestpath as-path multipath-relax
 neighbor 10.1.5.5 remote-as 65010
 neighbor 10.1.6.6 remote-as 65010
 maximum-paths ibgp 2

R1#show bgp ipv4 uni | be Network
     Network          Next Hop            Metric LocPrf Weight Path
 *>i  100.100.100.100/32
                      10.1.5.5                      100      0 65020 65002 65004 i
 *mi                  10.1.6.6                      100      0 65020 65003 65004 i

If I change peering to use loopbacks, and change the IGP metric for R6 to be higher, the path from R6 can no longer be used for multipath.

router bgp 65010
 bgp bestpath as-path multipath-relax
 neighbor 5.5.5.5 remote-as 65010
 neighbor 5.5.5.5 update-source Loopback0
 neighbor 6.6.6.6 remote-as 65010
 neighbor 6.6.6.6 update-source Loopback0
 maximum-paths ibgp 2

! Before making any changes, both paths are installed into the RIB for multipath
R1#show bgp ipv4 uni | be Network
     Network          Next Hop            Metric LocPrf Weight Path
 *mi  100.100.100.100/32
                      6.6.6.6                       100      0 65020 65003 65004 i
 *>i                  5.5.5.5                       100      0 65020 65002 65004 i


R1#conf t
R1(config)#int gi2
R1(config-if)#ip ospf cost 10
R1(config-if)#end

! The path from R6 is no longer valid for multipath consideration
R1#show bgp ipv4 uni | be Network
     Network          Next Hop            Metric LocPrf Weight Path
 * i  100.100.100.100/32
                      6.6.6.6                       100      0 65020 65003 65004 i
 *>i                  5.5.5.5                       100      0 65020 65002 65004 i

If we want to disable the IGP metric check, we can use the following command:

router bgp 65010
 bgp bestpath as-path multipath-relax
 
 neighbor 5.5.5.5 remote-as 65010
 neighbor 5.5.5.5 update-source Loopback0
 neighbor 6.6.6.6 remote-as 65010
 neighbor 6.6.6.6 update-source Loopback0
 maximum-paths ibgp 2

R1#show bgp ipv4 uni | be Network
     Network          Next Hop            Metric LocPrf Weight Path
 *mi  100.100.100.100/32
                      6.6.6.6                       100      0 65020 65003 65004 i
 *>i                  5.5.5.5                       100      0 65020 65002 65004 i

Be aware that igp-metric ignore is slightly different than as-path multipath-relax. Disabling the IGP metric check affects all best path decisions, not just multipath consideration. The content of the AS Path is never a consideration for bestpath (only the AS Path length is), so the as-path multipath-relax command only affects multipath consideration.

On IOS-XR you can disable the IGP metric check for only multipath consideration using the following command:

router bgp 65000
 address-family ipv4 unicast
  maximum-paths ibgp 2 unequal-cost

eiBGP multipath

Once you understand iBGP and eBGP multipath, eiBGP multipath is fairly straight-forward. The only difference is that with eiBGP multipath, both iBGP and eBGP paths can be used for ECMP to a destination.

R1 now has one eBGP peer and one iBGP peer. To enable load sharing between both paths, we use the maximum-paths eibgp command.

R1(config)#router bgp 65010
R1(config-router)#maximum-paths eibgp 2 
%BGP: This may cause traffic loop if not used properly (command accepted)
R1(config-router)#end
*Nov 29 15:28:36.651: %BGP-4-MULTIPATH_LOOP: This may cause traffic loop if not used properly (command accepted).

R1#show run | sec bgp
router bgp 65010
 bgp bestpath as-path multipath-relax
 bgp bestpath igp-metric ignore
 neighbor 5.5.5.5 remote-as 65010
 neighbor 5.5.5.5 update-source Loopback0
 neighbor 10.1.6.6 remote-as 65006
 maximum-paths eibgp 2
R1#show bgp ipv4 uni | be Network
     Network          Next Hop            Metric LocPrf Weight Path
 *>   100.100.100.100/32
                      10.1.6.6                               0 65006 65003 65004 i
 *mi                  5.5.5.5                       100      0 65020 65002 65004 i

The router warns us that when using eibgp multipath, loops are possible. If two iBGP peers use eiBGP and have both an eBGP route and iBGP route pointing to each other, the peers could loop traffic between each other (if both routers select the iBGP path at the right time).

• R1 has two ECMP routes via R3 and R2

• R2 has two ECMP routes via R4 and R1

• R1 and R2 could loop traffic between each other

For this reason you must be very careful when using eiBGP multipathing.