RTBH

In this article, we will configure RTBH (remote triggered black hole). This feature allows you to trigger the black holing of a prefix from a single router. Upon triggering this black hole at this single point, all PEs will immediately black hole traffic for the prefix.

We’ll see how to trigger this based on a particular destination or source. This is called destination-based RTBH and source-based RTBH.

Lab

We will run OSPF and LDP on all service provider routers, as well as BGP. P4 is the RR. All routers run the internet table in the default VRF. This will include the 100.1.1.X/30 prefixes, as well as the customer’s prefix 100.100.100.0/24 behind CE1. All routers are CSR1000v, except for P4 which is XRv.

Here is the startup config on all routers:

#CE1
int Gi1
 ip address 100.1.1.2 255.255.255.252
 no shut
!
ip route 0.0.0.0 0.0.0.0 100.1.1.1
!
int lo0
 ip address 100.100.100.1 255.255.255.0

#CE2
int Gi1
 ip address 100.1.1.6 255.255.255.252
 no shut
!
ip route 0.0.0.0 0.0.0.0 100.1.1.5

#CE3
int Gi1
 ip address 100.1.1.10 255.255.255.252
 no shut
!
ip route 0.0.0.0 0.0.0.0 100.1.1.9

#PE1
int Gi1
 ip address 100.1.1.1 255.255.255.252
 no shut
!
int Gi2
 ip address 10.1.4.1 255.255.255.0
 no shut
 ip ospf network point-to-point
 mpls ip
!
int lo0
 ip address 10.1.1.1 255.255.255.255
!
router ospf 1
 network 10.0.0.0 0.255.255.255 area 0
!
ip route 100.100.100.0 255.255.255.0 100.1.1.2
!
router bgp 65000
 neighbor 10.4.4.4 remote-as 65000
 neighbor 10.4.4.4 update-source lo0
 neighbor 10.4.4.4 next-hop-self
 network 100.1.1.0 mask 255.255.255.252
 redistribute static

#PE2
int Gi1
 ip address 100.1.1.5 255.255.255.252
 no shut
!
int Gi2
 ip address 10.2.4.2 255.255.255.0
 no shut
 ip ospf network point-to-point
 mpls ip
!
int lo0
 ip address 10.2.2.2 255.255.255.255
!
router ospf 1
 network 10.0.0.0 0.255.255.255 area 0
!
router bgp 65000
 neighbor 10.4.4.4 remote-as 65000
 neighbor 10.4.4.4 update-source lo0
 neighbor 10.4.4.4 next-hop-self
 network 100.1.1.4 mask 255.255.255.252

#PE3
int Gi1
 ip address 100.1.1.9 255.255.255.252
 no shut
!
int Gi2
 ip address 10.3.4.3 255.255.255.0
 no shut
 ip ospf network point-to-point
 mpls ip
!
int lo0
 ip address 10.3.3.3 255.255.255.255
!
router ospf 1
 network 10.0.0.0 0.255.255.255 area 0
!
router bgp 65000
 neighbor 10.4.4.4 remote-as 65000
 neighbor 10.4.4.4 update-source lo0
 neighbor 10.4.4.4 next-hop-self
 network 100.1.1.8 mask 255.255.255.252

#P4
int Gi0/0/0/1
 ip address 10.1.4.4/24
 no shut
!
int Gi0/0/0/2
 ip address 10.2.4.4/24
 no shut
!
int Gi0/0/0/3
 ip address 10.3.4.4/24
 no shut
!
int lo0
 ip address 10.4.4.4/32
!
router ospf 1
 area 0
  int Gi0/0/0/1
   network point-to-point
  int Gi0/0/0/2
   network point-to-point
  int Gi0/0/0/3
   network point-to-point
  int lo0
!
mpls ldp
 int Gi0/0/0/1
 int Gi0/0/0/2
 int Gi0/0/0/3
!
router bgp 65000
 address-family ipv4 unicast
 !
 neighbor-group IBGP
  remote-as 65000
  update-source lo0
  address-family ipv4 unicast
    route-reflector-client
 !
 neighbor 10.1.1.1 use neighbor-group IBGP
 neighbor 10.2.2.2 use neighbor-group IBGP
 neighbor 10.3.3.3 use neighbor-group IBGP

Destination-Based RTBH

You get a call that 100.100.100.1 is currently undergoing a DDoS attack. The customer asks if you can drop all traffic destined to that IP.

You could simply create a null0 route on PE1, but that still allows the traffic to traverse your MPLS network. It would be better if all PEs on the internet edge could block the traffic so it is not even routing through your internal network.

To solve this we’ll use RTBH. The RR, P4, will act as the remote trigger. First we’ll define a common destination IP that all your internal routers will drop traffic to.

#PE1, PE2, PE3
ip route 192.0.2.0 255.255.255.255 null0

P4 will advertise all traffic that you want to drop with a next-hop of 192.0.2.0. Immediately once P4 advertises this, all the PEs will drop traffic destined to that prefix at the edge, based on the local static route pointing to null0. First we’ll configure a route-map that will set the next-hop to 192.0.2.0, and also set LP high so that all PEs will select this route that P4 advertises as the best path. We will only match routes with a tag of 666 so that we can still redistribute regular static routes into BGP.

#P4
route-policy RTBH
 if tag eq 666 then
  set next-hop 192.0.2.0
  set local-preference 500

Now we’ll simply create a static route for the IP under attack, 100.100.100.1, and tag it with 666. We also need to redistribute statics into BGP with the RTBH route-policy.

router static address-family ipv4 unicast
 100.100.100.1/32 null0 tag 666
!
router bgp 65000
 address-family ipv4 unicast
  redistribute static route-policy RTBH

Take a look at P4’s BGP table and verify that 100.100.100.1/32 is now being injected into BGP.

RP/0/0/CPU0:P4#show bgp ipv4 uni | begin Network
Fri Jul 22 20:45:48.604 UTC
   Network            Next Hop            Metric LocPrf Weight Path
*>i100.1.1.0/30       10.1.1.1                 0    100      0 i
*>i100.1.1.4/30       10.2.2.2                 0    100      0 i
*>i100.1.1.8/30       10.3.3.3                 0    100      0 i
*>i100.100.100.0/24   10.1.1.1                 0    100      0 ?
*> 100.100.100.1/32   192.0.2.0                0    500  32768 ?
*>i192.0.2.0/32       10.1.1.1                 0    100      0 ?

All PEs learn this route, and since 192.0.2.0 has a null0 next hop, all PEs will drop traffic to this destination.

PE3#show ip cef 100.100.100.1 det
100.100.100.1/32, epoch 2, flags [rib only nolabel, rib defined all labels]
  recursive via 192.0.2.0
    attached to Null0

When CE3 tries to ping 100.100.100.1, it fails. The beauty of this design is that PE3 is dropping the traffic directly at the edge. If you only configured the null0 route at PE1, the DDoS traffic would be routed through the provider’s internal network.

CE3#ping 100.100.100.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 100.100.100.1, timeout is 2 seconds:
U.U.U
Success rate is 0 percent (0/5)
  
CE3#traceroute 100.100.100.1
Type escape sequence to abort.
Tracing the route to 100.100.100.1
VRF info: (vrf in name/id, vrf out name/id)
  1 100.1.1.9 2 msec 3 msec 2 msec
  2 100.1.1.9 !H  *  !H

That’s really all there is to it. What we just configured is destination-based RTBH. We are blocking all traffic destined to 100.100.100.1/32.

Source-Based RTBH

The customer calls and says that they found that 100.1.1.6 is the bad actor. They ask if you can remove the destination based filter, and simply block traffic sourced from 100.1.1.6. (Note that in our lab 100.1.1.6 is actually the provider’s customer, but we’ll ignore this fact. If this was literally your own customer, you would simply shutdown the port facing 100.1.1.6).

To configure source-based RTBH, we use the same mechanism as destination-based RTBH but with uRPF at the edge ports. So P4 will still statically route the source prefix to null0, and redistribute this into BGP with the 192.0.2.0 next-hop. But since this is source-based, we need some mechanism to identify the source traffic. With destination-based, we simply rely on destination-based IP forwarding to do the black holing.

We can either configure uRPF on the edge interfaces in strict mode, or loose mode. In strict mode, the packets from the customers must have a source IP that matches a route in the RIB which has the corresponding interface as the outgoing interface. Simply put, the reverse path towards that source IP must take the same interface if you were to route towards that source IP as a destination. In loose mode, there just needs to be some route in the RIB for that prefix. It doesn’t have to have the incoming interface as the outgoing interface for the route. However, a null0 route does not pass the loose mode check. We’ll use this to our benefit to implement source-based RTBH.

#PE1, 2, 3
int gi1
 ip verify unicast source reachable-via any
  • The any keyword implements loose mode uRPF. The rx keyword would implement strict mode uRPF.

On P4 we’ll simply remove the 100.100.100.1/32 static route and add 100.1.1.6/32

router static address-family ipv4 unicast
 no 100.100.100.1/32
 100.1.1.6/32 null0 tag 666

Now CE3 (100.1.1.10) can ping 100.100.100.1, but CE2 (100.1.1.6) cannot:

CE3#ping 100.100.100.1      
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 100.100.100.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 6/8/19 ms

CE2#ping 100.100.100.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 100.100.100.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

Notice the difference in the failure on CE2 compared to the failure on CE3 with destination-based RTBH. In destination-based, the PE sends Unreachables. In source-based, the router simply drops the traffic silently.

PE2#show ip cef 100.1.1.6
100.1.1.6/32
  nexthop 192.0.2.0 Null0

PE2, upon receiving traffic from 100.1.1.6, checks the RIB and sees that the IP is reachable via a null0 route. This fails the loose mode uRPF check configured on the interface, so the traffic is simply dropped.

Conclusion

RTBH is quite an elegant solution to drop traffic at the edge, whether you want to drop traffic based on source IP or destination IP. The pre-work involves configuring a static route for an identical destination IP to null0 on every PE, and and redistributing static routes into BGP on the router you want to use to trigger the black hole. Once that is in place, triggering the black hole is done on a single router (in our lab, P4). You simply create a static route for the prefix you want to black hole, and this is automatically redistributed into BGP and the iBGP route is reflected to all PEs.

Last updated