One of the more significant DDoS-related threats enjoying a renaissance in recent times is the UDP-based attack. In contrast to TCP – the reliable bitstream transfer protocol which dominates the Internet and underpins most web applications – UDP has quite different characteristics. It’s generally best suited to lightweight request-response type applications that require a simple record-delimited method to talk to a peer, with or without data integrity checks. UDP is also well suited to real-time media applications where retransmission of audio/video codec samples is too costly on latency and jitter.
The reason that UDP is attractive for DDoS purposes though, is it’s lightweight session setup. Compared with TCP where both endpoints must synchronise state – involving a randomly generated sequence number – before sharing data, UDP peers require no such invitation. Applications can simply send data, whether its expected or not, and the receiver has to deal with it. Indeed malicious applications can send large amounts of data to a receiver, faking their source address to make tracing difficult, and causing excessive resource consumption to the receiving victim.
Spoofed Source Address Attack
In more complex attacks, malicious actors can make use of imbalanced UDP protocols – ones where a small request can result in a large response – and, by again faking the source address to that of their victim, have an innocent third-party, or more likely many innocent third-parties, create a burden of traffic through reflection for the victim.
Reflected Attack
Besides UDP’s lack of requirement for initial connection and state synchronisation, the one thing that makes these two attack vectors possible is the ability to spoof packets, and specifically, spoof the IP address that one is communicating from. To a layman the ability to lie about one’s identity (or routing indicator, at least) might seem a bit of a schoolboy error on the parts of our Internet forefathers, but it stems from a time when the Internet trust model was quite different to what it is today.
The Internet community have strived to remove this particular avenue of exploit: the IETF published RFC 2827 – now established as BCP 38 – in 2000, and network equipment manufacturers have since deployed features to implement BCP 38 in modern routing gear. In summary, BCP 38 tell us that when a router forwards a packet, rather than simply routing based on the destination address, it should also check the source address and consider whether traffic routed to that source address would be sent via the interface that the packet was received on. More details here.
The logic is quite simple, but it enforces a symmetrical routing model which when applied to the edge of a network should result in an Internet topology where a lot more assurances can be made about the source address.
But lots of service providers have been slow and lethargic to adopt. It’s not totally surprising because the mitigating measures aren’t exactly without consequence. Because of the symmetrical routing model, the measures must really only be applied at the network edge, where asymmetric routing isn’t an accepted norm. Deep inside BGP-running ISP cores, dynamic routing protocols – the staple of a highly-available Internet – mean that asymmetry is much more common-place and deploying BCP 38 mechanisms on an interface with dynamic routing information can effectively cause a black-out as legitimate traffic is blocked.
Originally, many people thought that wide BCP 38 deployment was needed on the access networks that service consumer and residential broadband access, since these are the places where malware can fester in older PCs with out-of-date software. More modern thinking has made us realise that the almost universal proliferation of source-based NAT (network address translation) – a method for preserving IP address usage – is effectively doing a good BCP 38-like job for most broadband interfaces.
That leaves, then, the co-location and hosted Internet services. Because these sites usually publish content, they’re rarely subjected to source-based NAT, and malware infected servers in these areas could quite easily spoof addresses and contribute to the DDoS problem.
At Interoute, with our investment in Virtual Data Centre technology closely fused with our backbone IP network, this has been a particular concern for us. The advent of utility computing has brought with it commonplace try-before-you-buy trial schemes for cloud compute that are ripe for abuse by the criminal classes of Internet society.
Like many, we’ve not been squeaky-clean on our deployment of BCP 38 to prevent source address spoofing, and while our Juniper MX platforms support RPF – Reverse Path Filtering – configuration to achieve our objective, the challenge for our NOC operators is twofold: balancing the significant labour-intensive reconfiguration activity with other more overt activities, and ensuring against any misconfiguration of RPF and its painful consequences.
What we’ve really needed is an assisting tool for our Juniper MX-960 based backbone that would quickly audit the interfaces on a device and identify those that can accept BCP 38 filtering without consequence, and optionally go and configure them thus. For safety’s sake, we’d also want a way that we could quickly de-configure the feature in order to respond to any resulting code-brown moments.
Enter JUNOS SLAX op scripts, and rpf-tool – a tool that I’ve been trialling recently for its suitability in this role. Written in Juniper’s rather polarising hybrid XML / Perl / C language SLAX which I’ve written about before, rpf-tool aims to do exactly what’s needed.
Primarily, we can run the tool to get a list of interfaces that should and shouldn’t have RPF deployed. On my sample virtual Firefly, for instance, that looks a bit like the following list of interfaces, where the plus (+) sign indicates that we’re good to go (be sure to scroll wide).
adamc@VirtualSRX> op rpf-tool Interface RPF analysis/configuration tool Flags: + eligible, - ineligible, * running, ! excluded Interface Instance Description Address/Config ! ge-0/0/0.0 CST 10M FOOB1/CPE/12345 + ge-0/0/1.1 CST 10M FOOB1/CPE/10001 inet 10.0.1.1/24 + ge-0/0/1.2 CST 10M FOOB1/CPE/10002 inet6 fd00:0001::1/64 + ge-0/0/1.3 CST 10M FOOB1/CPE/10003 inet 10.0.3.1/24 inet6 fd00:0003::1/64 + ge-0/0/1.4 CST 10M FOOB1/CPE/10004 inet 10.0.4.1/24 inet 10.1.4.1/24 inet 10.2.4.1/24 inet6 fd00:0004::1/64 inet6 fd00:0104::1/64 inet6 fd00:0204::1/64 + ge-0/0/1.5 VPN-1 CST 10M FOOB1/CPE/10005 inet 10.0.5.1/24 + ge-0/0/1.6 VPN-1 CST 10M FOOB1/CPE/10006 inet6 fd00:0006::1/64 + ge-0/0/1.7 VPN-1 CST 10M FOOB1/CPE/10007 inet 10.0.7.1/24 inet6 fd00:0007::1/64 + ge-0/0/1.8 VPN-1 CST 10M FOOB1/CPE/10008 inet 10.0.8.1/24 inet 10.1.8.1/24 inet 10.2.8.1/24 inet6 fd00:0008::1/64 inet6 fd00:0108::1/64 inet6 fd00:0208::1/64
Note that the tool looks at interfaces regardless of whether they are related to the global table or a VRF routing instance. This is significant because by the time we get to a global table interface, we might already have a dynamic protocol involved.
Moving further down the test list of interfaces, we spot OSPF running, which disqualifies these interfaces. If we have OSPF running on an interface, we don’t have a guarantee that we’re going to symmetrically route back to the addresses we see sourced from the interface.
- ge-0/0/1.9 CST 10M FOOB1/CPE/10009 inet 10.0.9.1/24 OSPF - ge-0/0/1.10 CST 10M FOOB1/CPE/10010 inet6 fd00:000a::1/64 OSPF - ge-0/0/1.11 CST 10M FOOB1/CPE/10011 inet 10.0.11.1/24 inet6 fd00:000b::1/64 OSPF - ge-0/0/1.12 CST 10M FOOB1/CPE/10012 inet 10.0.12.1/24 inet 10.1.12.1/24 inet 10.2.12.1/24 inet6 fd00:000c::1/64 inet6 fd00:010c::1/64 inet6 fd00:020c::1/64 OSPF - ge-0/0/1.13 VPN-2 CST 10M FOOB1/CPE/10013 inet 10.0.13.1/24 OSPF - ge-0/0/1.14 VPN-2 CST 10M FOOB1/CPE/10014 inet6 fd00:000e::1/64 OSPF - ge-0/0/1.15 VPN-2 CST 10M FOOB1/CPE/10015 inet 10.0.15.1/24 inet6 fd00:000f::1/64 OSPF - ge-0/0/1.16 VPN-2 CST 10M FOOB1/CPE/10016 inet 10.0.16.1/24 inet 10.1.16.1/24 inet 10.2.16.1/24 inet6 fd00:0010::1/64 inet6 fd00:0110::1/64 inet6 fd00:0210::1/64 OSPF
A bit further down the list, we see the same logic applied when we have a BGP relationship established on an interface. While the BGP configuration on JUNOS doesn’t allow us to directly determine the interface involved, the JUNOS SLAX facilities allow us to take the peer address and qualify it with the networks on the connected interfaces.
- ge-0/0/1.17 CST 10M FOOB1/CPE/10017 inet 10.0.17.1/24 BGP + ge-0/0/1.18 CST 10M FOOB1/CPE/10018 inet6 fd00:0012::1/64 - ge-0/0/1.19 CST 10M FOOB1/CPE/10019 inet 10.0.19.1/24 inet6 fd00:0013::1/64 BGP - ge-0/0/1.20 CST 10M FOOB1/CPE/10020 inet 10.0.20.1/24 inet 10.1.20.1/24 inet 10.2.20.1/24 inet6 fd00:0014::1/64 inet6 fd00:0114::1/64 inet6 fd00:0214::1/64 BGP - ge-0/0/1.21 VPN-3 CST 10M FOOB1/CPE/10021 inet 10.0.21.1/24 BGP + ge-0/0/1.22 VPN-3 CST 10M FOOB1/CPE/10022 inet6 fd00:0016::1/64 - ge-0/0/1.23 VPN-3 CST 10M FOOB1/CPE/10023 inet 10.0.23.1/24 inet6 fd00:0017::1/64 BGP - ge-0/0/1.24 VPN-3 CST 10M FOOB1/CPE/10024 inet 10.0.24.1/24 inet 10.1.24.1/24 inet 10.2.24.1/24 inet6 fd00:0018::1/64 inet6 fd00:0118::1/64 inet6 fd00:0218::1/64 BGP
And finally, we get the all important exclusion feature. We may not be able to rely on the tool to be able to determine every single interface that should be excluded from RPF, but we can at least give the operator a hook or a flag in order to warn the tool not to try to configure RPF on an interface. Built-in to the tool, we have several conditions that can force an exclusion:
- The interface has a special prefix, eg. Lo0 or fxp0
- The interface is running DHCP
- The interface is unnumbered
- The interface description includes a special token { no-rpf }
The exclamation mark in the output indicates the exclusion.
adamc@VirtualSRX> op rpf-tool Interface RPF analysis/configuration tool Flags: + eligible, - ineligible, * running, ! excluded Interface Instance Description Address/Config ! ge-0/0/0.0 CST 10M FOOB1/CPE/12345 ! lo0.0 inet 127.0.0.1/32 inet 10.255.0.1/32 ! fxp0.0 (MGMT)
adamc@VirtualSRX> show configuration interfaces ge-0/0/0
description Main;
unit 0 {
description "CST 10M FOOB1/CPE/12345 X {no-rpf, if-acct} // Foobar Saunders and his Unix Entendres";
family inet {
dhcp;
}
}
Finally, after we’ve checked the list, we can either give rpf-tool the thumbs up to configure RPF on all eligible interfaces, or we can simply configure RPF on an interface by interface basis.
adamc@VirtualSRX> op rpf-tool mode apply Applying RPF inet configuration to interface ge-0/0/1.1 Applying RPF inet6 configuration to interface ge-0/0/1.2 Applying RPF inet configuration to interface ge-0/0/1.3 Applying RPF inet6 configuration to interface ge-0/0/1.3 Applying RPF inet configuration to interface ge-0/0/1.4 Applying RPF inet6 configuration to interface ge-0/0/1.4 Applying RPF inet configuration to interface ge-0/0/1.5 Applying RPF inet6 configuration to interface ge-0/0/1.6 Applying RPF inet configuration to interface ge-0/0/1.7 Applying RPF inet6 configuration to interface ge-0/0/1.7 Applying RPF inet configuration to interface ge-0/0/1.8 Applying RPF inet6 configuration to interface ge-0/0/1.8 Applying RPF inet6 configuration to interface ge-0/0/1.18 Applying RPF inet6 configuration to interface ge-0/0/1.22 Apply RPF to all interfaces: 10 interface(s) affected: 6 inet, 8 inet6
adamc@VirtualSRX> show system commit
0 2014-03-01 18:36:27 UTC by adamc via junoscript
Apply RPF to all interfaces: 10 interface(s) affected: 6 inet, 8 inet6
It’s useful to note that if we try to apply RPF on an interface that the tool thinks is inappropriate, it won’t do it. In this case, ge-0/0/0.0 has got the magical description tag (it’s the management interface to my Firefly VM and I didn’t want to take any chances).
adamc@VirtualSRX> op rpf-tool mode apply interface ge-0/0/0.0 Interface ge-0/0/0.0 specifically excluded by local-policy-exclusions Apply RPF to interface: ge-0/0/0.0: 0 inet, 0 inet6 No affected interfaces: configuration not applied
And of course we need the emergency, “No! Turn it off!” switch, so we can quickly reverse any changes and problems caused by deploying RPF.
adamc@VirtualSRX> op rpf-tool mode remove Removing RPF inet configuration from interface ge-0/0/1.1 Removing RPF inet6 configuration from interface ge-0/0/1.2 Removing RPF inet configuration from interface ge-0/0/1.3 Removing RPF inet6 configuration from interface ge-0/0/1.3 Removing RPF inet configuration from interface ge-0/0/1.4 Removing RPF inet6 configuration from interface ge-0/0/1.4 Removing RPF inet configuration from interface ge-0/0/1.5 Removing RPF inet6 configuration from interface ge-0/0/1.6 Removing RPF inet configuration from interface ge-0/0/1.7 Removing RPF inet6 configuration from interface ge-0/0/1.7 Removing RPF inet configuration from interface ge-0/0/1.8 Removing RPF inet6 configuration from interface ge-0/0/1.8 Removing RPF inet6 configuration from interface ge-0/0/1.18 Removing RPF inet6 configuration from interface ge-0/0/1.22 Remove RPF from all interfaces: 10 interface(s) affected: 6 inet, 8 inet6
At this time, rpf-tool is still under testing and investigation at Interoute, but the whole Internet ISP community can benefit by the reduction in DDoS and address spoofing, so I encourage external experimentation and feedback. The code is available here for those willing to do so. All of the usual disclaimers apply, and explicitly I provide no warranty.
Update: For those of you reading this and looking to deploy Juniper’s RPF functionality on your MX-based network in order to implement BCP-38, it’s worth mentioning PR 873709: a particularly nasty defect that is triggered when you remove RPF from an interface, and another seemingly unrelated interface with RPF configured then stops forwarding. The situation can be remediated by removing RPF from the affected interface and re-adding it, and perhaps rpf-tool is useful in that regard. Most likely to stumble onto this on a box with lots of FPCs and lots of logical interfaces. Juniper also document an interesting mitigating step which is to reboot the REs with a flag in the boot loader that prevents the situation of overlapping squashed interface indices:
debug.pfe_local_idx_override=1
Obviously, you’re on your own with that one!
I find it interesting how BCP 38 aims to improve source address validation.