12.04.2013 13:05

Lessons from the Stophaus/CloudFlare/Spamhaus DDoS for ISPs

No, the Internet is not breaking down, we did not have a doomsday scenario over the last week.

We did have an interesting situation, there were some disruption in some parts of the Internet, and there were a good number of overtime hours being put in to mitigate these disruptions.

Here are some links:

• http://www.bbc.co.uk/news/technology-21954636
• https://greenhost.nl/2013/03/21/spam-not-spam-tracking-hijacked-spamhaus-ip/
• https://www.nytimes.com/2013/03/27/technology/internet/online-dispute-becomes-internet-snarling-attack.html?pagewanted=all
• http://rt.com/news/spamhaus-threat-cyberbunker-ddos-attack-956/
• http://blog.cloudflare.com/the-ddos-that-almost-broke-the-internet
• http://gizmodo.com/5992652
• http://cluepon.net/ras/gizmodo

There is some discussion in international network security forums on what lessons should be taken from those events. This is all pretty preliminary, but boils down to the following:

Control Plane Protection

After the attackers failed to bring down the anycasted webservers of CloudFlare (that part is described at the CloudFlare blog), they aimed at the network equipment supporting those nodes.

For one view on how to protect your own routers from attacks, see https://www.box.com/s/osk4po8ietn1zrjjmn8b

There are also some lessons for the operators of Internet Exchange Points.

Network Hygiene

The attacks were mainly done using amplification via DNS servers. In order to reduce these attacks, the following three points need to be taken care of:

1. Implement BCP38. The attackers need to send out forged packets; restricting their ability to do so from a hacked box inside *your* network helps.
2. Recursive nameservers should not be open to the word. See RFC5358. There are a few projects starting up which scan the Internet for such open recursors in order to get them all fixed. One is http://openresolverproject.org. Warning: the data-quality from that service is not optimal yet.

   If you want to scan your own netblock for open recursors, have a look at Aaron's software.

3. Authoritative nameservers can also be abused as traffic amplifiers. There are patches out which implement rate-limiting for the common implementations. See e.g. http://www.redbarn.org/dns/ratelimits

Forgery tracking capability

In the case when someone is abusing ISP customers as amplificators to attack a.b.c.d, then he is forging the packets claiming to originate from a.b.c.d. As we need to track down the forgers, it is necessary that all networks are capable of investigating where these forgeries originate from (inside your own network, from a peering, or from your upstream).

This needs to be a fast process.

It does not matter how this is achieved; the most obvious tool to do this is a decent netflow coverage of your network. For smaller networks, this can be done with open source tools like nfsen.

Author: Otmar Lendl