As mentioned briefly in a post last Friday, between 5 to 10 percent of all broadband subscribers in the Netherlands had their machines recruited into a botnet at some point in 2009. This is a conservative estimate, based on the unique IP sources present in three distinct datasets of malicious network hosts: a large spam trap, the DShield distributed intrusion detection system, and Conficker sinkholes. Figure 1 shows the breakdown of the 1.1 million unique IP addresses from the Netherlands split among the Dutch ISPs. More than 80% of these addresses are located in the access ISPs, with remaining share being mostly located within the academic network Surfnet and at hosting providers.
Access providers in the Netherlands typically use very long DHCP lease times, a year or even longer, which renders their IP addresses more or less static. This means that we can conservatively relate the 900k IP addresses to roughly 450-900k broadband subscribers – not too high a figure to cause panic, but certainly large enough to warrant attention.
Figure 1
The pie chart slices roughly correlate with the size of the ISPs, as one would expect. However, if we correct for size – by dividing the absolute infection figures by the number of subscribers of each ISP – we get a better sense of how well the providers perform relative to each other. Figure 2 represents
these relative infection rates over the period of January 2009 to June 2010.
Now we see that of the three large Dutch ISPs, one is doing very well (NL13), one average (NL06) and one rather badly (NL05). This difference warrants an explanation, especially since all these ISPs are operating under the same regulatory framework and more or less similar market conditions. The question was put forward to the ISPs, all participants in the Dutch anti-botnet treaty. Some of the differences may be caused by differences in the user population, but these three ISPs actually felt that their customer bases were quite similar. So what explains these differences of up to three times more infections per subscriber? The ISPs focused on the different mitigation practices that they have in place.
Figure 2
The best-performing ISP had a fully automated infrastructure in place for the detection and mitigation of bots on their networks. The infrastructure works in this fashion: real-time processing of multiple (publicly available) feedback loops, making judgements on whether a report is a false positive, and finally contacting and quarantining the infected subscriber if certain thresholds are met.
This ISP was also the only one having a full-fledged quarantining network in place. Multi-level quarantining systems are necessary for this automation to work: many ISPs nowadays provide service packages that include VoIP and IPTV; it is important to be able to just disconnect only a user’s Internet access, and provide her with the ability escape the walled garden – if the machine has been cleaned up. The other alternative is to completely shut the port of infected customers; understandably, in this case, the threshold for action will be set to much higher, making overall the system less effective.
From an economic perspective, the use of automation in botnet mitigation has an interesting effect on the incentives of ISPs. Although setting up the infrastructure requires some initial investment, automation reduces one of the main disincentives ISPs have in handling infected machines: it lowers the costs of customer support and abuse management. This lower cost enables an ISP to contact, quarantine and help more customers within the same resources, which promises lowers infections levels and more protection for users. This explanation not only holds for the Netherlands, but is also consistent with patterns that we had previously observed at the global level.
There are other interesting points which we shall reflect on in future blog posts. In the meanwhile, you can download the complete report here if you are interested.
