Spam filtering with countries.nerd.dk considered harmful
Saturday, December 1, 2007
Spam filtering with countries.nerd.dk considered harmful
DNS blacklists (DNSBL) provide information about characteristics and past observations of IP addresses and have been used in filtering spam for more than a decade. In short, a spam filter may check one or more DNSBL services to determine if the network address from where an e-mail is delivered is trustworthy or suspicious.
Besides listing addresses of known spam sources or virus-infected machines, there are lists for criteria such as network type (dial-up/cable/DSL) and configuration issues (open relays, RFC non-compliance).
One of my colleagues recently had e-mail to a client rejected by their mail gateway with the error message "554 Your Host 32.nn.nn.nn was found in the DNS BlackList at uk.countries.nerd.dk."
When he asked for help with this, my first thought was that one of our addresses had, rightly or wrongly, been listed as a spam source. However, after looking around countries.nerd.dk it became clear that the recipient was blocking all mail that appeared to come from certain countries according to the countries.nerd.dk database despite the disclaimer on that Website that "countries.nerd.dk is NOT a list of spammers, it is an IP-to-country DNS mapping service."
What's worse in this case is that the mapping was incorrect: The whole 32/8 netblock is declared to be based in the UK: "32.0.0.0/8 :127.0.0.2:Your IP is in uk, rejected based on geographical location". There may be some UK based addresses in that netblock but others are located in North America and possible other places too, and similar geographic mapping services managed to get the location of the particular mail server (almost) right.
Although many open source and commercial mail filters rely on DNSBLs, there has been valid criticism, and even lawsuits against DNSBL operators. The main concern I have is that administrators may rely on a single DNSBL service to mark messages as spam and reject them without understanding the service's reliability and limitations.
Besides listing addresses of known spam sources or virus-infected machines, there are lists for criteria such as network type (dial-up/cable/DSL) and configuration issues (open relays, RFC non-compliance).
One of my colleagues recently had e-mail to a client rejected by their mail gateway with the error message "554 Your Host 32.nn.nn.nn was found in the DNS BlackList at uk.countries.nerd.dk."
When he asked for help with this, my first thought was that one of our addresses had, rightly or wrongly, been listed as a spam source. However, after looking around countries.nerd.dk it became clear that the recipient was blocking all mail that appeared to come from certain countries according to the countries.nerd.dk database despite the disclaimer on that Website that "countries.nerd.dk is NOT a list of spammers, it is an IP-to-country DNS mapping service."
What's worse in this case is that the mapping was incorrect: The whole 32/8 netblock is declared to be based in the UK: "32.0.0.0/8 :127.0.0.2:Your IP is in uk, rejected based on geographical location". There may be some UK based addresses in that netblock but others are located in North America and possible other places too, and similar geographic mapping services managed to get the location of the particular mail server (almost) right.
Although many open source and commercial mail filters rely on DNSBLs, there has been valid criticism, and even lawsuits against DNSBL operators. The main concern I have is that administrators may rely on a single DNSBL service to mark messages as spam and reject them without understanding the service's reliability and limitations.
Labels: spam, technology