Trips from June 2008

The problem with running my own mail server is that I have to deal with spam. Luckily, running my own mail server lets me configure my own plan for dealing with it. Spam has been a problem with e-mail for quite some time and it's only getting worse. Luckily, people have developed ways of automatically identifying spam. The first such filters were purely based on common phrases or patterns in spam. As spam grew, one of the major advances that came about was statistical filtering, primarily based on Bayesian analysis. Eventually spammers figured out how to get around these filters (which is why spam messages often have lots of random content in them), and new techniques like greylisting have emerged.

Greylisting returns a code specifying a temporary failure when a server that has not been seen before tries to contact the greylisting server initially. Mail servers are required to attempt redelivery at a later time, but most spam software does not follow this rule. The problem with greylisting though is that it delays message delivery, which I don't like. Unfortunately with the amount of spam (I receive around 300 messages/day), I've been forced to incorporate greylisting into my stack of filters.

Because I don't like greylisting, my new plan of attack, while using it, does not use it for all messages. Here's what happens to all messages as they come into my mail server:

1. Most spam is sent from home computers that have compromised by viruses. This means most spam comes from users on typical cable/DSL/dial-up connections. So every message that arrives at my server has the address of its server checked against several lists (RBLs) that contain lists of such addresses as well as known addresses that send spam. Any message with an address that is in one of the RBLs is considered suspicious.
2. If the address is not found, another issue with many spam servers is that they are not properly configured. My mail server already demands compliance with standards, but very often the originating server does not have what's known a reverse DNS record. This is checked, and if there is no reverse record, the message is considered suspicious.
3. Finally, the SPF records (which identify valid mail servers for particular domain) are checked and if there's any failure here, the message is also flagged as suspicious.
4. The previous 3 steps are carried out by software called ACLPolicyDaemon. If a message is flagged as suspicious, it is then greylisted using software called sqlgrey.
5. If a message makes it past the policy daemon and greylisting, the next line of defense is a combination bayesian analysis software and content filter called SpamAssassin, which is the most well known and overall best such software.

Hopefully, this should drop the number of spam messages to almost 0 (about 5-10 messages were getting through before). The advantage that I get here is that I can selectively greylist messages that are most likely spam, while not rejecting just about any valid server. We'll see how well it works.