Saturday, October 21, 2006

Catch Spam as it Arrives

I have been using Microsoft Outlook now for I don’t know how many years and as of the last couple of years Spam has become a huge issue. I get bombbarted with spam every day. A couple of years ago I came across a great tool from a friend; it was a Spam blocker program that works on top of Outlook. The program is called SpamBayes and can be found at http://spambayes.sourceforge.net/.

That's great, but what's SpamBayes?
(the non-technical hand-waving answer)
SpamBayes will attempt to classify incoming email messages as 'spam', 'ham' (good, non-spam email) or 'unsure'. This means you can have spam or unsure messages automatically filed away in a different mail folder, where it won't interrupt your email reading. First SpamBayes must be trained by each user to identify spam and ham. Essentially, you show SpamBayes a pile of email that you like (ham) and a pile you don't like (spam). SpamBayes will then analyze the piles for clues as to what makes the spam and ham different. For example; different words, differences in the mailer headers and content style. The system then uses these clues to examine new messages.

For instance, the word "Nigeria" appears often in spam, so you could use a spam filter which identifies anything with that word in it as spam. But what if your business involves writing a guidebook on Nigerian Wildlife Conservation? Clearly a more flexible approach is necessary. Additionally spammers will adapt their content over time and will no longer use the word "Nigeria" (or the words "Lose Weight Fast", or any number of other common lines). Ideally the software will be able to adapt as the spam changes.

So, that is what SpamBayes does. It compares the spam and the ham and calculates probabilities. For instance, for me, the word "weight" almost never occurs in legitimate email, but it occurs all the time in 'lose weight fast' spam. SpamBayes can then look at incoming email, extract the most significant clues and combine the probabilities to produce an overall rating of "spamminess". It flags the messages so that your mailer can handle the different message types. You might set it up so that ham goes straight through untouched, spam goes to a folder that you ignore (or delete without checking) and the unsure messages go to another folder which you can review for errors.

No comments: