Study: 95% of all e-mail sent in 2007 was Spam

CNET reports,

There was a time – 2004 to be precise – when spam “only” consumed 70 percent of all e-mail. Those were the good old days. Today, as Barracuda Networks’ annual spam report shows, upwards of 95 percent of all e-mail is spam.

Personally I think those numbers are skewed. Barracuda is a professional provider of spam blocking, and their customers are businesses that can afford to install a dedicated machine for spam filtering. We had one at Funcom. Me on the other hand, I don’t have one of those. And at least judging from myself, I’m less likely to enter my home email address into a form than my work email.

Still, 95% is insane. Spam processing is probably the major part of what my server is doing. About 2-3 of them every day make it through my SpamAssassin and end up getting filtered by the Thunderbird rules or myself. Some years ago I set up an automated process that takes the Junk folder on my IMAP server and teaches it to SpamAssassin so that it learns from the mistakes it made, so I’m staying ahead of the deluge.

Could everybody please stop buying from these people so they give up?

read more | digg story

Get off my network if you can’t update your computer!

By now, we all know that the main reason there are so many viruses and spam going around these days are Zombie PCs. These are Windows machines which heir owners did not upgrade, and which got attacked by one of the many exploits for Outlook Express, Outlook or Internet Explorer.

Why do these people not update Windows? Because to them, there doesn’t seem to be a problem, not until it’s too late. So let’s tell them – reject email from Outlook/Express if it’s not one of the latest versions. Some people might want to reject all Outlook email, but I wouldn’t go that far, yet.

There are two good ways to go about this: at SMTP time, or in your mail filter. I’m using Exim 4 and procmail for my two examples here, YMMV but you’ll get the point.

To make exim reject old Outlook versions, we can write a system filter. In general, this is sotred in /etc/exim/system-filter.exim. Your rule could look something like this:

if $h_x-mailer: contains "Outlook"
   and ( $h_x-mailer: matches "5\.[50]0\." or $h_x-mailer: matches "6\.00\.2[678]" )
then
    fail "<> \
         This message has been rejected because it was sent from an \n\
         unsafe computer.\n\
         If you intended to send us email in the future, please go to \n\
         http://windowsupdate.microsoft.com/ and install any available \n\
         security updates."
    seen finish
endif

This rule will reject mail from versions 5.0, 5.5 and from older 6.0 versions at SMTP time (so the mail never really makes it into your system) and send a failure message back to the server. You can easily extend it to cover more versions. If you want to know which version of Outlook / Outlook Express is currently considered “safe” by Microsoft, you can find them on this page.

You may not have access to your system mail filter, or may not want to go so far as to reject the mail – maybe a warning is all it takes? And maybe you don’t have exim on your system. Then you can try combining a procmail recipe with a script. In your .procmailrc file, simply add these lines:

:0 ihc:oe.lock
* ^X-Mailer: Microsoft Outlook Express \/.*
| $HOME/bin/oewarn.sh $MATCH

The oewarn.sh bash script is a wrapper around a python script doing the detection, that will send a reply in case we don’t like the version. It looks like this:

#!/bin/sh
SENDMAIL="/usr/sbin/sendmail"
$HOME/bin/oewarn.py "$1" || ( formail -r -I"Precedence: junk" -A"X-Loop: eressea@eressea.upb.de" ; \
  cat $HOME/bin/oewarn.txt ) | $SENDMAIL -t

The text file oewarn.txt contains your nastygram message – what you want the sender to receive. The python script oewarn.py contains the magic to decide what version we have and whether we like it:

#!/usr/bin/env python
from sys import argv, exit
from string import split

def verify(versionstr):
    version = split(versionstr, '.')
    if len(version)==4:
        try:
            major, minor, release, build = map(lambda x: int(x), version)
            if major < 6 or release < 2800 or build < 1123:
                return 1
        except:
            pass
    return 0

exit(verify(argv[1]))

In my case, the message you receive would read like this:

Your computer is a danger to the Internet!

You are running a severely outdated version of Outlook Express (and possible
Internet Explorer). These two programs are the main reason the Internet is
clogged with spam and viruses today. Using Outlook Express is bad enough;
but failing to install critical updates is criminal negligence.

You will be given up to 3 warnings before we refuse to accept further emails
from your account. If you want to continue sending email to this address,
please update your software. Or even better, use a modern, safe Mail
program: http://www.mozilla.org/projects/thunderbird/

If you have a question regarding this policy, please contact
postmaster@eressea.de

Where does Spam come from?

The original SPAM comes from Hormel Foods, of course. But as Ben Goodger writes, Spam emails and Viruses come from Microsoft’s Outlook products. He says he’d like to simply block them all.

I’ve done a count through my spam: For 298 emails I received from Outlook (and OE), I received 2269 viruses and spams from them. During the same time frame, I got more emails from Thunderbird users, but only 9 spams.

Outlook and it’s little brother Outlook Express are the prime tool for the propagation of spam, viruses and phishing emails these days, and I think it’s almost getting to be a good idea to reject mail originating from them. After all, if 89% of all phonecalls you get were from telemarketers, wouldn’t you get an unlisted phonenumber? Damn right you would. Now that 89% of the mail reaching me through Outlook are bad, I don’t want Outlook to reach my mailbox anymore.

Well, not quite yet. But I’m *this* close.

Comments disabled

I had to disable comments after getting spammed with 1000 casino-advertising comments. I’m going to re-enable comments as soon as I find a way to fight the spam – until then, why don’t you comment in your own blog and use trackback?

Train your Spam filters

If you have an intelligent spam filter like the one built into Thunderbird or Spam Assassin, you need to train it right.

The most common problem I’ve seen is taht people only tell the spam filter about their spam, but not about the good mails (the ‘ham’). In Mozilla, you do this by selecting all the good emails and saying “Mark this Message as Not Junk”. This is important, nay, vital information for the filter.

I did this with Spam Assassin a while back, and any time I get a spam now, I feed it back into the filter. The results are pretty impressive: My spam filter now catches 99.2% of all the spam. I’m tracking this and making a diagram from it. You can see the improvement over the untrained filter I had two months ago.

Click for full size

That’s pretty darn good. I get in excess of 100 spams a day, which isn’t as much as some of my friends get, but still bad enough.

86% Spam, 14% Mail, 100% Wahnsinn

Heute mal gezählt: Seit Montag habe ich 404 Mails bekommen. Davon hat der Spamfilter auf dem Server 283 weggeschmissen. Thunderbird hat weitere 64 Spams erkannt (oder ich hab Mails als Spam markiert). Übrig geblieben sind 57 Mails. Das sind 14% des Gesamtvolumens.

Ich habe das zum Anlaß genommen, mir mal die Filter von spamassassin genauer anzusehen und anzupassen. Ich bewerte jetzt vor allem HTML-Mails wesentlich härter als zuvor – ganz wegwerfen tue ich sie nicht, aber wenn sie in Kombination mit anderen Merkmalen auftreten, ist die Chance groß, dass die Mail nicht durchkommt. False positives von Leuten, die HTML nicht abschalten können, nehme ich in Kauf – gesehen habe ich allerdings noch keine.

Songs inspired by SPAM

Durch Penny Arcade bin ich heute auf diese coole Sammlung von Musik aufmerksam geworden. Songs, die sich eine Zeile oder mehr aus einer Spam email nehmen, und nicht nur ein oder zwei, nein ein ganzes Album voll. Sehr unterschiedliche Musik von Independent Bands, einiges sogar recht lustig.

Mein Favorit: Uncle Azathoth – Urgent business Confidential. Überhaupt nicht meine Musik, aber gut gemacht 🙂 Und Justin Bacon – My Parents Are Gone For The Weekend (explicit lyrics) ist auch Spitze.

Spam of the week

Click to enlarge...I’ve got pretty good spam filters, but I get the feeling that I might be missing out on a lot of fun because of that.

Here’s a really absurd example. What does the spammer think? Coud people really believe he is selling heroin, tomahawk missiles or slaves?

It made me shake my head in wonder. The phone number is for tech support at datacolo.com NOC, maybe the originator of the spam has had their spam site taken down by them?

The website it refers to is russian, and it is an information site for new internet users – explaining internet criminality, among other things. It’s quite probably not the spam originator. But their forum is full of people trying to sell fake id or credit cards – that’s it!

Spam, paranoid mail scanners

Somebody is using forged email addresses in the eressea-pbem.de domain to send out spam. Since I’m the postmaster for that domain, all the undeliverable mails land in my inbox – or they did, I’m now filtering on them.

There was one real gem among the replies. This stupid mail scanner rejected the mail not because it was spam, but because it contained the word online. That’s paranoid. I’d hate to be one of their users, really…