What

Akismet is a comment spam filtering service, provided by Automattic, the company founded by WordPress developer Matt Mullenweg (together with some other core WordPress developers). Recently, the leader of Yahoo developer network joined the startup, so there apparently is money and potential…

How

You need to install a plugin in your blogging engine. There (obviously) already is such a plugin for WordPress (and it comes by default with WordPress 2.0), but there will soon be plugins for other blogging tools as well. In fact, Akismet is suitable to check any kind of user-submitted content for spam. There are generic software libraries that use the Akismet API on top of which anti-spam plugins for all kinds of social software tools can be built: forums, wikis… Have a look at the Akismet development.

Behind the scenes…

The Akismet plugin sends a submitted comment to the Akismet server. The server runs a series of tests and the result classifies the comment as spam or not. Basically, the server works as an email spam filter. I wouldn’t be surprised if a lot of the Akismet code was based on Spamassassin (an open source email spam filter). Like Spamassassin, Akismet learns from its mistakes: so if you notice a comment that got through and reclassify it as spam, that result is sent to the Akismet server and similar comments will be more likely to be marked as spam. Vice versa for “false positives”. Note that this way you profit from other Akismet users’ efforts on their blogs as well (and they from yours)! The Akismet service is comparable to the email spam database Vipul’s Razor (used by SpamAssassin) and its commercial version Cloudmark, that is used by some commercial email spam filters.

Results

I have Akismet running on this blog and blogs I installed for other people: I could safely turn of comment moderation, which makes the interaction between the commenters a lot faster and livelier. I’ve only had 2 false positives in about a month (very short comments or trackbacks). In order to get Akismet running, you need to get an API key by registering as a user at WordPress.com (the hosted version of WordPress provided by Automattic). The Akismet service is free for personal use, let’s hope they don’t run into performance problems any time soon…

Web2.0

If you were looking for a quintessential Web2.0 application, then Akismet is one:

• decentralised service
• open api
• the more people and applications that use it, the better it gets
• results are driven by the collective judgement of the users, each of them driven by self-interest
• business model:
• build up a user base with free software (WordPress), then provide that audience with a paid service.
• the lock-in is in the data, not in the software
• the software is open source, and helps drive the adoption of the API
• and it has nothing to do with Ajax :-) (unless you count the WP admin panel in)

“I googled for something completely different, but found your page… and have to say thanks. nice read.”

Wouldn’t you immediately hit the “approve” button?

In this case, the comment poster’s url pointed to http://otaro.co.uk/, a page full of “cheap car hire” ads and links. Ironically, the poster’s email was the confidence-building “please.no.spam@otaro.co.uk”. But the best part of course was the combination “I googled for something completely different” (so you don’t expect me to write anything relevant) and “have to say thanks, nice read” (so you approve in a wink). The blogger’s ego… the weakest spot in comment spam protection.

Now this was a simple one, because the hurried spammer left his link load on the first script run. A more patient strategy would have been to build confidence in the targeted blogs by not giving away his intentions immediately. Lots of (at least WordPress blogs) hold comments from unknown posters (ip’s, mail addresses…) in moderation by default. Once a comment is approved, the poster is whitelisted. The technique would work on the level of individual blogs, and, when smart enough to fool lots of users, maybe even for systems using the kind of collaborative filtering such as used in Akismet.

So you better think twice before approving another vague “thank-you” or “great reading” post….

BTW: Seo BlackHat is an interesting (and entertaining!) resource on all kinds of web spam – he hasn’t written on this yet, so maybe it was his? :-)