Problem solve Get help with specific problems with your technologies, process and projects.

Reputation systems gaining credibility in fight against spam

Now that nearly all organizations are employing some sort of anti-spam technology, spammers know their only hope for success lies with outwitting spam-detection strategies. But as Mike Rothman writes, the emergence of reputation-based systems is making it easier to weed out spam before it ever reaches the network gateway.

What's in a reputation? In a lot of cases, it can mean the difference between being overrun by spam, phishing and other undesired email, and keeping your users' inboxes free of potentially dangerous payloads. Spam has undergone a radical evolution during the past few years, and reputation systems are now a key technology in dealing with the ever-increasing volume of unwanted messages.

Historically, spam has been detected using a set of signature-matching and heuristic approaches optimized using a weighting system. This worked fine for a while, but as volumes continued to increase exponentially, spam gateways and services couldn't keep up. It was too resource intensive to conduct a detailed scan on every message.

Listen to Mike Rothman's tip

This tip is also featured as this week's Threat Monitor podcast. Download his spam prevention advice to your PC or favorite MP3 player.
Lately, the problem has been become more acute due to the wave of image spam. These messages use randomly sliced and pixilated images to evade spam detection. Many vendors have responded with optical character recognition (OCR)-based approaches to interpret words in the images. But OCR is resource intensive, further impacting the scalability of current anti-spam gateways.

Enter reputation systems, stage left. Reputation systems have been in use for the past three years, but are only now becoming "table stakes" for any vendor offering email security solutions. That is, it's hard for any vendor to substantiate a high spam detection rate without relying on reputation.

The general concept behind a reputation system is that you can, with some precision, figure out the likelihood of a message being spam, based on who is sending it. That's right: based upon the IP address of the sender, it has become possible to determine the sender's intent.

Why not use the sender's address -- or some other attribute of the message -- to assess reputation? Basically because IP addresses cannot be spoofed; they identify the sender and receiver of an email message and are essential to ensuring a message gets to its destination. You can fake pretty much everything else about a message, but not the originating IP address.

In order for a reputation system to be effective, the reputation provider needs to see a bunch of data -- think billions of messages -- in order to conduct a comprehensive analysis that will yield accurate results. Look for vendors that have access to a tremendous amount of message traffic and have sending histories for millions of IP addresses. Don't forget to ask each vendor how many reputations it has.

So how does a reputation system actually help your organization? Its data serves as another ingredient in the spam-detection cocktail that your company uses to help determine which messages are unwanted. Adding a measure of sender intent will definitely help make the cocktail more effective. Spam-detection cocktails use hundreds of attributes, scored and optimized to determine whether a message is spam. No one attribute is fool-proof, so in general the more data you have, the more optimized your cocktail will be. It's not that reputation data will help catch spam that no other technique would catch, but another "juror" weighing in with a guilty verdict increases the confidence level of the spam decision.

For more information:

See how image spam is clogging up networks and tricking the spam filters. Contributor Sue Hildreth reports.

Is the CAN-SPAM Act a help or a hindrance? Security expert Joel Dubin examines the effectiveness of the regulation.

Which email attacks are coming next? In this Messaging Security School lesson, Tom Bowers explains what kinds of malicious email code we can expect in the future.
The best way to use reputation is for connection management. Connection management is an additional layer of protection positioned at the perimeter, acting as a coarse filter on incoming message traffic. By checking inbound messages against data from a reputation database and discarding traffic from obviously "disreputable" senders, an organization can often block at least 50% of spam before it ever enters the network. That takes a lot of traffic away from the gateway, making it possible to do a full analysis on every message.

So what's the catch? First, as with every other spam detection technique, reputation systems are an inexact science. And every email that gets flagged at the perimeter is essentially getting the death penalty. Most vendors' systems place borderline messages in quarantine so false positives can be retrieved. But that's why most organizations set their connection management settings conservatively, so messages from only the most egregiously bad senders will be discarded.

Now spammers are not stupid, so when they realized that these reputation systems were affecting deliverability, they started looking for other ways to obscure sender identities. Since IP addresses can't be spoofed, they did the next best thing: they recruited an army of anonymous zombies to do their dirty work for them.

Zombies are actually the fatal flaw within reputation systems. Some reputation systems assume that unknown senders (which are most likely zombies) are good, and others figure they are bad. Neither method is ideal; assuming unknown senders are bad can result in more false positives, and assuming they are good inhibits the effectiveness of the connection management function. Personally, I favor systems that take a "guilty until proven innocent" approach; it's pretty clear that a great majority of the email senders out there have bad intentions. But that approach may not work for everyone. Fortunately for end users, reputation is only one piece of the spam-detection puzzle.

In today's enterprise messaging environment, there is too much spam traffic to scrutinize every message. Reputation systems used to discard spam before a message passes through the perimeter, alleviating pressure from the gateway. While they aren't the answer to all of an organization's spam woes, when used in conjunction with other technologies, reputation systems can be a valuable addition to a corporate anti-spam strategy.

About the author
Mike Rothman is president and principal analyst of Security Incite, an industry analyst firm in Atlanta, and the author of The Pragmatic CSO: 12 Steps to Being a Security Master. Rothman is also's expert-in-residence on information security management. Get more information about the Pragmatic CSO at, read his blog at, or reach him via e-mail at mike.rothman (at) securityincite (dot) com.

This was last published in April 2007

Dig Deeper on Email and Messaging Threats-Information Security Threats

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.