What's in a reputation? In a lot of cases, it can mean the difference between being overrun by spam, phishing and other undesired email, and keeping your users' inboxes free of potentially dangerous payloads. Spam has undergone a radical evolution during the past few years, and reputation systems are now a key technology in dealing with the ever-increasing volume of unwanted messages.
Historically, spam has been detected using a set of signature-matching and heuristic approaches optimized using a weighting system. This worked fine for a while, but as volumes continued to increase exponentially, spam gateways and services couldn't keep up. It was too resource intensive to conduct a detailed scan on every message.
Lately, the problem has been become more acute due to the wave of image spam. These messages use randomly sliced and pixilated images to evade spam detection. Many vendors have responded with optical character recognition (OCR)-based approaches to interpret words in the images. But OCR is resource intensive, further impacting the scalability of current anti-spam gateways.
Enter reputation systems, stage left. Reputation systems have been in use for the past three years, but are only now becoming "table stakes" for any vendor offering email security solutions. That is, it's hard for any vendor to substantiate a high spam detection rate without relying on reputation.
The general concept behind a reputation system is that you can, with some precision, figure out the likelihood of a message being spam, based on who is sending it. That's right: based upon the IP address of the sender, it has become possible to determine the sender's intent.
Why not use the sender's address -- or some other attribute of the message -- to assess reputation? Basically because IP addresses cannot be spoofed; they identify the sender and receiver of an email message and are essential to ensuring a message gets to its destination. You can fake pretty much everything else about a message, but not the originating IP address.
In order for a reputation system to be effective, the reputation provider needs to see a bunch of data -- think billions of messages -- in order to conduct a comprehensive analysis that will yield accurate results. Look for vendors that have access to a tremendous amount of message traffic and have sending histories for millions of IP addresses. Don't forget to ask each vendor how many reputations it has.
So how does a reputation system actually help your organization? Its data serves as another ingredient in the spam-detection cocktail that your company uses to help determine which messages are unwanted. Adding a measure of sender intent will definitely help make the cocktail more effective. Spam-detection cocktails use hundreds of attributes, scored and optimized to determine whether a message is spam. No one attribute is fool-proof, so in general the more data you have, the more optimized your cocktail will be. It's not that reputation data will help catch spam that no other technique would catch, but another "juror" weighing in with a guilty verdict increases the confidence level of the spam decision.
The best way to use reputation is for connection management. Connection management is an additional layer of protection positioned at the perimeter, acting as a coarse filter on incoming message traffic. By checking inbound messages against data from a reputation database and discarding traffic from obviously "disreputable" senders, an organization can often block at least 50% of spam before it ever enters the network. That takes a lot of traffic away from the gateway, making it possible to do a full analysis on every message.
So what's the catch? First, as with every other spam detection technique, reputation systems are an inexact science. And every email that gets flagged at the perimeter is essentially getting the death penalty. Most vendors' systems place borderline messages in quarantine so false positives can be retrieved. But that's why most organizations set their connection management settings conservatively, so messages from only the most egregiously bad senders will be discarded.
Now spammers are not stupid, so when they realized that these reputation systems were affecting deliverability, they started looking for other ways to obscure sender identities. Since IP addresses can't be spoofed, they did the next best thing: they recruited an army of anonymous zombies to do their dirty work for them.
Zombies are actually the fatal flaw within reputation systems. Some reputation systems assume that unknown senders (which are most likely zombies) are good, and others figure they are bad. Neither method is ideal; assuming unknown senders are bad can result in more false positives, and assuming they are good inhibits the effectiveness of the connection management function. Personally, I favor systems that take a "guilty until proven innocent" approach; it's pretty clear that a great majority of the email senders out there have bad intentions. But that approach may not work for everyone. Fortunately for end users, reputation is only one piece of the spam-detection puzzle.
In today's enterprise messaging environment, there is too much spam traffic to scrutinize every message. Reputation systems used to discard spam before a message passes through the perimeter, alleviating pressure from the gateway. While they aren't the answer to all of an organization's spam woes, when used in conjunction with other technologies, reputation systems can be a valuable addition to a corporate anti-spam strategy.
About the author
Mike Rothman is president and principal analyst of Security Incite, an industry analyst firm in Atlanta, and the author of The Pragmatic CSO: 12 Steps to Being a Security Master. Rothman is also SearchSecurity.com's expert-in-residence on information security management. Get more information about the Pragmatic CSO at http://www.pragmaticcso.com, read his blog at http://blog.securityincite.com, or reach him via e-mail at mike.rothman (at) securityincite (dot) com.
