Using email spam filtering techniques to get rid of spam

Filters stem the flow of junk email, but spam remains a big distraction. Learn how to use spam filtering techniques to get rid of spam.

This article can also be found in the Premium Editorial Download: Information Security magazine: Are you secure? Adam Putnam says, "Prove it!":

Lee Churchman is exhausted by spam--the constant drumming of unsolicited email that tumbles across his mail server. And he isn't the only one.

One-third of the calls Churchman receives at his network consulting firm, System Wide Resources (SWR) in Ridgefield, Wash., are from customers wanting to do away with e-mails peddling cheap Viagra, easy weight-loss plans and hassle-free mortgages.

"Spam is not a popular subject with me or my customers," he says. "We are all basically sick of it."

In a new survey conducted by Information Security/SearchSecurity.com on the business implications of spam, 63 percent of respondents said the responsibility of paring bulk e-mail falls to senior executives and security managers, forcing them to devote time they don't have to solving this seemingly trivial problem.

"I can't skip any of my other tasks that help to keep the agency running," says Norman McIntosh, the information systems manager at D.A. Blodgett Services for Children and Families, a family resource center in Grand Rapids, Mich. "That means extra time used that I can't spare."

Persistent nuisance
Everyone has grown accustomed to the daily chore of weeding through the long list of spam messages. Despite our best efforts, offers for African money-laundering schemes, male enhancement products and discounted software still reach our inboxes.

Lost productivity (92 percent) and clogged e-mail servers (62 percent) were cited as the most egregious consequences of spam, according to the survey. More than half of the respondents said they were concerned about malware infections, but few made a distinction between spam-borne viruses and self-propagating viruses that acted like spam.

"It's a terrible nuisance," says Dwight Cook, operations manager for Webs.com, an Internet hosting and design division of Sound Works. "Fifty to 60 percent of our daily e-mails are spam. And, when we had different viruses hitting at the beginning of the year, it was up to 80 percent daily."

For Webs.com, this translates into more than 40,000 quarantined messages per day that consume staff time, money and bandwidth. In addition to two servers that solely handle the increased e-mail traffic caused by spam, Cook estimates his company has invested more than $20,000 in the past year to combat the nuisance. "I can't imagine the cost of spam across the country."

A new Yankee Group report estimates that spam costs U.S. businesses $4 billion annually in lost productivity.

Though expenses are big factors in the spam war, the threat of viruses propagating via spam is a perceived as a security risk. More than half of the security professionals responding to the Information Security/SearchSecurity.com survey believe spam to be a direct source of malware. Worse, they fear that malware writers and spammers could collaborate on more invasive ways of compromising networks and circumventing filters.

"If the spam people and the virus people got together, we could have a big problem," says Chuck Mulleady, IT manager at Granite Systems, a developer of inventory solutions for wireless service providers. "There's no telling what would happen."

Nevertheless, AV vendors can't point to a single incident in which spam was used as an infection medium. Most will say it has the potential to spread viruses, or that self-propagating malware is a form of spam. By definition, spam is unsolicited bulk commercial e-mail; self-propagating malware isn't spam.

Filtering out
End users, sometimes the most vocal people in the antispam chorus, need training on how to properly identify and purge unwanted e-mails--both as spam and malware prevention processes.

"I train people to delete everything from an unknown person," says D.A. Blodgett's McIntosh. "If it's from a known person or company and it has an attachment that you're not expecting, call them to make sure they sent it to you. When in doubt, delete."

But education can only go so far, and spreading the antispam message diverts attention from other security imperatives, such as guarding passwords and protecting data.

"I've been hammering it over and over again," says McIntosh. "It's a training issue, and end users just don't absorb it."

There are effective spam-blocking products on the market today, and more than half of the Information Security/SearchSecurity.com respondents have picked their weapons of choice: Bayesian filtering and heuristics are the most popular, followed by signature/content matching and blacklisting (see "Canning Spam," below).

Using filters and scanners, especially those that use multiple spam identification methods, helps reduce the volume of spam hitting enterprise mail servers and users' desktops, but none are foolproof. Spammers and security managers are engaged in an information arms race.

"As fast as we can put blocks up, spammers come out with a way to work around them," says SWR's Churchman. "It's a Catch-22."

False identification
All antispam solutions suffer from the same problem: false positives and false negatives. Even the best systems won't catch every spam message, and rigid tuning will flag a fair number of legitimate e-mails.

At Granite Systems, Mulleady's Bayesian e-mail filter was so sensitive and its tuning so complicated that all of Granite's incoming messages were blocked. The false positives caused such a problem that Mulleady had to cancel his service and stop using the spam blocker.

"Our company, like many others, is dependent on e-mail," Mulleady says. "We were getting all of our e-mails blocked."

The efficiency of most spam filters ranges from 70 to 90 percent. Newer solutions, which incorporate multiple methods for identifying spam, continue to bring down the number of false positives. Only 46 percent of respondents cited false positives as having a moderate impact on their business operations. Fewer than 4 percent said it has a high impact (see "False Positives," right).

Sealing the envelope
SWR's Churchman sees light at the end of the tunnel.

"Under the federal and state laws, there's a legal precedent for cease and desist," he says. "We can have the spammer's business license pulled," something he has done to many spammers worldwide. "I've even shut down satellite transmission from the Far East."

But, this is hardly a lasting solution. "We've found that over the past six months, the federal laws haven't really stopped anything," Churchman says. "As soon as the spammers are shut down, they move somewhere else and start up again."

The CAN-SPAM Act, which took effect Jan. 1, is designed to regulate the distribution of commercial e-mail and curb the volume of unsolicited electronic messages. Enterprises and spam monitors haven't seen an appreciable decline in spam traffic, but AOL, the country's largest ISP, claims its spam numbers are dropping significantly. Most companies, though, don't believe regulations will have much of an effect; spammers can simply move their operations offshore and out of the reach of U.S. authorities.

For Peter Piluk, an independent consultant with NowWhat Computer Solutions, spam is more than the flippant annoyance of pornographic and sales-driven e-mails.

"Spam has serious ramifications for security," he says. "The wording of many of these e-mails can trick even the most experienced computer user into agreeing to something they normally wouldn't. Anytime spam makes it through your filters is another potential breach."

About the author:
Amber Plante is assistant editor at Information Security.

This was first published in May 2004

Dig deeper on Email and Messaging Threats (spam, phishing, instant messaging)

Pro+

Features

Enjoy the benefits of Pro+ membership, learn more and join.

0 comments

Oldest 

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

SearchCloudSecurity

SearchNetworking

SearchCIO

SearchConsumerization

SearchEnterpriseDesktop

SearchCloudComputing

ComputerWeekly

Close