Published: 20 Aug 2004
While millions of Gold's Gym members across the country were trimming down and pumping up, the corporation's IT infrastructure and security was in bad shape.
"There was no connectivity among the gyms or corporate offices except e-mail," recalls corporate IT manager Kurt Koenig.
Worse, they were pretty much open to attack. "The Austin office didn't even have a firewall, just NAT behind a router," he says. The Church Falls, Va., and Venice, Calif., offices had inadequate "home-type" legacy firewalls.
"I gave my CIO the quote for three dedicated [Check Point] firewalls and the installation, and he couldn't believe the cost," says Koenig.
Faced with a dilemma, Koenig looked into multifunction turnkey appliances, appealing alternatives to pricey high-end firewalls and other point solutions for SMB managers with tight budgets and limited staff.
Koenig got a firewall, gateway antivirus, site-to-site VPN and IDS/IPS in one box, a FortiGate appliance from Fortinet. And, he got it "for less than the engineering services alone were going to cost to install the other firewalls."
But "all-in-one" security appliances offering similar features can be surprisingly different under the hood. To better understand the capabilities of these security gateways, Information Security compared six appliances designed for midsized companies of 500 employees: Barbedwire Technologies' DP Inspector 500X, Fortinet's FortiGate 800, Internet Security Systems' Proventia M30, Secure Computing's SidewinderG2 1100, ServGate Technologies' EdgeForce Accel and Symantec's Symantec Gateway Security 5420. These appliances met our requirements for at least four key security services: packet filtering or proxy firewall; site-to-site VPN; network intrusion detection; and gateway-based AV (see "About This Review").
With "deep" inspection, it's hard to define where firewalling ends and intrusion detection and AV begins; defending yourself against blended threats requires all three to work harmoniously. Companies with more than one office should scrutinize intersite traffic with the same rigor to stop backdoor attacks. These tested products are all intended to meet these needs.
Most of these appliances also provide remote access VPN, intrusion prevention, Web and spam filtering, or Internet services like split DNS. We examined these value-added services but focused on the services most midsized companies require from a perimeter security gateway.
We put these appliances through their paces, then graded them based on five criteria: functionality and flexibility; price; ease of installation; policy configuration; and security monitoring. We didn't test performance, which depends on traffic and policy depth. While balancing security and performance is important, it is specific to each organization.
Out of the Box
One of the benefits of going the all-in-one appliance route is ease of installation. These appliances didn't disappoint us, each installing in 5 to 45 minutes.
The appliances' setups require minimal input and are streamlined with wizards and quick start cards. ISS deserves special mention for "Getting Started" help on first login. Fortinet's Setup Wizard was MIA in our beta, but manual setup required only three easy steps. Secure Computing had the longest and most involved installation, but, even so, spending less than an hour isn't too bad.
The Barbedwire and Fortinet appliances were running in 10 minutes or less because they avoided the cumbersome steps, such as activating licenses and requiring PC software installation and setup via serial port or boot disk, found in the other products.
Symantec took 10 minutes, plus a tech support call to fix a license problem that kept the firewall from starting. ServGate's setup took 19 minutes, most of it spent activating licenses on its Web site.
Of course, the addition of a firewall impacts adjacent devices and traffic patterns. Although one box is relatively easy to integrate, planning is still required to avoid mistakes. Some of these appliances provide documentation and features to ease integration; others overlook essential details.
For example, integration requires topology design and IP address assignment. ISS makes you guess whether DMZ IPs must be public or private, while ServGate explains both options and implications.
Secure Computing offers thorough advice on everything from product placement to policy design, but its manuals lack cabling requirements. Fortinet delivers exceptional supplementary guides that explain esoteric VPN and IPS parameters. In contrast, Barbedwire's disjointed manuals are long on how to integrate the box but frustratingly short on why you're performing each step.
Barbedwire, Fortinet, ISS and ServGate support transparent mode bridging to avoid changing subnetting or routes. Our plans called for NAT mode, which is supported by all six appliances. Routing protocols--supported by Barbedwire, Fortinet and Secure Computing--help distribute subnet changes to other devices.
Every appliance required DNS/host tuning. Barbedwire, Secure Computing and ServGate support gateway DNS, but Symantec requires it, forcing a reconfiguration of all inside hosts or DNS servers. ISS sends dynamic DNS for every DHCP lease; we'd prefer an option to disable these requests in networks that don't use DDNS. Secure Com-puting offers the most flexibility, including split DNS to keep inside addresses private while advertising public addresses.
Configuring Firewall Policies
We defined our access requirements, letting clients reach selected Internet services, including ping, UDP/DNS, Web, FTP, e-mail and H.323. Inbound policies permitted Web-only access to public, DMZ and mapped intranet servers. We then configured the policy into every appliance to compare ease of administration and overall methodology. Differences were obvious from the start:
- Barbedwire, Fortinet and ServGate are primarily application-aware, stateful inspection firewalls. Security managers familiar with conventional stateful inspection firewalls will feel at home configuring these appliances with network objects, combined into rules.
- Secure Computing and Symantec are mainly proxy firewalls with stateful inspection capabilities. Policy definition is driven by application proxy configuration. Application-specific proxies are provided for some protocols; others go through a generic proxy or IP/TCP/UDP filters.
- ISS feels like a stateful inspection firewall, but that's because the GUI doesn't include proxy configuration. Under the hood, it uses several application proxies.
Because firewalls are the foundation for other security services, this architectural difference has a ripple effect on VPN, IDS and AV policies. All six appliances supported our target policies.
Security in a Box?
Attend our webcast:
"Security Appliances: Hype vs. Reality,"
August 25, noon ET, (on demand thereafter).
Register at https://searchsecurity.techtarget.com/..
There were no show-stoppers at this basic level, but overall--not just in configuring firewalls--we made fewer errors with ServGate, Fortinet and ISS simply because they're easier to configure, though ServGate's simplicity reflects a paucity of options.
Secure Computing, on the other hand, is complex, increasing the risk of mistakes that can lead to security breaches. By the same token, the depth of Secure Computing and Symantec firewalls can reward security managers for their effort. Barbedwire has a GUI, but it's really a command-line interface that may deter non-Unix admins.
Barbedwire uses Linux iptables firewall in its appliances to cut costs; an optional softblade adds Layer 7 inspection. Basic configuration took just 10 minutes, but we spent an hour searching for virtual IPs buried in the GUI.
Fortinet's secure Web GUI is responsive and easy to navigate. Adding objects, such as custom services, groups and virtual IPs, was simple. The firewall table is grouped by zone to help users eyeball related policies.
ISS's GUI is intuitive, with warnings and validation to reduce error. Options are hidden under "Advanced" strings to keep the presentation simple. Unfortunately, the GUI is Java-sluggish, and too much is crammed onto a few tabbed pages.
Secure Computing's basic configuration is relatively smooth, despite SidewinderG2's complex policy model. There's a lot to learn, so experience is required to get the most out of SidewinderG2. The complexity also means plenty of opportunities for mistakes.
ServGate has the plainest GUI of the bunch?and therefore the fastest. Even a novice can get this appliance running correctly. There are comparatively few advanced policy options, and this simplicity reflects its lack of flexibility elsewhere, such as AV and intrusion detection.
The way these appliances perform firewalling--packet filter, proxy or stateful inspection--has a ripple effect on VPN, AV and IDS functionality.
Symantec's interface suffers from Java slowness, an inability to paste copied values and distributed parameters that require too much hopping around. Despite this, configuration went smoothly. There's considerable option depth that security managers will appreciate if they invest the effort.
Protecting Intersite Traffic
Once our firewalls were running, we tied them together with site-to-site VPNs, which are widely used to secure intersite traffic over shared links.
Again, we defined a target policy: IPSec ESP in tunnel mode with TripleDES encryption, SHA-1 integrity and perfect forward secrecy using Diffie-Hellman Group 2 or better. We started with preshared secrets and upgraded to certificates. This policy is a subset of ICSA and the Virtual Private Network Consortium requirements and is supported by most IPSec gateways, so we were surprised when it didn't quite work in every box.
Barbedwire uses open-source FreeS/WAN v1.97, which has good interoperability but lacks the features and vendor support. It proved the most challenging: Help files were missing, and documentation was incomplete.
For the rest, we had little trouble establishing VPN connections.
Pairing gateways is one thing; adding an appliance to a network with gateways from other vendors is quite another. Debugging requires visibility into security association (SA) status and proposed/negotiated parameters. Fortinet's and Symantec's GUIs provide good SA detail. Debug-level traces are available on Fortinet, Barbedwire and, to a lesser extent, ServGate.
Other noteworthy differences between site-to-site VPN features:
- While most used AES and DH5, Secure Computing doesn't support AES. Symantec supports AES for ESP but not IKE, and there's no AES in FreeS/WAN.
- When a peer reboots, some gateways have a hard time noticing. Fortinet and ServGate implement dead peer detection, while Secure Computing and Symantec offer options that initiate detection in the absence of traffic. Fortinet's keep-alive option is handy for tunnels that must be available 24/7.
- Symantec supports Entrust certificates for remote access but not for site-to-site VPNs. We upgraded our policies on the other five appliances to use certificates and RSA authentication.
- We used internal certificate authorities on Barbedwire, Secure Computing and ServGate. Self-signed certificates are handy for companies that just need to authenticate a few gateways.
All of the appliances also support secure remote access.
Checking Viruses at the Door
Some may argue against virus scanning at the gateway, but the payoff is eradicating viruses, worms and Trojans before they enter your network. Why should deep packet inspection scan SMTP for malformed headers but ignore virus attachments?
After opening up Atlanta West Carpets' network to e-mail and the Web, Gerad Simpson didn't think the flooring installation company, with fewer than 75 users in its Atlanta and Birmingham, Ala.-offices, needed gateway protection. Then bad things began to happen.
"Blaster hit every machine, but there was nothing destructive," recalls Simpson, Atlanta West's information systems manager. "Then Sasser took us down. We had no firewall. Nothing--only client AV. That wasn't going to cut it."
Simpson, like Gold's Gym's Koenig, considered firewalls (Cisco Systems' PIX in Simpson's case) but "our IT budget is negative." He eventually selected ISS's Proventia. "It catches everything."
All six appliances we tested offer gateway AV and AV-tuning options. Overall, we found AV scan delays tolerable. However, we couldn't perform an apples-to-apples test. Four of these appliances support FTP, three scan POP and another trio scans HTTP. Barbedwire only scans SMTP addressed to the firewall; Secure Computing scans both hosted and transparent SMTP; and the rest transparently scan SMTP to any destination.
Our recommendation: Select an appliance that scans protocols important to your business, block unsupported protocols and use desktop/server AV as a layered defense.
Barbedwire and ISS use Sophos AV; Secure Computing and ServGate use McAfee. Symantec obviously uses its own AV, and Fortinet has its own AV engine.
All six appliances block attachments based on extension and/or MIME type. Fortinet also blocks a short list of "grayware" (spyware, etc.), and ServGate detected suspicious attachments in several messages ignored by the others.
"Sasser took us down. We had no firewall. Nothing--only client AV. That wasn't going to cut it."
-- Gerad Simpson, information systems manager, Atlanta West Carpets
ISS's AV is the least configurable, while Symantec is the only appliance that doesn't offer on-firewall quarantine.
All of the appliances log virus events and issue alerts. Some notify the SMTP sender when viruses are found in outbound mail; others quarantine/delete and advise the recipient. All but Symantec present virus statistics in the GUI. Barbedwire can generate a wide range of reports in different formats, such as a bar graph of virus frequency or a pie chart of the top 10 viruses.
Even the protection offered by products that scan the same protocol can be quite different:
- Barbedwire scans only mail to its hosted server, so it's critical to block SMTP to any destination other than the firewall.
- ISS and ServGate have one AV on/ off switch per protocol, which means, for example, that security managers need to scan either all SMTP or none.
- Fortinet allows policy-specific profiles by enabling AV scanning for selected protocols. It missed one virus test file (fixed the next day), and, when scanning 9,600-plus messages, it passed two files that other products quarantined as Netsky.
- Secure Computing's policies use "Application Defenses" to set MIME scan options for HTTP and SMTP, but a global AV toggle must be on for all scanning.
- Symantec also provides a mix of controls. Firewall policies refer to groups of services, but AV must be turned on for each service. Other AV parameters are configured at the proxy level and apply globally.
Going Deep to Detect Intruders
With the rise of "blended threats," it's getting harder to differentiate viruses from nonviral exploits. Viruses that slip past AV scans may generate traffic that IDSes can catch. Some products look for CVE-based signatures, attack methods and exploits, while others look for anomalies such as malformed packets, state violations and prohibited values. A few appliances monitor excessive session and packet rates. The tested appliances use one or more of these techniques, some accompanied by preventive actions.
Automated prevention can either save your hide or get in the way. We prefer systems that can bypass hosts and disable IPS when needed. Barbedwire, ISS and Symantec IPS are disabled with one click; Fortinet, ISS and Barbed-wire support host/network bypasses per signature. Secure Computing's "strikeback" responses, which include a command (e.g., whois) and an optional host discard (e.g., ignore traffic from source for "N" seconds), can be as granular as any audit log filter. Symantec can't bypass IPS for selected hosts. And, there's nothing to bypass or turn off in ServGate.
Barbedwire's detection engine is based on open-source Snort. A pair of IDS sensors apply deep inspection to external and DMZ interfaces to spot packets that match attack signatures in Barbedwire's extensive database, which doesn't take preventive action unless modified to do so.
Fortinet uses signatures and/or anomaly detection. As with AV, IPS is applied using protection profiles for each firewall policy. Thresholds, custom signatures and other settings are tuned at a global level.
ISS combines signature matching and anomaly detection, applying issues and actions defined by ISS X-Force. ISS is preconfigured to drop packets, reset connections and quarantine suspicious traffic, Trojans or worms by IP/port/protocol.
Secure Computing leverages proxies and application defenses to provide anomaly detection. For example, a Web application defense can be configured to permit only selected HTTP headers. When the Web proxy detects a denied header, it triggers strikebacks. Custom alarms can be added to the small default set.
ServGate examines multiple packets per session to detect and block viruses, spam and Layer 4 attacks, such as SYN floods. For example, it will ignore subsequent SYN requests during a large SYN scan. But, ServGate doesn't detect Layer 7 attacks, such as buffer overflows.
Symantec uses proxy-based anomaly detection in concert with a signature database. Rules enable detection and "gating" for events grouped into categories (e.g., DoS attacks) for easy on/off. Recommended settings are easily tweaked but have global effects. For example, if you disable detection for a specific DoS attack, it's off for all traffic, not just a particular policy or source/destination.
Attack and Response
To get a feel for the kinds of attacks detected and how they were reported by each appliance, we launched both Layer 4 and 7 attacks.
Layer 4. All six firewalls alerted, logged or silently deflected high-volume TCP SYN, XMAS, FIN, NULL and version scans.
Our UDP/ICMP floods triggered default threshold alerts on ISS, ServGate and Symantec. We tripped Fortinet's alert by dropping its threshold. Barbedwire's firewall provides flood detection, but we couldn't find a GUI threshold adjustment knob. Secure Computing detected only SYN and TCP proxy floods.
Secure Computing, ISS and Symantec issued alerts for random packet floods aimed at ports like DNS, while the three stateful packet inspection firewalls set off alerts no matter which port we tried to flood.
ISS noticed a WinTrin00 probe and Fortinet flagged a Stacheldraht, but, overall, probing a few common Trojan ports didn't raise many alerts.
ISS alerted us to both BO2K Trojan requests and responses but didn't block them by default. Secure Computing and Symantec stopped our BO2K requests to nonexistent proxies but allowed BO2K responses through generic proxies. BO2K is in Barbedwire's database but is disabled by default. Fortinet has Back Orifice in its database and added BO2K in July, well after our test period. ServGate didn't detect BO2K traffic or any other nonviral application payload.
Don't read too much into our sample Trojan test; had we picked other Trojans, the results would have been different. Instead, consider how this example illustrates why a combination of IDS methods is helpful.
Layer 7. We aimed Syhunt, HTTPrint and other attack tools at a DMZ server see if the detection engines picked up application payload attacks such as HTTP directory traversal, CGI bin access, bad HTTP requests, ISAPI printer overflows, password file access and JRun overflow exploits. ServGate didn't detect any of these. Other appliances generated alerts, ranging from dozens (Symantec, Secure Computing) to hundreds (ISS, Fortinet) to more than a thousand (Barbedwire).
Secure Computing and Symantec generated alerts only for attacks on listening proxies, ignoring attacks through other ports. ISS, Fortinet and Barbedwire detected attacks aimed at any port. This default posture can be refined by adding more services to generic proxies or by disabling signatures for unused services.
Secure Computing issued alerts for prohibited HTTP headers and badly formed HTTP requests, including chunked buffer overflows, but not to syntactically valid attacks like HTTP directory traversal. Other products used signatures to recognize these kinds of attacks and, in some cases, combined them with heuristics to reduce false positives/negatives.
Knowledge Is Power
IPS is a balancing act between early prevention and overreaction. To strike a balance, security managers must see what's going on with their firewalls, especially the ones with IDS/IPS. All six appliances can log events locally, relay them to another manager or (in most cases) forward them to a syslog server. All can send e-mail and/or SNMP trap notifications.
During an attack, you want to be notified quickly but not overwhelmed. Secure Computing and ServGate have configurable thresholds for each alarm, while ISS will generate one alarm for a set number of duplicate events. Fortinet sends notifications at priority-based intervals, and Symantec's notifications are on/off by priority.
Event logs and alarm notification details vary:
- Barbedwire's firewall log is basic, but its IDS alarm database, reports and graphs deliver incredible detail.
- Fortinet presents a nice dashboard summary and detailed logs for traffic, viruses and attacks.
- ISS extracts firewall, AV and IPS alerts into a nicely formatted alert log. Active alerts are hot-linked to X-Force descriptions.
- Secure Computing's audit database is complete but cryptic. Records are viewed through the GUI, but we preferred command-line inputs for event watching, reviewing and searching. Audited events (but not triggered alarms/strikebacks) are written to a SQL database.
- ServGate's traffic logs are broken down by session, and packet lists can be viewed by clicking on a session. However, the security logs (IDS alerts) are too brief.
- Symantec's event logs are very thorough and easy to view. Intrusion alerts are extracted to a separate log for deeper examination.
It's Your Choice
All-in-one appliances look similar on the specification sheet, but security managers need to dig deeper to spot the differences. AV is the best illustration here: SMTP is the only universally supported protocol.
Decide which security functions are most important to your business; each multifunction appliance is stronger at providing some functions than others. For example, Barbedwire is thin on VPN but thick on IDS/IPS. ISS prefers to make IPS decisions for you, while Barbedwire provides a framework to add your own; Fortinet plays in both camps.
If figuring out what's happening is a monumental chore, incidents aren't going to receive proper scrutiny or remediation. A midsized business shouldn't require an enterprise management system to monitor turnkey appliance security events and responses. ISS, Symantec and Barbedwire gave us the most readily accessible detail.
We were generally able to get these appliances to do what they are advertised to do. Operator error was encountered far more than bugs. Some of these products--ServGate, Fortinet and ISS--are much easier to use than others.
Picking a winner in such a diverse field is tough. Smaller companies will appreciate ServGate's simplicity and Barbedwire's price, while larger companies will benefit from policy depth and flexibility in ISS, Secure Computing and Symantec. But, considering our criteria and the four applications we required, Fortinet gets our vote.
About the author
LISA PHIFER is VP at Core Competence, a consulting firm specializing in network security and management technology.
Tell us what you think of this article. E-mail the editors with your thoughts at firstname.lastname@example.org