Examining Windows Server 2003 operating system security

Microsoft promised Windows Server 2003 would be "secure by design, secure by default and secure in deployment." We took the wrapper off this new OS to see if it lives up to expectations.

Microsoft hasn't had a lot to show for its Trustworthy Computing initiative. Since launching the campaign to "get secure, stay secure" more than a year ago, the software giant has spent tens of millions of dollars cleaning up the security of its operating systems and applications. What it has lacked is a centerpiece to show off the fruits of its labor.

That changes this month with the release of Windows Server 2003 (Win2003), the successor to Windows 2000. Microsoft has devoted the bulk of its security effort toward making Win2003 "secure by design, secure by default, secure in deployment"--the three "Ds" of Trustworthy Computing.

And it has had good reason to devote so much time, money and resources on securing Win2003. If Microsoft is to erase years of public perception that it's apathetic toward security, and that its products are inherently insecure, the company has to score big with its new OS.

Microsoft is downplaying the importance of Win2003's security improvements, but the user community is looking to see if Win2003 meets the basic tenets of Microsoft's 3-D promise. While few are expecting perfection, many are waiting to see if hackers and security researchers poke holes in the OS the way they did with NT.

To see exactly what Microsoft has done to improve security, we tested an advance version (release candidate 2) of Win2003. We looked at its basic feature sets, architecture, security functions and the security of subsystems.

What we found is Microsoft has definitely raised the bar, but by how much remains open to interpretation.

Architecting Security
Microsoft says it's retiring the Windows NT code base with the introduction of Win2003. Still, the new OS uses the same core NT architecture, and the same object-centric model for controlling access to files, printers and other network resources through access control lists (ACLs).

Win2003's architectural changes are minor refinements that mostly enhance manageability and scalability. Including support for .NET services did require some major changes, but they're mostly add-ons to the OS that don't affect security. But architectural changes really weren't needed, since most of the security problems since the introduction of Windows NT 3.51 are a result of sloppy coding habits, poor implementation of encryption and wide-open default settings.

NT and Win2K weren't secure by default. They had too many services enabled, which exposed systems to many more exploits and attack vectors than necessary. Also, most security controls were left inactive--password policy, account lockout, auditing, anonymous restrictions, and protection of password hashes and authentication packets.

Win2003 has addressed "secure by default," in part. It has many more services disabled by default, including the most dangerous--World Wide Web Publishing Service, which makes the system into a Web server and exposes it to frequently discovered HTTP-related exploits. Win2003 has more security controls activated, but still leaves password policy, lockout policy and auditing either disabled or relaxed.

Table 1: Windows security features

Microsoft did overhaul the built-in Internet Information Server (IIS 6.0), which has been responsible for numerous Windows vulnerabilities over the years (see "Locking Down IIS"). Unlike previous Windows versions, IIS doesn't automatically install with the OS, and its services are off by default when it's activated.

Microsoft improved manageability by introducing more granular permissions modeling and dynamic inheritance than were in Win2K. The permissions work the same in Win2003, except Microsoft has improved the interface and added a management feature called Effective Permissions. This feature allows admins to view the access a user or group would receive on an object without implementing the ACLs and group memberships. Win2003 member servers still have a Security Account Manager (SAM) database, which facilitates local user accounts usable only for the local computer.

Win2003 controls the authority to perform system-level operations (changing the system time, shutting down, etc.). It's the same function that was in NT through user rights, but Win2003 adds some badly needed rights for controlling Terminal Services logons.

Win2003's Active Directory structure remains the same as Win2K--forests, trees, domains, organizational units and sites. Admins can still get to centrally managed security settings on computers throughout the domain with group policy objects. But now multiforest organizations that need to give users access to resources across forest boundaries can create trust relationships between forests that allow an admin to leverage user accounts from a different forest.

Win2003 introduces some additional built-in groups that allow enterprises to reduce the number of admins assigned to the all-powerful "Administrators Group" when they only need a subset of that group's authority. This allows enterprises to closely follow the policy of least privilege and reduce the number of people with potentially dangerous administrator authority.

New and Enhanced Features
Beyond enhancements to the core security architecture, Win2003 has a laundry list of new and enhanced features, including expanded encryption, a built-in firewall, support for .NET Passport, greater Network Address Translation (NAT) support, native support for PKI and increased wireless interoperability ("Windows Security Features").

The Encrypting File System (EFS) that was introduced with Win2K now supports the Advanced Encryption Standard (AES) and adds a feature for encrypting offline files. Win2003 gives organizations the option to use EFS with its data recovery feature disabled.

Some of these new features were introduced in Windows XP, including the imbedded Internet Connection Firewall, a bare-bones firewall for computers connected directly to the Internet; and Credential Manager, a vault for storing Web sites, servers and applications passwords. (Credential Manager's storage is ostensibly secure, but Microsoft acknowledges it may not be appropriate for high-security accounts.)

.NET Passport now can be mapped to Active Directory accounts, providing SSO support for consumers and business partners. Certificate Services, Window's built-in certificate authority, now supports delta certificate revocation lists (CRLs), crucial to large-scale PKIs.

From Win2K forward, Windows automatically hashes passwords in the SAM, which is encrypted by a random 128-bit key. Domain accounts are stored in Active Directory on domain controllers. Win2003 introduces an enhanced capability to control the proliferation of local accounts, a chronic weakness because of poor passwords and a historic lack of centralized controls.

Win2003 has the same nine audit categories as Win2K (NT had seven), which control the types of activity recorded in the security log. Microsoft has made the security log a little less cryptic by providing a text explanation of some of the codes used in event descriptions. The most significant auditing enhancement is with object auditing, which is the tracking of access to files.

Until now, Windows could only tell you that a user opened a file for write access and later closed it, but didn't show if data was written to the file. Win2003 introduces operation-based auditing that closes that gap by recording the first time the user actually performs a given operation while the file is open.

Microsoft engineers are enthusiastic about Win2003's wireless network security features, which allow enterprises to leverage Win2003 components, including Active Directory, Certificate Server and Internet Authentication Server (ISA), to secure wireless LANs (WLANs).

Windows XP and Win2003 help enterprises deploy secure, mutually authenticated wireless networks with their implementation of 802.1X, the IETF protocol that fixes the authentication limitations and encryption key vulnerabilities of Wired Equivalent Privacy (WEP). With support for Protected Extensible Authentication Protocol (PEAP), Win2003 makes it a lot easier for smaller organizations to go wireless by eliminating the need for client-side certificates and the supporting PKI.

On the networking side, Win2003 supports traversal of NAT boundaries for L2TP VPNs, allowing remote users secure access even if they're behind another enterprise's firewall. IPSec breaks when its integrity-checking logic detects the IP address change made by NAT. Windows clients already support NAT traversal (NAT-T) for IPSec (except for XP, which requires the installation of the Microsoft L2TP/IPSec VPN client). Win2003's NAT-T support should allow organizations to get away from PPTP, which is less secure than L2TP. That's the cheapest two-factor authentication you can implement.

Win2003 provides a new feature called Software Restriction Policies (SRP) to address the risks associated with unknown and untrusted software, such as unlicensed applications or hacker tools. It also includes software inadvertently executed by users browsing compromised Web sites, or introduced via e-mails and worms.

With SRP, admins can lock down exactly what software is allowed to run, using restrictions based on hashes, path names, digital signatures and Internet Explorer's security zones. Although highly valuable, SRP requires significant setup and testing to implement.

Click to enlarge

Testing, 3-D Style
New security features are great, but they don't make an operating system more secure. To assess Win2003's security, we looked at it from the perspective of Microsoft's "3-D" mantra--"secure by design, secure by default and secure in deployment."

Secure by design. Win2003's security design is consistent with a secure kernel-based operating system. That's no surprise, since Windows always has been based on a secure OS model, sharing many of the same qualities as Unix.

Exploits of Windows systems over the years are more a result of fundamental flaws in Microsoft's development process than in the OS's architecture. Most Windows vulnerabilities are careless coding errors, such as forgetting to check a call process' authority before performing a high-risk operation. Coding against buffer overflows and obtuse things like double UTF-8 encoded strings requires extra coding and prioritizing security over functionality--something Microsoft has been reluctant to do until recently.

One area where Win2003's security is lacking is in Internet Explorer (IE). Web browsing (and other activities, such as reading HTML-formatted e-mails which use IE to render the HTML) is inherently dangerous, since it executes code from an untrusted source within a trusted network and under the authority of the user doing the browsing. An IE vulnerability is ultimately a Windows vulnerability, because tricking IE into running arbitrary code allows attackers to access any information or perform any operation that an authorized user can. IE needs some analog to the time-honored concept of the kernel/user mode boundary in classic OSes that keeps application code from breaking out of its controlled environment.

Another weakness that has plagued Windows since NT is anonymous connections. Since NT 4.0 SP3, there have been partial options for addressing anonymous risks. Updated handling for how Win2003 finds executables and DLLs makes Trojan horse insertions more difficult. Secure by default. Services constitute the doorways into a Windows system over the network, similar to the role of daemons in Unix. Essential to securing Windows (or any OS) is only enabling required services and ensuring that enabled services are running with the least possible privileges.

Click to enlarge

Prior to Win2003, Windows came out of the box with a number of risky services enabled, including World Wide Web Publishing Service. Most of the services ran as LocalSystem, an all-powerful account similar to root in Unix. Best practice suggests that services should run with the least authority possible. If the service is compromised, the attacker's options are limited.

Microsoft promised that Win2003 would ship "with more than 20 services turned off by default or running with reduced privileges." In the pre- release version we reviewed, 16 services run under either LocalService or NetworkService--two limited-authority accounts created in Win2003 to mitigate the potential damage of a compromised service (see "Authority Accounts").

Another 10 services are disabled in Win2003 that were configured for automatic or manual startup in Win2K or XP (see "Disabled Services"). Some of the services pose little risk, and turning them off doesn't do much to improve security. Others are much more significant. For instance, if the SMTP Server service had been disabled by default in Win2K (as it is in Win2003), three SMTP-related security vulnerabilities would have posed no risk to default Win2K installations.

Through default settings, Microsoft cleaned up a number of other longstanding issues. Win2003 blocks user accounts with blank passwords from remotely connecting, which addresses part of the risk posed by insecure local accounts found in NT and Win2K. And Win2003 allows enterprises to rename, disable or delete the Administrator account on member servers and workstations--something that simply wasn't possible in NT or Win2K. While Win2003 won't stop admins from setting up an Administrator account with a blank or default password, it will at least warn them of the security risks.

Introduced in Win2K, Kerberos remains the primary default authentication protocol. However, NTLMv2, the authentication protocol based on technology developed in the 1980s for LAN Manager and used extensively in NT, remains in place for legacy system support and some domain authentication functions.

Prior to Win2003, all nine auditing categories defaulted to off, which left users with empty security logs. Win2003 has auditing enabled by default for six of the nine categories, but only for successful security events. Unfortunately, this configuration doesn't record failed logons. From a forensics and analytics perspective, it would have been more useful to record fail logons by default.

Another first is that default permissions for root directories are no longer "Everyone Full Control," which would have prevented at least one security bulletin in 2002.

Of the 53 Microsoft security bulletins published in 2002 that affected Windows NT, Win2K and XP, 16 wouldn't have been a threat to an out-of-the-box Win2003 installation. The ratio is even better when IE vulnerabilities are excluded, but they pose a threat to the server OS if an admin browses the Web from a server (this doesn't include thin-client scenarios, where Terminal Server users browse the Web via a remote desktop session). Following just a few best practices cuts the list of immediate threats to 17--not bad.

This isn't to say that evaluating the security of Win2003 or any other software should be based on the number of vulnerabilities. The raw number of vulnerabilities is meaningless unless they are evaluated against criteria that distinguish probable and viable risks from those that aren't serious. Software producers, security organizations and media outlets can provide some analysis, but organizations must measure their exposure by comparing threats against their environments.

Secure in deployment. Win2003 is more secure out of the box than NT or Win2K, but that shouldn't lull users into a false sense of security.

Secure by default is sort of a "one-size-fits-all" philosophy, but that doesn't apply to an operating system that scales from SOHO environments to enterprise data centers, and from bastion hosts exposed to the Internet to servers warm and cozy on an isolated network. Each enterprise has different requirements for its networks, which in turn affects security.

Microsoft is making a huge effort to provide information and tools for deploying its products securely, efficiently and dependably. Two of those initiatives are the Microsoft Systems Architecture (MSA) and Microsoft Solutions for Security (MSS).

MSA is a cookbook for constructing data centers with Microsoft and partner products. It covers everything from switches and storage systems to mouse and word-processing applications with configuration settings, test scripts and deployment guides for each component gleaned from real networks set up in MSA labs.

The goal of MSS is "the creation of security- focused guidance centered on the Microsoft platform." One of the projects in development is the "Windows Server 2003 Security Guide," which Microsoft plans to release with Win2003. Similar guides have been released for NT and Win2K.

In the past, Microsoft got away with generating tools and information, but left it up to users to implement them. These initiatives show Microsoft realizes that it must go further than just adding new security features that attempt to compensate for poor care and feeding.

Meeting Expectations?
Despite the rhetoric, Microsoft hasn't completely abandoned usability in favor of security, but it's much closer to where it should be. Most of the new features don't do much to prevent the exploits that have plagued the Windows OS family over the years. That said, the new Software Restriction Policies stands out as something that could reduce the viability of many exploits by preventing the execution of unauthorized code. Otherwise, most of Microsoft's efforts to make Win2003 more resilient to exploits are changes to default configuration and aids to deploying Win2003 securely.

Win2003 is more secure by default with regard to things like open ports, the information it broadcasts to anonymous connections and NTLM-related vulnerabilities. But it's still up to admins to configure the fundamental controls, such as strong password policies, account lockout and audit policy.

Only time will tell if there are holes in Win2003. Microsoft has made significant strides in making this latest version of Windows more secure out of the box, but that doesn't ensure security. Ultimately, the most effective way to mitigate security risks is locking down each Windows system according to the organization's environment and operating requirements.

Microsoft has made substantial progress with delivering free tools and guidance for achieving security in deployment. As one Microsoft representative said, this makes Microsoft and Win2003 users partners in securing Win2003 and other applications.

About the author:
Randal Franklin Smith is president of Monterey Technology Group, a consultancy that specializes in securing Windows environments.

This was last published in April 2003

Dig Deeper on Windows Security: Alerts, Updates and Best Practices



Find more PRO+ content and other member only offers, here.

Start the conversation

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.