Poor change control can send your organization's security tumbling. Follow these 5 steps
for a strong change management program.
Mysterious activity shows up on key network devices at 2 a.m., setting off alarms in the network operations center. You're blocking Telnet and outside connections to the network, so this activity makes no sense. After 15 minutes of scrambling for answers, you find those connections weren't blocked at all. Earlier in the week someone made a change to grant network access to a vendor's technician, unintentionally creating a nightmare for you.
This simplistic example illustrates an important point: all too often, inadequate change control is the root cause of many headaches and sleepless nights for secu-rity managers. Keeping networks locked down is incredibly difficult because networks are in constant flux; patches must be applied, partners must be granted access and new services must be provisioned. One unmanaged change could send the neatly aligned and secured blocks of your network tumbling to the ground.
Change management can mean a number of things to security managers. To some,
Requires Free Membership to View
SearchSecurity.com members gain immediate and unlimited access to breaking industry news, virus alerts, new hacker threats, highly focused security newsletters, and more -- all at no cost. Join me on SearchSecurity.com today!
Michael S. Mimoso, Editorial Director
change
management constitutes the processes in place to enact changes in the IT environment, such as
new firewall rules or router access control list entries and system upgrades. This process includes
approvals, testing and scheduling. Too frequently, though, IT security teams equate the concept of
change management with configuration control and management, which includes activities like file
integrity monitoring and system scanning. Although they are distinct concepts, both are vitally
important to security, and the ties between change management and configuration management are
getting stronger.
Security's challenge is to integrate itself into existing IT change management processes, while developing change and configuration management processes for systems and devices it controls. Organizations must include security to help evaluate requested and planned system changes and advise business management of associated risks. Getting a seat at the table is becoming easier because organizations understand the risks some IT changes could introduce.
However, the security team should act as a trusted adviser, not a dictator of whether changes are allowed or not, and demonstrate willingness to work with business owners and facilitate business initiatives.
Let's examine security's role in five key steps for a well-developed change management program.
1. REQUEST & APPROVAL
The initial phase of a change management process is the request for change, or RFC. In most
cases, this will be entered into a change management application and submitted for consideration by
the change advisory board (CAB). Usually this board includes representatives from the help desk,
network operations, servers and system administration, security, internal audit, database
management and application development. Other members may represent legal, operations and
administrative departments.
Once the CAB allows the RFC to proceed, it must progress through a chain of approval that allows all stakeholders involved to determine whether the change should be executed. Information security teams should be involved in the approval process, and make approval decisions based on perceived risk to the organization for the requested change. This initial phase of the change management process should have a built-in feedback loop, so approval workflow members can ask the change requestor questions or discuss ramifications with other stakeholders.
Security must:
- Ensure the request comes from a valid source that is authenticated and authorized to request a change by checking user accounts and repositories, identity management systems and log files.
- Establish an audit trail for change requests--who made the request, who logged in to the change management system to review the change, who was involved in approval, and who made comments on the change.
For security teams implementing changes for their own systems, such as IPSes and firewalls, important in-formation to provide in the RFC includes the actual technical changes to be made, systems to be affected, and the possible risks to those or other systems in the environment. Potential risks resulting from not implementing the change should also be recorded.
Takeaways
REQUEST & APPROVAL
Security's Role...
- Authenticate change requestor
- Establish an audit trail for requests
2. PLANNING & TESTING
The planning phase of change management is where real scrutiny of proposed changes occurs.
During this phase, security teams should assess in detail the change's impacts to the
organization's risk posture. For example, security may determine that granting network access to a
partner would open critical resources to new threats, or that a software implementation is risky
due to known vulnerabilities.
Implementation and back-out plans should be provided by the change requestor and any other groups
involved in implementation. These should also be assessed in some detail by security personnel to
ensure that organizational policies are adhered to and that the plans are reasonable. For
security-related changes, the planning phase should also include a determination of personnel that
will need to be involved in the change, and very specific criteria used to determine if back-out is
necessary.
The testing phase in-volves lab testing and simulation of the proposed changes and back-out plans. Standard organizational policies and procedures regarding lab testing should be adhered to, and the results of testing should be clearly articulated to all members of the CAB. For changes involving other groups' systems and applications, security teams will often perform some form of risk assessment after standard functionality testing is complete. For example, they might perform a vulnerability scan against new systems or applications. Security teams testing their own changes and back-out plans for security devices and applications should also follow approved lab testing procedures.
Takeaways
PLANNING & TESTING
Security's Role...
- Examine risk associated with changes
- Ensure back-out plans are within policy
- Perform risk assessments on changes in a lab setting
|
3. SCHEDULING & COMMUNICATION
Scheduling changes is straightforward. All interested and affected parties should make sure the
changes are scheduled to take place at a reasonable time. In many organizations, there are standard
change windows--usually late night or early morning when fewer people are working and less business
is being conducted--and these should be used whenever possible.
Effective communication is vital here. The team in charge of implementing the change should communicate by email and phone that the change is occurring and specify when it will occur. Who should be notified will differ from one organization to another, but an excellent rule of thumb is to contact everyone on the incident-response team's notification list, often referred to as a "call tree." Security teams should be notified of impending changes, as should IT operations teams and other groups that may be affected. In the case of security changes, security crews must contact everyone.
The key to successful change scheduling and notification is to turn it into a well-documented process with checklists. With some change management tools, this functionality is a standard component and can usually be accomplished within the application workflow, which in turn is linked to enterprise contact repositories.
|
Takeaways
|
4. IMPLEMENTATION
During this phase, changes are implemented according to plan. Security team members will either
have a standard on-call rotation where someone stands by, or they will be implementing the change
if it is related to security systems or applications. Any changes deemed capable of causing
significant security risks, or changes to mission-critical systems, should be monitored carefully.
Security teams should also leverage monitoring solutions like IDS during changes when
possible.
For changes that are problematic or need to be backed out, reaching staff members can be difficult during the standard off-hours change windows, which makes it vitally important that security personnel be available in some capacity
Takeaways
IMPLEMENTATION
Security's Role...
- Be on call, especially with problematic changes
- Monitor changes that could add risk to systems
5. DOCUMENTATION & FOLLOW-UP
Documentation of all changes is critical for maintaining a sound audit trail. All configuration
changes should be documented in a centralized database, and all approvals, comments, testing plans,
back-out plans and other workflow-related documentation should be retained according to
organizational policy. One reason for doing this is regulatory compliance (see "E-commerce
Control," below). For any systems that store, process or transmit sensitive data that is
covered under mandates like Sarbanes-Oxley and the PCI Data Security Standard, auditors will often
ask to see documentation for all changes made since the last audit. Audit trails of changes also
are needed for troubleshooting systems and applications, as well as security investigations and
incident-handling procedures.
Follow-up meetings or analysis should occur whenever changes do not proceed as planned or a back-out is necessary. Any security issues caused by changes, such as data exposure or the introduction of system and network vulnerabilities, should be the basis for follow-up meetings led by security staff.
Takeaways
Documentation & Follow-up
Security's Role...
- Maintain audit trails for compliance and troubleshooting
- Lead follow-up meetings regarding security problems
Case Study
E-commerce Control
Retailer deploys change-control software for PCI compliance and security.
The Payment Card Industry Data Secu-rity
Standard led Restoration Hardware to implement software to control changes on its e-commerce
platform.
PCI requirements 10.5.5 and 11.5 call for deployment of file integrity monitoring software. In
looking for ways to comply, Restoration Hardware--an upscale home furnishings retailer based in
Corte Madera, Calif.--looked at several products but chose Solidcore S3 Control, which tracks
changes in real time and enforces change policy.
"We needed to be compliant, so Solid-core allows us to satisfy the file integrity check portions of
PCI," says Bobby Wen, manager of IT operations. "We got the security as a bonus."
The software prevents unregistered binaries from running, so it performs as an intrusion-detection
and prevention system, he says. If an intruder manages to penetrate a system, he would not be able
to upload a binary and hijack it. That security was what the retailer wanted, but the need to
comply with PCI was ultimately what got the project funded, he says.
Solidcore easily integrated with the retailer's change management system, ControlTier, and ensures
that change policy is followed, both from a timing and content perspective, Wen says.
Restoration Hardware deployed the software in its e-commerce platform, primarily in Web and
application servers that handle credit card transactions. The software is comprised of agents
installed on systems and a central console for administration.
Since Restoration Hardware was an early adopter of Solidcore, it had to do a little explaining of
the new technology to auditors, who have since "blessed" the software as satisfying the PCI
requirement, Wen says. With Solidcore's reporting functionality, the retailer can show auditors
graphical views of system changes instead of simply showing them logs.
The company plans to expand use of the software beyond PCI compliance to Sarbanes-Oxley
compliance.
--MARCIA SAVAGE
STUMBLING BLOCKS
For security teams, there are a few points to keep in mind to avoid pitfalls with change
management:
- Understand the distinction between change management and configuration management. Change
management is a process framework, aided by technology. Configuration management is more often
equated with tools themselves, like a software agent on a server that monitors critical files for
changes or a configuration database (see "Tools for Change," below). However, well-designed
configuration management can greatly facilitate proper change management for security teams.
- The biggest obstacles to achieving effective, risk-focused change management are people and
processes, not technology. Focus on working with peers in different IT groups and business units to
understand security's role in the change management workflow. "Security should become more of an
'active partner.' Tools should be less of a discussion than the policy and process creation," says
Kris Brittain, a Gartner vice president.
- When you evaluate configuration monitoring and implementation tools, look for those that can be integrated into organization-wide change management products. Don't create silos for security and security products.
Although configuration monitoring tools are beginning to integrate with change management products, convergence of the two is a ways off. For now, security professionals should focus on policies and processes, refine internal processes for risk analysis of proposed changes, and ensure that proper audit trails are kept of all steps in the change management workflow.
Products
Tools for Change
Several products help streamline change and configuration management.
A variety of tools exist that perform
centralized change and configuration management functions.
Some combine traditional patch management with configuration management options via agents or
appliances that can inventory systems and applications, apply and monitor patch levels and control
configuration details. These include BigFix Enterprise Suite, Configuresoft Enterprise
Configuration Manager (ECM), Kace's KBOX Systems Management Appliance, BladeLogic Operations
Manager and Lumension's PatchLink.
Other tools are more concerned with policy management and configuration monitoring, and perform
file integrity monitoring and policy enforcement based on an organization's specific settings.
Organizations define baseline configurations that map to policy, and once applied, the tools
monitor for changes. These tools can either alert or automate configuration and remediation based
on policy. These include Tripwire Enterprise, nCircle Configuration Compliance Manager and
Solidcore S3 Control.
A third type includes traditional change management products that may integrate help-desk
ticketing, change request and approval workflow tools, and built-in auditing functions. These
include BMC Remedy, HP Change and Configuration Center, and IBM Tivoli Change and Configura-tion
Management.
--DAVE SHACKLEFORD
This was first published in May 2008