Information Security

Defending the digital infrastructure


Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

GRC Tools Help Manage Regulations

We look at three GRC products and the distinct ways these tools can help organizations navigate the complicated regulatory game.

A decade ago, regulated industries were the rare exception; today, the industry that isn't regulated is the exception....

In fact, most firms have multiple sets of regulatory requirements they need to address.

As the regulatory burden increases, businesses are finding themselves in an increasingly complex ecosystem of governance--we audit our contractors and clients to ensure their compliance to our security requirements, and the firms we service audit us.

As we implement security controls related to compliance, as well as controls contractually required of us by our clients, we put into production an ever more complicated laundry list of security controls to manage. Making risk decisions in this hive of controls, regulation and contractual obligations is nigh onto impossible.

IT governance, risk and compliance (GRC) tools promise to help us meet these challenges. They promise to help us make smarter risk decisions, manage our compliance efforts and govern everything about our security program, from security awareness to technical controls.

GRC is the latest information security buzzword, but marketing hype is doing a disservice to this array of products that address an organization's policy governance, risk management and compliance needs. Most deliver only part of the picture they promise, and every tool in this market has its own focus, areas of maturity and strategies for solving the same business challenges.

To help you figure out what approaches might be right for your organization, Information Security took a close look at three GRC products that are very different in focus, coverage and technology: Archer Technologies' SmartSuite Framework 4.1, Symantec's Control Compliance Suite 8.60 and Modulo's Risk Manager 5.0

Our goal was to create tests that address the promise of GRC while not favoring any particular technical strategy for getting there. We wanted to test the heart of GRC, the products' ability to:

  • Author, distribute and map policy and controls to the governing regulation, as well as to keep track of exceptions to those policies/regulations (compliance)
  • Assess the proper technical and non-technical operation of controls, and to mitigate/remediate areas where controls are lacking or not operating properly (governance)
  • Assist in quantification, analysis and mitigation of risk within the firm (risk)
Promising GRC Products

Purchasing a GRC product is difficult, so we designed a flexible testing approach tied to real-world deployment scenarios to account for the range of corporate requirements, the expansive nature of the products and their varying levels of maturity. To do this, we foremost wanted to create a set of hypothetical scenarios that simulate how most organizations would typically use and deploy GRC products. We drew on real-life experiences and pain points to create regulatory, oversight and technical challenges, such as any organi- zation might face, and how the products might solve these challenges in a typical deployment context. Specifically, our goal was to test the "promises" of GRC (see "'Promising' Products").


We evaluated how these GRC products might facilitate compliance efforts by determining how they can help organizations understand, record and document where and how they meet specific regulatory requirements. How do they help you author policy, map regulatory requirements to policy, and, in turn, map specific technical controls to that policy? We also looked at the ability to create highly granular policies. For example, can you map a specific technical control on a particular server all the way back to the driving requirement for that control?

We created test policies and attempted to link those policies to both the regulatory requirements as well as technical controls used to implement the governing policy. In other words, can you actually use the tool to track compliance activities, track the implementation of technical controls specifically required by the regulation, and track the operation of those controls in the field.

Risk management

Analyzing business risk is tough enough, but regulatory requirements add a layer of complexity that is fueling the market for specialized tools. Think of your own environment, where the data required to determine what risk applies to a particular set of devices, applications or processes is probably spread throughout the company.

For example, in order to understand what risk applies to a legacy system that doesn't support a particular control, you need to know what the system does, how it's used, what compensating controls might be in place and what systems are dependent on it. Typically, that means getting data from the business, IT, external parties (such as service providers) and the compliance department.

A tool that can automate this process and preserve the information gathered in a central repository is essential to conduct formal risk analyses. To this end, we looked at the ability of the products to help gather data about particular systems/processes and their relative risk, evaluate that risk and put it in context. A key related area is the products' ability to record and track areas of the firm where technical controls could not be implemented, as well as features that analyze the level of risk associated with those exceptions.

Technical controls

Finally, we considered how products manage the many technical controls that firms might be interested in from a compliance and governance perspective. We assumed from the get-go that different products would have varied ways to monitor controls. For example, a product might use an agent on the remote host to periodically poll the device, and/ or import data from other sources, such as vulnerability assessment tools to gain information about the status of system and application controls. The bottom line: Does the product provide enough information and the right kind of information to be of real use?

Archer SmartSuite Framework 4.1

Archer focuses primarily on the non-technical aspects of GRC. The core of the product is a central framework within which a customer can use various modules that target the issues that an information security practitioner might experience within a regulated industry. For example:

  • Policy Management addresses the need of information security practitioners to author, organize and publish policy.
  • Vendor Management provides tools to manage vendor relationships and track adherence to contractual obligations.
  • Incident Management allows you to manage the workflow associated with a security breach.

There's no installation to speak of, as the recommended customer interface is a Web portal for an ASP-type service offering. (Customers can also choose to host the product.)

Admins use their interface to create users and groups, modify roles, permissions and security parameters of the system, manage content, or change the appearance of the portal. However, the real magic happens within the customizable interfaces for the installed modules.

Archer's strengths

Policy management is a real strong suit. To test how SmartSuite would be used in an actual organization, we wanted to create policy we could tie to the regulatory requirements of our hypothetical company.

However, we didn't need to. Archer ships with a stock policy that is pre-mapped to a large number of regulatory frameworks. Given Archer's roots in the financial services sector, we were not surprised to see some relatively specific requirements such as FTC 16 CFR Part 314 (GLBA) and the FFIEC Information Security Booklet. We were, however, pleasantly surprised to also find more general guidance, such as COBIT and ISO 17799 (although they still need to update the numbering), as well as specific guidance for other regulated industries, such as HIPAA for healthcare and PCI for retail.

Although the stock policies are quite comprehensive, most firms will need to modify them to reflect their own requirements. We found this process a bit counterintuitive. The editing function allows you to directly modify the policy supplied by Archer, but you're better off avoiding that and using Archer's somewhat kludgy alternative.

The problem is that Archer's periodic updates to the stock policy (as well as the mapping to the regulatory frameworks) will overwrite any custom changes you make to the stock policy directly. Archer recommends that instead of modifying its policies, you create a new policy statement with customized text, link it to the stock policy, and update your firm's views to display the new statement. The stock statements live on--just out of view of the users.

The upshot is you will need to periodically revisit your custom policy to ensure that it reflects updates, such as changes to regulatory requirements.

Nevertheless, exceptions are easy to create and relatively straightforward. You simply select a control to associate with the exception and enter information along with compensating controls to address the issue. The workflow allows exceptions to go from user entry to information security review and keeps track of approvals and timeframe for expiration.

The risk management feature is straightforward. You assign risk to entities entered via the asset module and score them according to a number of different risk vectors. For example, we used the asset module to create a new application and assigned an initial business criticality weight as well as risk profile (high, medium or low.) From there we were able to apply questionnaires to the asset to determine how it performed relative to items of interest, such as whether cryptography was employed.

These questionnaires targeted specific controls that have an impact on the overall risk of the application and include factors like vulnerabilities, cryptographic controls, access control, and so on. The responses to the questionnaires fit directly into the overall risk ascribed to the application. The workflow ensures that appropriate personnel review the submission and are alerted if it is completely noncompliant.

In navigating and using SmartSuite, we found the Archer community to be head and shoulders above what you typically get with a vendor knowledge base or other support portal. The community allows users to interact with each other, ask questions of the Archer engineering team, and receive extensive training on use and configuration of the product.

Archer's weaknesses

While the product was very strong in policy and risk management, the more technology-centric pieces are not as automated as the other products. There's no autodiscovery function--you add assets by submitting a spreadsheet. While this will satisfy the needs of many organizations, larger firms with extensive asset inventories may find this process error-prone and difficult to maintain.

Monitoring technical controls is also less automated than some of the competition. Archer provides instructions on how to create linkages between automated vulnerability assessment tools (e.g., Qualys), but automated vulnerability assessments may not give you the whole picture. There's little out-of-the-box integration of additional tools, such as other vulnerability assessment scanners, IPSes, SIEMs, etc., but you can use the flexible API to allow custom data consumption applications to be written using feeds from files/databases, etc.

One nice feature lets you correlate information from a number of threat publication sources, such as Verisign iDefense and Symantec DeepSight, in addition to custom entry of threat data.

While Archer is heavy on policy management, Control Compliance Suite 8.60 (CCS) has a deep focus on the management and monitoring of technical controls, providing quite a bit of functionality to assist in tasks like network discovery, automated validation of host technical configuration, and so on.

The software can be installed in standalone or enterprise mode, depending on whether you intend to host the database on the same box as the information server or use a different box for the database. Additionally, enterprise mode is required if you intend to make use of the Web portal integration with Microsoft IIS. We installed the product in enterprise mode, as this allowed access to the Web portal and supported a remote database and remote data collection.

Symantec's strengths

Symantec Control Compliance Suite

At first, we were a bit skeptical about the policy creation interface (not the prettiest interface we've ever seen), but using it to write policy was straightforward despite the initial awkwardness.

We were able to author policy, import existing policy from Microsoft Word documents and approve publication to the CCS Web portal. The tool supports a policy authorship workflow in much the same way Archer does, allowing us to defer publication until approval and to keep a recorded archive once a new version is created.

Surprisingly, we found ourselves missing the kind of stock policy supplied by Archer. Symantec has a number of sample policies (templates), but we found that importing our own policies or creating new policies from scratch using the policy import and creation tools took less time than customizing the templates.

One feature that really stood out was the flexibility provided to map policy to the compliance frameworks and regulations CCS provides. The mechanism is a mapping editor that's reminiscent of the relationship manager feature of Microsoft Access. Though it took us a while to figure out how to use it, the mapping editor provides tremendous flexibility in making connections between policy, framework and regulatory items. The ability to see these relationships visually had a definite "cool factor." Of course, while this is a flexible approach, it requires a bit of manual interaction to maintain. An enterprise seeking to make heavy use of the policy portion of this tool would require more ramp-up time to get ready for full deployment.

CCS is very strong on technical controls. The product ships with a large number of technical standards packs that can be used as a benchmark against which to compare devices that it is aware of. The standards packs draw on familiar source material, such as NSA configuration guides and the CIS configuration benchmarks.

The technical information-gathering feature supports a very large number of devices for remote profiling. CCS can use an agent or agentlessly retrieve data across a diverse range of platforms, such as various Windows versions and multiple flavors of Unix and Linux.

The product also ships with many benchmark standards to evaluate against, to ensure that appropriate patches are applied and that appropriate configuration steps are taken.

CCS also ships with network mapping capability that allows automatic discovery of devices, which can then be imported into the risk management and asset management view.

We expected CCS to perform very strongly in technical controls validation, but were unprepared for the product to perform equally well in policy and risk management. We were pleased to see the range of technical standards and regulatory frameworks that the product ships with: multiple versions of COBIT (both 3 and 4), FDA regulations, FISMA, HIPAA, NERC (North American Electric Reliability Corp.) guidance and NIST SP 800-53 were all included.

Symantec's weaknesses

While CCS is heavy on governance and technical compliance, we found the risk piece difficult to use. Technical risk is assessed directly within the CCS console via evaluation of technical security controls; for the less technical areas, the product leverages customized questionnaires.

CCS allowed us to import Symantec's questionnaires using content packs or create our own. We used the tool to create an ad hoc vendor evaluation, and found the process painful.

Each questionnaire is represented as a tree view to which questions are added. Questions can require single or multiple-choice answers, or written responses. Creating a questionnaire required us to manually enter a large number of customized answers (the templates, which were fine for yes/no questions, rarely supplied the answers we needed).

Once the questionnaire was complete, we used a wizard to assign weights to each of the questions and answer choices. All told, the process took us about an hour to create a 20 questions. If you're planning to make extensive use of this functionality, we recommend using the content packs that supply stock questionnaires rather than creating customized questionnaires from scratch.

Many vendors in the GRC space try to take the "boil the ocean" approach by being everything to everybody. Not Modulo. It doesn't have the compliance-framework creation and policy-centric features of Archer, or the technical control validation capabilities of Symantec. Instead, Modulo's aptly named Risk Manager focuses almost exclusively on the risk aspects of the GRC equation. The functionality within the other areas of GRC serves only to support the risk management mission.

Risk Manager does not have a Web front end (although you can submit questionnaires via the Web), and relies on a number of client-side applications to implement various features.

The installation process gave us quite a bit of trouble initially. Insufficient RAM on the first few lab machines we attempted to install caused the installer to fail (the test machines had double the memory requirements specified in the manual). However, with some coaching from the Modulo engineers--followed by a hardware upgrade beyond the recommended requirements--we completed the installation.

Modulo's strengths

Modulo Risk Manager 5.0

Risk Manager allows enterprises to categorize themselves into one or more "organizations" that are represented by a tree view in the client. It auto-populates this view by importing information from a variety of sources, such as Active Directory, asset spreadsheets/ databases and manual entries.

Its real power lies in its ability to categorize every asset in the organization--processes, applications, technical components and facilities--associate a risk level to each, and keep track of the controls that are implemented on an asset-by-asset basis. The tool also facilitates keeping track of personnel associated with the assets and threats to it.

Risk information is collected using one or more questionnaires applicable to different assets, based on their categorization. For example, data centers can be assigned one or more data center-specific questionnaires to appropriate personnel. Risk Manager gathers information about all the assets in a particular scope and quantifies the associated risk, keeping track of controls' status on an asset-by-asset basis.

You can link evidence with particular answers as well. For example, to support a response to a questionnaire about authentication, you can attach evidence in the form of policy, an export of the appropriate group policy objects governing password characteristics, and so on.

This ability to associate evidence with questionnaires should please auditors, who require proof of a particular control, rather than simply validating that a governing policy exists.

Auditors will also appreciate the ability to generate remediation plans for particular assets based on the results of the questionnaires. The remediation guidance provided for each of the assets in scope is concise, yet thorough.

Risk Manager facilitates governance of vendors and external relationships in a way the other products do not. For example, Risk Manager ships with the ability to perform a risk assessment using the Financial Institution Shared Assessments Program Standardized Information Gathering questionnaire. It also allows you to create "perimeters" (nodes on the organizational tree) for vendors and third parties. While the other products can be configured to do similar things, native support for FISAP out of the box is a real plus for organizations who use Risk Manager in an auditing context.

Other questionnaires can be assigned to assets within the vendor perimeter. This enables you to keep track of assessments performed of a particular vendor, the evidence collected during the assessment, the vendor's compensating controls, etc.

Modulo's weaknesses

Risk Manager has a few rough edges. First and foremost, the lack of a fully functional Web interface is a significant drawback. While questionnaires can be submitted over the Web, a portal view of the application (including a Web-enabled dashboard) was a sorely missed feature and would provide quite a bit of benefit.

Additionally, installation was challenging; the application has very specific installation prerequisites, and any failure of the installation process (due, for example, to lack of a prerequisite, insufficient memory or a populated database instance) resulted in an error message that required technical support to interpret.

Further, the product appears to be difficult to customize. For example, some of the built-in databases (such as the threat database) are static, precluding user customization.

One size doesn't fit all

Each of the products we looked at interprets governance, risk and compliance in a different way and has a feature set tailored to its vision. Archer emphasizes regulatory compliance, most useful for the compliance or security group in a heavily regulated industry. Modulo focuses on risk management, which is of special value to the auditor or consultant out in the field validating organizational compliance to controls. Symantec focuses on technical control validation, most useful to information security technical personnel.

But in order to know how the vendor interprets the GRC vision, you must look beyond the marketing. All of these products are marketed similarly; they get coverage from analysts in the same reports and they're lumped together in the industry press. But they're really very different.

What does that mean to the industry? Maybe we should start segmenting the GRC market to reflect the fact that these products aren't the same. What does it mean for GRC vendors? Maybe it's not a threat if your product doesn't do exactly the same thing as the other guy's product. And what does that mean for the consumer? It means you need to be extra careful before you buy: Make sure your vendor's vision of the market aligns with yours, and that the product you're buying does what you think it will.

It's time to recognize the industry's best

Information Security magazine and SearchSecurity will honor innovative security practitioners in seven vertical markets this fall with our annual Security Seven Awards. The awards, to be handed out at the Information Security Decisions conference in Chicago and featured in the magazine's October issue, will recognize the efforts, achievements and contributions of practitioners in financial services, telecommunications, manufacturing, energy, government, education and healthcare.

While vendor executives are not eligible, we're inviting you to nominate your most innovative practitioners. Nominees must have made a noteworthy contribution to their organizations or the security community in areas including research, product development and standards.

Download the nomination form at
and email it to
Nomination Deadline: June 25

Article 10 of 14

Dig Deeper on Risk assessments, metrics and frameworks

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.

Get More Information Security

Access to all of our back issues View All