Database activity monitoring (DAM) has emerged as a powerful and effective tool for security and compliance. By design, DAM technologies have the ability to monitor all database activity, including administrators, and alert on policy violations. These features enable compliance controls, operations monitoring and data protection not otherwise possible, and does so without interfering with business processes. While the promise to advance security and compliance is significant, not all tools are created equal, with fundamental differences in architectures, database support, blocking capabilities and performance.
We'll discuss the business use cases, explore the inner workings of these tools, and make recommendations on evaluating, purchasing, and deploying database activity monitoring. We'll provide a definition for DAM that explains how it differs from database auditing and intrusion prevention systems, and then illustrate these difference in applied use cases for compliance and security tasks. We will then drill into the technology as to understand the difference between network, external monitoring and agent architectures, evaluate the major features of DAM, including workflow and advanced capabilities such as change management, and determine deployment expectations to aid buying decisions.
WHAT IS DATABASE ACTIVITY MONITORING?
Database activity monitors captures and records database events, which at a minimum includes all Structured Query
Further, the ability to analyze use of objects, user behavior, volume of data, source and destination, application and content means we can apply compliance and security policies in a very granular and precise way. While a number of tools can capture various level of database activity, database activity monitors are distinguished by several features:
- The ability to independently monitor without help from database administrators to collect information or enforce policies.
- Collecting activity from multiple sources in and around the database.
- The ability to apply multiple forms of analysis and react in near real time.
- The ability to store this activity securely outside the database.
- The ability to aggregate and correlate activity from multiple heterogeneous database management systems (DBMS).
- The ability to enforce separation of duties on database administrators, auditors and security personnel.
While commonly confused with database auditing, the ability to collect from multiple databases of different types, the capture of SELECT statements to understand how data is viewed and not just changed, and capture important system alterations not stored in the audit trail sets DAM apart. When coupled with real-time analysis of content and behavior, not just attribute-based reports, dozens of new uses for controlling and securing databases and data are possible.
WHAT IS THE BUSINESS VALUE OF DAM?
Let's move beyond the technical nitty-gritty and jump into the real reasons you consider database activity monitoring: the business problems it solves. Database activity monitoring tools are very flexible, and purchases are typically prompted by one of the following drivers:
* Auditing for compliance. The single greatest adoption of DAM has been increasing auditor requirements to record database activity for Sarbanes-Oxley (SOX) compliance. Despite the fact that nowhere in SOX or accounting guidelines does it specify the need for database monitoring, accounting fraud has evolved to bury evidence within the millions of daily transactions that occur in automated financial systems. Bogus general ledger and accounts-receivable entries are easily lost in the vast sea of transactional information. Prompted by the discovery of fraud in the WorldCom scandal by scanning raw database activity, companies have demonstrated the ability to implement financial controls at the database level. This type of monitoring is simply not possible for auditors to manually accomplish, both because of the volume of activity, and their inability to navigate the complexities of the database system. Some external auditors recommend the collection of all database activity for SOX, and DAM tools can do this with less overhead and cost than alternatives.
* As a compensating control for compliance. Monitoring adoption is used to complement Payment Card Industry's Data Security Standard for access to credit card related information, as well as monitoring access to sensitive health care data as it relates to HIPAA requirements. We are seeing greater use of DAM tools to address specific compliance requirements, even though database auditing itself isn't the specified control. While access controls and encryption are specified technologies, their respective cost, impediment to normal business operations and lack of ability to verify data usage has prompted firms to support other security efforts with DAM, or replace them entirely.
• As a security control. DAM tools offer significant security benefits and can sometimes even be deployed in a blocking mode. They are particularly helpful in detecting and preventing data breaches for Web-facing databases and applications, or to protect sensitive internal databases through detection of unusual activity. Advancements in statement analysis of the DAM platforms makes it possible to detect and stop SQL Injection attacks, as well as insider misuse. As both of these attack vectors remain the biggest security threats to databases, the relevance and value of DAM for security is unquestioned.
WHAT ARE SOME EXAMPLE USE CASES?
The following are several use cases that illustrate how database activity monitoring is used to enact security, operations and compliance policies:
Security: Security is the reason DAM products exist. The ability to detect and respond to any activity that appears malicious or where the database is being mis-used, either by attackers or insiders, was missing from the market. Some of the things DAM can sniff out include:
- If an application typically queries a database for credit card numbers, a DAM tool can generate an alert if the application requests more card numbers than a defined threshold (often a threshold of "1")
- Detection of SQL injection variants to confuse the database into revealing information, or allow the execution of arbitrary code.
- Recording failed logins and other events that indicate an attack or attempted mis-use.
- Blocking unwanted statements
- Application white listing by blocking connections of unapproved applications.
- Alteration of user administrative permissions.
Regulatory Compliance: It's beyond the skill of most IT security and audit personnel to locate information within a database, and separation of duties requires information collection and policy analysis be implemented independent of DBAs and IT administrators. DAM platforms provide this separation of duties, and most vendors pre-package thousands of compliance polices to aid in the deployment. Typical polices include:
- Enforcement of separation of duties on database administrators for SOX compliance by monitoring all their activity and generating SOX-specific reports for audits.
- Verification the audit trail is being produced
- Access to sensitive data audit reports
- Change order verification
- Access control and authorization reports
Operations: Changes to database applications are complex, often comprised of hundreds of individual steps, with the ultimate result not evident from any single action. Transactional analysis of common database administrative tasks are recorded, analyzed, and depending upon the result, results can feed workflow or trouble-ticket systems. The types of operations may include:
- Installation of patches
- Appropriate use of service accounts
- Alteration of database function, access or accessibility
- Backup and recovery detection
- Change order verification
- Business process failures
HOW DOES DATABASE ACTIVITY MONITORING WORK?
We have covered the business value, now let's talk about the tools and platforms to help understand the core functions, and how to differentiate one vendor's offering from another. Were not going into a lot of detail here, but just enough to guide you in a selection process. Note that to perform their function, DAM products follow a consistent process: collection of the events from the database, analysis of the activity in relation to established policies, and alerting when a policy violation is detected. As each phase represents a core piece of the product, let's look at each of these in greater detail:
Monitoring systems are deployed as software, an appliance, or in some cases as a virtual appliance running on top of a virtualization platform. These platforms are remote and not stored on the same platform as the database. All will offer a Web interface for remote administration, policy management and report development. All of the products will have some form of internal database to store collected data, policies and reports. You will likely select a deployment option that matches your environment today, but keep in mind that each option has different performance, flexibility and cost associated.
Where the platforms begin to diverge is on one of the most important features: data collection. The collection of SQL statements, in addition to monitoring of programs and batch jobs stored within the database, are the types of activity collected. There are three methods used to collect activity: network monitoring, local agent and remote credentialed access. Network monitoring is still used as a lightweight, non-invasive method to collect activity, but fails to collect administrative commands and is blind to the use of encrypted sessions. Remote credentialed access is a common collection technique when using native auditing and tracing functions, but is limited to native database capture methods. Agent based data collectors are increasingly common as they allow for local, credentialed access to leverage native data capture, but can also employ other collection techniques such as protocol monitoring, memory scanning and event tracing.
It should be noted that you will likely use not only multiple deployment models, depending upon the business need that prompts the use of activity monitoring, but multiple data collection techniques depending upon the specific security or regulatory requirement. Having a platform that offers, at a minimum, remote credentialed and database agent options is important. Further, you should look for multiple collection options for each platform, such as a method that captures the database audit trail and one that collects all console and administrative activity so you have the ability to enforce a breadth of compliance and security policies.
The second major platform differentiator is analysis; this is what separates DAM from auditing. Monitoring systems are designed to find violations instantly, utilize more advanced inspection techniques, alert, and even block activity.
All monitoring and auditing platforms provide basic reporting based upon the analysis of data and SQL query attributes. What we mean by attributes are things such as which user, what time of day, what query operation, what application issued the query, the columns affected and other associated variables. Unlike auditing, when a statement matches the attributes specified, an alert is generated and, with some platforms, the action is blocked. Common policy examples are more than three failed logins, a single user issuing queries from different locations at the same time, or when any user selects more than one customer record at a time.
As threats evolve, such as SQL Injection and buffer overflow attacks, new analysis techniques have been implemented. Statement or lexical analysis is one such variant, where you examine the structure of the SQL statement. Examining components of the query for such tricks as "where 1=1" to force statement execution, abnormal activity is detected because the statement just doesn't look right.
Behavioral analysis is another advance in analysis techniques used to detect insider threats and misuse. This variation combines one or both of the above analysis techniques, but augments the comparison with a behavioral profile. The profile is created by establishing a baseline of user behavior that represents normal activity to establish a reference point. Every subsequent query is examined not only for typical attribute violations, but attributes that differ greatly from the established norm.
Other important considerations are performance, policy management, integration and reporting. During your review, make sure you mimic load and scalability test to ensure that the product you purchase will in fact cover your entire organization.
With policy management, as this is where your IT team will spend the majority of time making updates, verify that policies are easy to create, adjust and apply to different databases as needed. You will want to double-check vendor claims that their solution will integrate with any existing workflow, trouble-ticket or data management systems you have in place. And be sure if they do not cover all of the databases you need to protect, that they will commit to doing so within the next six months or you should receive a partial refund. Finally, as reports and alerts will provide key notification to events as they occur, verify that you can adequately build and maintain distribution of information, based upon the criticality of the data, to responsible parties. Take the time to understand these key aspects of the platform, as your satisfaction with any given product will be largely dependant on how easy it is to manage on a day-to-day basis.
COMPLIANCE DRIVES DAM ADOPTION
Solving database security problems was the genesis of the DAM market, but compliance is what drives adoption of the technology today. While there is overlap with other security and management platforms, database activity monitoring offers features and functions found nowhere else. Access control systems, SIEM, and WAF technologies can offer some of these features, but not all at once--and not from a single product. Database activity monitoring is a much more recent addition to our database security toolbox, and utilizes different approaches to analysis and data capture, and when coupled with near real time results, are much more appropriate for the securing data and keeping track of application activity.
This was first published in April 2010