This tip is part of SearchSecurity.com's Integration of Networking and Security School lesson, Getting the most out of your SIM deployment. Visit the school and lesson pages for additional learning resources.
Security information management systems, or SIMs for short, are effective platforms for the collection, analysis and storage of events from a broad range of systems and devices within the IT infrastructure. Many enterprises are overwhelmed by the myriad of choices and types of data a SIM can provide. For example, any given application might produce an access log, an event log, a transaction log and an audit log, each used for a slightly different purpose and containing slightly different information. Taking some time to understand the difference and selecting just the information you need helps produce better reports and reduce resource overhead. In this tip, we will explore efficient ways to get the most relevant data from enterprise security information management systems.
Understand the regulations
With storage costs incredibly low, many organizations manage compliance by taking the path of least resistance. Rather than evaluate the regulatory requirements for data retention -- like what data needs to be kept and for how long -- many IT professionals will simply turn logging and auditing functions on and collect every available event. In turn, the record retrieval, analysis and reporting process becomes overwhelming, taking days instead of minutes.
Many regulations, however, don't actually require long-term storage. As an example, many have treated the amended rules for the Federal Rules of Civil Procedure (FRCP) as a requirement to store audit trails for an extended period. But a review of the Electronic Discovery Rule Amendments actually reveals that an organization has the ability to define "electronic data which is reasonably accessible." It is perfectly acceptable for an organization to dictate what is appropriate, since few people know a business as well as those helping to run it. As long as the policy documentation specifies what is considered appropriate, you are in compliance. For example, for a university on the semester system, a 120-day email retention policy may be perfectly acceptable.
Collect the right data
I often see companies collecting system logs in their entirety or turning on audit features for their application platforms and incurring a great deal of overhead when all they wanted was a report detailing failed logons. What could have been done with simple network monitoring with no performance cost to the application instead generated gigabytes of log files and reduced overall application throughout. Even so, as some of the applications didn't view failed logins as an application-level event, most access control-related failures were not recorded by design, and service-level interruptions were missing altogether.
As many data collection methods have overlapping information, and each method has an associated cost in terms of performance and quantity of data -- not to mention quality of data -- spend some time ensuring that the data being collected is appropriate for the type of problem being addressed. Sometimes a simple adjustment in this area leads to much more accurate reports.
Assess your efficiency
As SIM is not a new technology platform, there are many companies that have had the technology for several years but are less than happy with the performance and value of the platform. A lot has changed in the last couple of years, however, with new methods of data collection, new ways of analyzing data, and new ways of solving IT problems. Enterprises that implemented a SIM product a few years ago should re-examine their assumptions and deployment choices now that the platforms have evolved and become more efficient. Your SIM may offer a new method of data collection, like network packet collection, which can provide a more efficient method of collecting activity. Or you may have implemented virtualization, which can affect the data that is collected. Most company networks undergo constant change, and vendors are improving their products, so periodic review is recommended.
Know the value of normalized data
When collecting data from hundreds or thousands of disparate devices and systems, normalization helps to provide a unified view of the events. Normalization for SIM means automatically pulling common data items from each event (like who, what, when and where) and storing this subset into a common format. In essence, SIM normalization is making dissimilar data all look the same. This process makes cross-system analytics feasible. And since all events share a common format, reporting and analysis is far easier as well.
But in many cases, the data that is kept in this normalized form is insufficient to really understand if there is a problem and what steps are necessary to remediate it. If a SIM platform identifies a failed or illegal transaction -- for example, if someone uses an unapproved application to make an ad-hoc adjustment to the general ledger -- a normalized event record will not include enough information. Identifiers like transaction ID, customer name, dollar amount, or any of the contents of the transaction needed to identify the transaction is missing. In order to fix review and fix the questionable entry, it will most likely be necessary to manually sift through hundreds or even thousands of legitimate changes.
Normalized records are a great way to reduce data volume and provide aggregation and correlation report, but to provide value from a security or audit standpoint, normalized event detection is not enough. Make sure to store enough of the original record, or better yet, have "drill down" capabilities, to cross reference the normalized record with the original record.
SIM platforms are both powerful and flexible. They provide a fast and efficient way to gather data, and a lot of options on how to process and analyze this data. What you collect and how you process it affects your ability to meet business drivers and compliance requirements, so take the time up front to understand your options and how best to achieve your goals; it will save you time and money in the long run.
About the author:
Adrian Lane is a senior security strategist with Securosis LLC, an independent security consulting practice. He has 22 years of industry experience, specializing in database architecture and data security. Prior to joining Securosis, Lane was the CTO at the database security firm IPLocks, and he has also served as the vice president of engineering at Touchpoint, three years as the CIO of the brokerage CPMi, and two years as the CTO of the security and digital rights management firm Transactor/Brodia.
