Security information and event management (SIEM) products and services collect, analyze and report on security log data from a large number of enterprise security controls, host operating systems, enterprise applications and other software used by an organization. Some SIEMs also have the ability to attempt to stop attacks in progress that they detect, potentially preventing compromises or limiting the damage that successful compromises could cause.
There are many SIEM systems available today, including "light" SIEM products designed for organizations that cannot afford or do not feel they need a fully featured SIEM. It can be quite a challenge to figure out which products to evaluate, let alone choose the one that's best for a particular organization or organizational unit. Part of the SIEM evaluation process should involve creating a list of criteria to be used to highlight SIEM capabilities that are particularly important to consider.
This article provides several criteria, stated as questions, that organizations may want to include in their SIEM evaluations. Because these "light" products offer few capabilities, they are much easier to evaluate, so they are out of scope for this article. Instead, this feature points out the aspects of the "regular" SIEMs that merit particularly close attention as compared to acquiring any other security technology.
How much native support does the SIEM provide for relevant log sources?
A SIEM is of diminished value if it cannot receive and understand log data from all of the log-generating sources of interest to the organization. Most obvious is the organization's enterprise security controls, such as firewalls, virtual private networks, intrusion prevention systems, email and Web security gateways, and antimalware products. It is reasonable to expect a SIEM to natively understand log files created by any major product or cloud-based service in these categories.
In addition, a SIEM should provide native support for log files from the operating system brands and versions the organization uses. An exception is mobile device operating systems, which often do not provide any security logging capabilities. SIEMs should also natively support the organization's major database platforms, as well as any enterprise applications that enable multiple users to interact with sensitive data. Native SIEM support for other software used by an organization is generally nice to have, but is not mandatory.
If a SIEM does not natively support a log source, then the organization generally can either develop customized code to provide the necessary support, or use the SIEM without the log source's data present.
Can the SIEM supplement existing logging capabilities?
Particular applications and other software in use by the organization may lack robust logging capabilities. Some SIEM products and services can supplement these by performing their own monitoring on behalf of other software. In essence, this extends the SIEM from being strictly a centralized log collection, analysis and reporting solution, to also generating raw log data on behalf of other hosts.
How effectively can the SIEM make use of threat intelligence?
Most SIEMs are capable of ingesting threat intelligence feeds. These feeds, which are often acquired via separate subscriptions to services, contain up-to-date information on threat activity being observed all over the world, including which hosts are being used to stage or launch attacks, and what the characteristics are of these attacks. The greatest value in using these feeds is the SIEM being able to identify attacks more accurately and to make more informed decisions, often automatically, about which attacks need to be stopped and what the best method is to stop them.
Of course, the quality of threat intelligence varies among vendors. Factors to consider when evaluating threat intelligence effectiveness include how often the threat intelligence is updated, and how the threat intelligence vendor indicates its confidence in the malicious nature of each threat.
What forensic capabilities can SIEM products provide?
Traditionally, SIEMs have only collected data provided by other log sources. However, recently some SIEM products have added various forensic capabilities that can collect their own data regarding suspicious activity. A common example is the ability to do full-packet captures for a network connection associated with malicious activity. Assuming that these packets are unencrypted, a SIEM analyst can then review their contents more closely to better understand the nature of the activity they carry. Another aspect of forensics is host activity logging; such logging could be performed at all times, or it could be triggered when suspicious activity is detected that involves a particular host of the organization.
What features do SIEM products provide to assist in performing data analysis?
SIEM products that are used for incident detection and/or handling should provide features that help people to review and analyze the log data for themselves, as well as the SIEM's own alerts and other findings. One reason for this is that even a highly accurate SIEM will occasionally misinterpret events, so people need to have a way to validate the SIEM's results. Another reason for this is that people who are investigating incidents need helpful interfaces to facilitate these investigations. Examples of such interfaces include sophisticated search capabilities and data visualization capabilities.
How timely, secure and effective are the SIEM's automated response capabilities?
Evaluating a SIEM product's automated response capabilities is necessarily an organization-specific endeavor because it is highly dependent on the organization's network architecture, network security controls and other aspects of security. For example, a particular SIEM product may not have the ability to direct an organization's firewall or other network security controls to terminate a connection carrying malicious activity. Besides ensuring the SIEM product can communicate its needs to the organization's other major security controls, it is also important to consider the following characteristics:
- Timeliness: How long does it take the SIEM to detect an attack and direct the appropriate security control to stop the attack?
- Security: How are the communications between the SIEM and the other security controls protected so as to prevent eavesdropping and alteration?
- Effectiveness: How effective is the SIEM product at stopping attacks before damage occurs?
Which security compliance initiatives does the SIEM support with built-in reporting?
Most SIEMs offer highly customizable reporting capabilities. Many of these products also offer built-in support for generating reports that meet the requirements of various security compliance initiatives. Each organization should identify which initiatives are applicable to it and then ensure that the SIEM product supports as many of these initiatives as possible. For any initiatives that the SIEM does not support, confirm that its reporting can readily be customized to meet those initiatives' reporting requirements.
Do your homework and evaluate
SIEMs are complex technologies that require extensive integration with enterprise security controls and numerous hosts throughout an organization. To evaluate which SIEM is best for your organization, it is helpful to define basic evaluation criteria. There is not a single SIEM product that is the best system for all organizations; every environment has its own combination of IT characteristics and security needs that must be taken into account. Even the main purpose for having a SIEM, such as meeting compliance reporting requirements or aiding in incident detection and handling, may vary widely among organizations.
Therefore, each organization should do its own evaluation before acquiring a SIEM product or service. This article presents several criteria that should be considered as part of an evaluation, but this is not meant to imply that other criteria are not necessary. Think of the listed criteria as a starting point for the organization to customize and build upon to develop its own list of SIEM criteria. This will help ensure the organization chooses the best possible SIEM product.