This content is part of the Essential Guide: How to assess security information and event management systems
Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Final considerations before purchasing SIEM technology

Get help evaluating security information and event management systems to ensure that you find the SIEM technology that best fits your organization's security needs.

Every company that wants to implement a security information and event management platform -- or even those that...

are considering switching vendors -- needs to properly evaluate two things: Why it needs a SIEM system and what vendor will best fulfill these needs.

Your evaluation criterion comes directly from your requirements and the features that deliver on those requirements. It is for this reason that using off-the-shelf "requests for proposal" or "requests for information" templates are a waste of time. The questions you ask a vendor must come directly from your requirements, otherwise it's just a waste of your time, and the vendor's. Sometimes customers even ask for features they don't care about just to gauge a vendor's depth of functionality, or ask if their appliance is made from "un-obtanium" just to see if the vendor will lie in its responses, but it's a waste of time to play these games.

At this point, you should have a handle on your requirements, you should have agreement from the different stakeholders on the priorities and ysou should understand what features are needed to support your needs.

Now it's time to construct the questions that will help you pinpoint which of the 20 plus vendors who provide SIEM features and platforms are the ones you want to evaluate. You need to get beyond the "PowerPoint stage" of the evaluation, and get the products into a real-world environment to see how they perform. We do not recommend going into a full proof-of-concept evaluation with more than two or three vendors; the time required to set up and evaluate all the products becomes too great. Look to narrow your search down to two vendors early in the process.

Set up your evaluation question so that you understand how critical features are delivered.

It's when you dig into the vendors' product that things get interesting. Once you understand what you need and how your environment is set up, you're going to have a good idea if any particular product is right for you. For example, if your goal is malware detection and your requirements list stipulates both application whitelisting and behavioral attack detection, then the SIEM platform must provide real-time analysis against pre-existing behavioral norms and approved application lists. While most vendors claim they have this, their products may require that you build and maintain the profiles yourself. Vendors can conceptually offer lots of features, but if they require full-time personnel to manage, then they're not useful.

When testing products, apply a set of real use cases. For security testing, seed log files with malicious events or captured attack data to see what the SIEM platform detects. Further, conduct a detailed analysis process to comb through the events, paying close attention to UI ease of use and how easy or difficult it is to navigate the data. For compliance, select one or two reports from each regulatory obligation, and see if the vendor can produce summary and complete findings to your satisfaction. Select one key application for integration, and ensure the two platforms communicate the necessary data and can work within your process model. Finally, remember to test scalability and performance, though this can be a bit tricky as it's hard to test scalability without a full rollout. The use of virtual platforms or third-party cloud services is recommended to perform a proof of concept to verify vendor claims of scalability. This level of testing is critical with any product to ensure it delivers.

Making the final deal

Take the time to see how your requirements will map to the specific feature sets as advertised by the SIEM vendors and, more importantly, how their implementation maps to your environment.

If your requirements dictate support for high-volume data loggers or extensive data formats to support new devices, then odds are a security information and event management platform that relies upon relational databases for event storage won't work for you. Set up your evaluation question so that you understand how critical features are delivered. Take the time to see how your requirements will map to the specific feature sets as advertised by the SIEM vendors and, more importantly, how their implementation maps to your environment. These areas will be where you focus a subsequent proof of concept and form the basis for your final judgment on how well the potential suitors meet your needs.

It's worth mentioning that lots of SIEM customers establish new use cases over time but don't have the internal know-how to manage SIEM under the new operating requirements. For example, not every company has the expertise in-house to know how an attacker may penetrate its defenses or even what sort of activity to look for. It's increasingly common for companies to engage third-party professional services to manage a security operations center. If you feel you lack the in-house expertise, consider third-party management as part of your requirements, or even ask service providers what tools they use and why.

Whether it's evaluating or managing a SIEM product, success is up to you. Holding your organization up to a mirror is the first and most important step. Once you've selected a platform, remind your stakeholders that platform implementation takes time; not every feature they want will be immediately available, so publishing a realistic deployment road map helps set expectations.

About the author:
Adrian Lane is CTO of Phoenix-based analyst firm Securosis. Adrian specializes in database security, data security and software development. He is a former executive at security and software companies such as Ingres, Oracle, Unisys and IPLocks, and is a frequent presenter at industry events. Adrian is a graduate of the University of California at Berkeley with post-graduate work in operating systems at Stanford University. Reach Adrian via email at

Next Steps

Explore the benefits of enhanced SIEM

Hear the top five lies you may have heard about SIEM systems

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.