PiChris - Fotolia

Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

The hunt for data analytics: Is your SIEM on the endangered list?

Analytics-driven security disrupts -- and sometimes supplants -- SIEM tools.

This article can also be found in the Premium Editorial Download: Information Security magazine: Are SIEM products delivering on advanced analytics?:

When defense contractor SAIC split in 2013, the services side of the business got the name but not the security operations center. The new SAIC, focused on providing services to government customers, had to rebuild its security operations from scratch.

Yet rather than create a team focused on the care and feeding of a security information and event management (SIEM) system, Jonathan Jowers, SAIC’s chief information security officer, built on the experience of its operations group and adopted Splunk’s machine data analytics platform to detect network anomalies and hunt down threats. Because SAIC was no longer an $11 billion giant -- the new spinoff was less than half that size -- it saved money and increased agility with an analytics-based system, according to Jowers.

For the past 11 months, the analytics system has worked well, he reports: "SIEMs are really good at giving you a view -- and often an advanced view -- into your environment, but having the analytics lets us go beyond that," he says. "Almost weekly, I will get information from our analysts about some investigation that they have done that goes beyond SIEM."

Adding more data analysis

Big data may have faded as the buzzword in the information security industry, but security companies are increasingly incorporating the analysis of data from a variety of security, network and business sources to build more accurate and sensitive systems. While few companies have adopted the strategy of SAIC -- leap-frogging over a SIEM deployment and moving directly to a full-blown machine data analytics-based platform -- it’s hard to find an enterprise installation that doesn’t build some form of data analytics into its SIEM infrastructure. Many security companies are also focusing on adding data-analysis capabilities to their SIEM products.

In the late 1990s, SIEM systems were born of the frustrations of managing the logs of an increasing number of network devices. Today, modern users of the systems are frustrated with the large number of false positives that the systems create. Analytics promises to help SIEMs recognize security threats better and reduce the workload on the analysts who are tasked with monitoring the systems.

"SIEM requires a change," says Robert Ma, Splunk’s senior director of security markets. "It solved the problems of the 2000s, where the security technologies were performing centralized reporting and monitoring. What we need [now] is more analytics-driven security."

In addition, SIEM systems need to change to keep up with attackers, experts say. Finding the signs of an ongoing attack is often easy for companies that have good visibility into their networks and user activities. But the more sophisticated actors are adopting more subtle techniques as well.

"The market has been changing at breakneck speed," says Charles Sterner, vice president of product innovation for HP ArcSight, whose enterprise security management system combines event correlation and security analytics. "Not because of the capabilities that the vendors have been delivering, but because of the cat-and-mouse game we have been engaged in with the attacker."

SIEMs need to move beyond just aggregating events from perimeter systems, what Sterner calls SIEM 1.0, and monitoring the security of applications, or SIEM 2.0. The next generation of SIEM platforms should be able to detect and predict threats based on the behavior across systems, he says.

A funnel or a fishing pole?

These types of SIEM platforms gather a variety of information from network and host devices, connecting the dots between, for example, an alert on a company’s Internet gateway and anomalous traffic from a user’s system. The system essentially reduces a deluge of data to a more manageable stream, says Nir Polak, co-founder and CEO of analytics tools provider Exabeam Inc.: "It's a funnel that uses correlation in the SIEM to produce alerts and reduce false positives."

SIEMs use data gathering, normalization and analysis techniques, such as correlation, to reduce hundreds of thousands or millions of events into a much smaller number of alerts on which analysts need to focus. For static environments, including healthcare and financial networks, the visibility that such rule-based correlation produces may be suitable.

Sometimes, this level of visibility is enough. Since Conficker began spreading in October 2008, almost every company has had to deal with the pernicious worm. Yale New Haven Health System was no different. The healthcare firm struggled to eradicate the malware, and the company’s service group declared victory in 2009, based on reports from the business’ endpoint antivirus systems. But the SIEM system the company used continued to detect signs of infection.

three greatest impediments to investigating attacks

"The team that was assembled to deal with the outbreak stood on the ‘deck of the aircraft carrier’ and said that we won," says Steve Bartolotta, a former information-security director with the company. "But we had not won, and our SIEM could see that."

Yet, exploratory analytics allows the analyst to dive back into the data, analyze the raw event information with a variety of techniques and find outliers and other anomalous activity.

The goal is to use as much data as possible, but to offer simpler views into information, so critical incidents are not missed, says Splunk’s Ma. In essence, the business wants a fishing pole, not a funnel.

"All security data is relevant so you need to look at as many systems as possible," says Ma. "Because you never know what system attackers will touch and what will lead to detection."

Reducing false positives

In November 2013, attackers began tapping into the transaction data processed by retail giant Target. The activity reportedly triggered an alert by the company’s intrusion detection system, but the event was lost in the noise of all the false positives generated by different security systems.

Such alerts are often overwhelmed by the level of noise produced by security products. The average large company must triage some 17,000 malware alerts every week, even though only 19% are considered reliable, according to a January 2015 survey, The Cost of Malware Containment, conducted by the Ponemon Institute. Only 4% of the events were eventually investigated by a human analyst, according to the report.

A former hunt-team leader at incident response firm Mandiant, now part of FireEye Inc., David Bianco has hard-won experience exploring clients’ event data to find anomalous events. Bianco, who is now security architect for big data security-intelligence firm Sqrrl Data Inc., says: "At the scale we are talking about, humans can’t look at everything. You are forced to have some sort of automated system, and the more correlation and analysis you can do, the better."

The data problem will only get worse. In 2000, when SIEM vendors ArcSight and Q1 Labs were both founded, SIEM systems regularly saw as many as 15,000 events per second. Now, 80,000 events per second is not uncommon, according to HP ArcSight’s Sterner. (HP acquired ArcSight in 2010, and IBM bought Q1 Labs in 2011.)

Reducing a key measure, the number of analyst-events per hour, can help. A security analyst trying to investigate 25 events per hour will start clearing potentially malicious events, a decision that will create a feedback loop that makes it more likely they will miss serious events in the future, says Sterner. "As an analyst starts to get swamped, that precognitive bias kicks in, and they say, ‘I’ve seen this alert before,’ and they will ignore it."

A shortage of data scientists and security analysts exacerbates the problem and can lead to a company’s visibility hitting a wall, he says. The best solution is to create better analysis techniques and incorporate them into the product so as to make the analyst smarter.

"We can very quickly say that this event is truly abnormal and reduce the number of events per analyst [per] hour," Sterner says. "Exploratory and detection analytics working together allow us to break through that wall."

Tracking users, not devices

With employees accessing work data and systems from a variety of devices, analytics are critical for tracking workers' activity as they move from a company system to their smartphone or a home device. Adding such functionality is a key component of the user behavior analytics provided to a SIEM by Exabeam’s physical or virtual appliance, according to Polak.

"The attacks today span across devices and across IPs," he says. "If a user just logged in and now switched to a different account, the system has to keep state and know that it is still the same user."

Without such analytics, SIEMs may generate false positive alerts or miss an attacker’s use of a credential on a system from which the victim never works.

Such analytical systems are not boolean, designating an activity bad or good. Systems that look for five failed logins, for example, will not detect an attacker with a stolen credential. Instead, a system focused on the user will see strange behavior -- the worker logging into systems that they normally do not use, Polak says.

"The more anomalies that occur for a particular user, the higher that activities risk score will be," he says. "It is not a yay or nay. Each one of these is a bump in the score. It is looking at multiple dimensions and connecting the dots."

Detecting an evolving attacker

While SIEM systems can help security teams harden their corporate networks, analytics can aid in tracking down sophisticated attackers who change how they operate in reaction to defenses.

Defenders must also evolve and need the flexibility of analytics to chase down malicious actors as they improve their tactics, says Sqrrl’s Bianco. "It is really about restricting the movement of the attacker. Our general philosophy is that companies need to drive up the cost to our attackers."

While SIEM allows companies to detect known attack techniques or anomalous activity, security analytics enables firms to explore outliers and determine their cause, allowing new attacker activity to be caught and classified. A Tier-3 analyst can then turn the activity into a correlation rule to feed back into the SIEM.

"I look at the exploratory analytics and regular analytics as yin and yang," says HP ArcSight’s Sterner. "If I can catch things in real time, I can reduce the cost of the attack drastically and, in the best case, prevent any true information loss."

By constantly having a hunt team run analytics, they can catch new activity. Using machine-learning techniques, such as affinity grouping and cohort analysis, mathematical anomalies can also be teased out.

"These math-based detection approaches are really useful, because in a SIEM, we see machines are repetitive -- they do repetitive event logging very well," he says. "What they don't do well is random."

While some companies such as SAIC are pursuing data analytics as a complete security strategy, a hybrid approach will continue to be common. The downside of SIEM systems and data analytics are that both systems require skilled analysts to operate effectively. Yet, poking around in the data and exploring the connections between systems, users and security can help hone analysts’ skills, says Splunk’s Ma.

"Companies need analytics-driven security," he says. "Everything needs to evolve to have an analytics component. Everyone has been talking context forever, and analytics delivers that -- it’s the missing layer."

About the author:
Robert Lemos is an award-winning technology journalist, who has reported on computer security and cybercrime for 17 years. He currently writes for several publications focused on information security issues.

This was last published in March 2015

Dig Deeper on SIEM, log management and big data security analytics

PRO+

Content

Find more PRO+ content and other member only offers, here.

Join the conversation

2 comments

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.

Well-researched Robert! SIEM is dead... long live SIEM. What we at AccelOps see is a growing need for cross-correlation of security and performance data from network devices plus the app stack. Modern security threats increasingly masquerade as availability issues and can only be detected by monitoring logs plus machine metrics and config changes. Oh, and 80K EPS as the new normal is quickly becoming obsolete. We see typical enterprise and MSP EPS volume growing 15% per year... with no end in sight. Well done!
Cancel
How organizations acquires and use data is transforming how we work and live. Many indicators shows analytics revolution that will change how businesses are managed.
Cancel

-ADS BY GOOGLE

SearchCloudSecurity

SearchNetworking

SearchCIO

SearchEnterpriseDesktop

SearchCloudComputing

ComputerWeekly

Close