|
TO SYSLOG OR NOT TO SYSLOG
Standby in the logging world is syslog, which provides a framework for collecting and storing log data but has well-known performance issues and can drop some data during periods of high network use. Some vendors also support a more recent version called syslog-ng (for next generation) that includes delivery using TCP instead of UDP.
"Syslog-ng tries to solve that problem with guaranteed delivery, but that can slow down the collection process," says ArcSight's Njemanze. The trade-off is having a high-performance collector that misses log events but keeps up with real-time traffic analysis for threat mitigation, versus having something more complete but lags behind in real-time collection.
"When you are capturing all this log data you shouldn't be forced to filter or normalize any of it, because that slows things down," says Stevens.
As a result, LogLogic offers two different log management product lines. One stores its logs in a SQL database, while the other uses raw files. "It is important to do both," says Anton Chuvakin, director of product management of LogLogic. "Some users of log data want the flexibility to do visualization and compliance reports, while others want to be able to do full text searches."
"In practice, most of our customers tend to go with traditional syslog because they want to see current messages, even if this means that they lose a few in the collection process. Whichever method you employ, make sure that the system you use to capture logs has the capacity to keep up with the message traffic," says Njemanze.
"Syslog is pretty bad and has all sorts of issues, but it's also really common, and there are millions of devices that write to its format," says Chuvakin. "Sometimes convenience can override security concerns."
Without a doubt, log management is a tough task to tackle, but the security and compliance benefits it can provide have become essential. And while the market of available tools that can help ease the process is rather convoluted, it may become clearer as vendors hone their products. Both log managers and SIMs will continue to converge as vendors add features to complement and extend their product lines. For the next few years, however, it is likely that IT and security managers will need both kinds of products to satisfy multiple needs.
|
