Q

How to estimate log generation rates

In this expert response, Mike Chapple explains why estimating log generation rates is so difficult.

Are there any calculators to help estimate log generation based on number of devices and best practices?
Estimating log generation rates is tricky, and it's difficult to create a credible, generic estimation tool. Many security information and event management (SIM/SIEM) vendors have proprietary Excel-based calculators that they offer to current and prospective clients, but the objectivity of these tools is questionable given their source. I was unable to locate an independent calculator that offers this functionality.

Why is this so difficult? Log generation rates vary significantly based upon the configuration of devices. For

example, you and I may both run Microsoft SQL Server databases, but I may have the logging and auditing settings configured to track almost every activity the database performs, while you may have minimal (or no) logging configured. Additionally, I may be in a high-load 24x7 data processing environment, while you may be running a database with low transaction volume. Therefore, it's impossible to provide a meaningful estimate of the log volume generated by a "typical" SQL Server database. Add in hundreds or thousands of other diverse devices, and the problem magnifies in scope quickly.

So how is it possible to develop a meaningful estimate for your environment? There's only one solution: measure your current activity by, for example, setting up a simple syslog server and measuring the volume of traffic it receives. If the systems are similarly configured, you can save time by measuring the logs generated by a representative sample of your organization's devices and extrapolate from there.

More information:

  • Learn how log management reins in security and network device data.
  • See why the PCI Data Security Standard has forced companies to seek log management help.
  • This was first published in February 2009

    Dig deeper on Security Event Management

    Pro+

    Features

    Enjoy the benefits of Pro+ membership, learn more and join.

    Have a question for an expert?

    Please add a title for your question

    Get answers from a TechTarget expert on whatever's puzzling you.

    You will be able to add details on the next page.

    0 comments

    Oldest 

    Forgot Password?

    No problem! Submit your e-mail address below. We'll send you an email containing your password.

    Your password has been sent to:

    -ADS BY GOOGLE

    SearchCloudSecurity

    SearchNetworking

    SearchCIO

    SearchConsumerization

    SearchEnterpriseDesktop

    SearchCloudComputing

    ComputerWeekly

    Close