bluebay2014 - Fotolia

Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

What security log management best practices should my team follow?

Security log management includes deciding what log data to retain and the length of time it should be stored. Expert Michael Cobb explains some challenges and best practices.

My newly formed security team is trying to create a policy for retaining security logs, and we want to have a timeline...

for how long these logs should be kept. As an auditor, I have seen requests for anywhere from six months to a year. The complaints are that the files are huge and saving them for a year would take up too much space. How long should security logs be kept? Do you know of any best practices for security log management?

Computer security log management requires the effective balancing of limited log management resources with a continuous supply of log data. It's getting harder to get this balance right due to the huge increase in the number, volume and variety of computer security logs enterprise IT networks now create. Despite the challenges, successful log management is essential to ensure that records are stored in sufficient detail for an appropriate period of time.

Certain logged data has to be stored to comply with federal legislation and regulations, such as the Federal Information Security Management Act, HIPAA, the Sarbanes-Oxley Act, the Gramm-Leach-Bliley Act and PCI DSS. Your legal team should advise security teams which laws and standards apply to the business and what data needs to be logged and kept for how long.

The type of log data that is generated should be reviewed; inconsistent log content and formats can increase the amount of storage required and the processing time during analysis. Networks can be compromised for some time before a breach is discovered, so keeping security logs for at least a year means forensics should be able to reconstruct a breach timeline or pinpoint responsibility. Unfortunately, there is no standard for the retention of audit log information and each law and standard sets its own requirements, some of which are unhelpfully vague. According to section 10.7 of PCI DSS, an audit trail has to be kept for at least one year, with a minimum of three months immediately available for analysis. HIPAA's requirements are a little less clear as the final Security Rule includes a documentation retention requirement of six years, but this only includes a record of the activity or assessment, not all the supporting logs.

Security log management also involves protecting the confidentiality, integrity and availability of logs. Logs must be kept in a secure area with adequate physical protection and monitored access. If log data needs to be retained for longer than five years, ensure the storage media has a suitable lifespan. Log data must not be destroyed before the duration of the required data retention period, but neither must PII data be kept beyond this time, and data destruction should follow the NIST Guidelines for Media Sanitization.

Logs play an increasingly important role in IT security and require a viable strategy to manage them. The NIST Special Publication 800-92 Guide to Computer Security Log Management provides further guidance for meeting the various challenges of security log management.

Ask the Expert:
Want to ask Michael Cobb a question about application security? Submit your questions now via email. (All questions are anonymous.)

Next Steps

Find out how log management begets better user acceptance testing

Learn how to ensure audit log security

Explore how a good system logging strategy can quickly identify issues

This was last published in June 2016

Dig Deeper on Information security policies, procedures and guidelines