Manage Learn to apply best practices and optimize your operations.

How to keep personally identifiable information out of access logs

Are there products available that can hide the internal IP addresses recorded in log files? Maybe not, but in this expert Q&A, Michael Cobb reveals which tools can prevent the transfer of personally identifiable information to third parties.

Which tools can hide the identity of Web server access logs so that they can be viewed and analyzed securely by a third party? I don't want to expose internal IP address information outside of the company.
Companies such as Metronome Labs offer tools that can keep log files from including personally identifiable information. The technology prevents the transfer of PII to a third party unless the owner provides explicit permission. However, I do not know of any specific tools that hide internal IP addresses recorded in log files. Before I discuss a possible action, I wonder whether you have considered the underlying issues that have resulted in your need for such a tool.

Firstly, if you do not trust the third party who is analyzing your logs, or do not feel that the company's service level agreement (SLA) provides you with enough assurances, then you need to find another organization to deliver the log-analysis service. Secondly, if you feel that it is imperative to hide internal IP address information, then you should look at undertaking log analysis on your own.

If this is not an option, you could simply use a text editor to do a "search and replace" of key IP addresses. Then, for each found address, you could substitute the IP string with a false one. There are some issues that you should be aware of, though, before you alter your log files.

When doing any log file analysis, you must never work with the original files. In the event of a security incident, log files will be an essential aid in forensic analysis. Therefore, you need to make copies before performing any post-processing or analysis. When used as court evidence, files must be presented in their original form. By making sure that your original logs are never altered, you can be sure that they are still authentic.

If you are running several Web servers, it would be my preference to send the log files to a central syslog server rather than have them written to the local file system. Many attackers now try to hide their tracks by altering or deleting the server's log files. Storing the files on a secure log server therefore makes it a lot harder for malicious hackers to hide their activities.

If you use a central server, it is important that you keep your system clocks synchronized using the Network Time Protocol (NTP). Otherwise, log entries will inevitably appear to be recorded out of order, causing difficulties for many analytical software programs. If you move your logs offline -- to a tape, for example -- you will need to record how the files were moved and where they were moved to. In a criminal investigation where the contents of a backup may need to be investigated, tracking custody of evidence is especially important.

One way to be absolutely sure that a log file has not been modified is to sign and encrypt it using a public-key encryption program such as PGP.

More information:

Next Steps

Why PII security needs to be addressed now

This was last published in September 2007

Dig Deeper on Web authentication and access control