rvlsoft - Fotolia

Manage Learn to apply best practices and optimize your operations.

Algorithm substitution attacks: Ensuring encryption algorithm security

Algorithm substitution attacks can decrypt secure communications and potentially expose enterprise data in plaintext. Learn how to mitigate the threat.

What is an algorithm substitution attack? How can my enterprise best defend against the threat?

Algorithm substitution attacks occur when an attacker surreptitiously replaces an encryption algorithm with a compromised algorithm that can monitor communication.

Attackers can swap the algorithm used, modify the existing algorithm or change the initial vector used to set up the encryption to use an insecure random number. These changes could be made by altering the cryptography libraries used by legitimate software or changing the software directly.

In a research paper by Mihir Bellare, Kenneth Paterson and Phillip Rogaway, there is an example of an attacker changing the algorithm used in a closed-source product to a less secure algorithm. Since the user cannot inspect the source code to identify the change, this type of attack is highly difficult to identify, and may even be classified as a supply chain security issue depending on when the application was tampered with.

This type of attack could also happen in open source software because few people have the cryptography skills needed to examine code and determine if algorithm security has been compromised. There could also be a weakness in the random-number generator used for the initial vector that could use the same random number to set up the encryption, making it easier to break the algorithm used.

Though few organizations have staff with sufficient expertise in cryptography to examine code and determine if an algorithm substitution attack has happened, enterprises can defend against algorithm substitution attacks by checking hashes on software used to validate authenticity of the software, using signed software and encryption software that has "survived" cryptanalysis.

Specifically, enterprises should check the hash on downloaded software to make sure it is the version from the vendor. They can also check to ensure the signatures on the files are the legitimate versions from the developers. This should be performed on a regular basis; files can be checked by file integrity monitoring software to monitor for changes. Any discrepancy in the hashes or signatures should be carefully investigated prior to using the software.

It is important to note that not all software with cryptography algorithms was written by skilled cryptographers -- and even software written by skilled cryptographers has implementation challenges. General software developers should use libraries provided for cryptography to ensure their software has a chance of being securely implemented.

The cryptographic community uses cryptanalysis to evaluate software for errors in cryptography. While this will not find all problems, it will reduce the risk of implementation issues in cryptography like an algorithm substitution attack.

Ask the Expert:
Want to ask Nick Lewis a question about enterprise threats? Submit your questions now via email.(All questions are anonymous.)

Next Steps

Get SearchSecurity's latest encryption news and advice

This was last published in April 2015

Dig Deeper on Disk and file encryption tools