Failed patch installation and fixes that don't correct vulnerabilities aren't an everyday occurrence, but they do happen. Oracle Corp. hit such a bad patch in April, and security managers should prepare to verify patches manually as Oracle gears up for another quarterly patch release next month.
"OPatch fails as often as it works," David Litchfield, managing director for Next Generation Security Software Ltd., says of Oracle's patching utility. "While you think you're protected against these flaws, you are, in reality, still vulnerable and open to exploitation."
The disconnect between a correctly installed patch and one that doesn't work impacts regulatory compliance, such as the Health Insurance Portability and Accountability Act [HIPAA], Gramm-Leach-Bliley [GLBA], Sarbanes-Oxley [SOX], and even VISA CISP and MasterCard PCI standards. As part of being compliant you need to prove patches are applied promptly.
"The VISA CISP and MasterCard PCI mandate, for example, that patches must be applied within one month of release," Litchfield said. "Any merchant that fails this test could end up losing the ability to process credit card transactions, which would be disastrous. Part of the requirements of HIPAA, SOX and GLBA dictate that a secure network is to be maintained. If you know of a critical security patch and you don't have any plans to install it 'soon' then this could be considered negligent."
Sometimes the CPU fails to install the updated fixed copies of the files, as it did in April. Then new Java classes designed to correct SQL injection errors failed to load on all platforms; in Windows it copied to the wrong directory and was never executed. And even if Ultrasearch was installed, the fixed versions of a number of WKSYS-owned packages weren't loaded into the database. Oracle later replaced the patch.
Patches fail to fix the flaw
On many occasions Oracle has supplied a "fix" for vulnerabilities discovered by NGSSoftware, yet on examining those "fixes" they fail to properly address the root of the problem. As such, the flaws can still be exploited on a "patched" server.
Failure to perform post installation tasks
This is a fairly common mistake. After running OPatch, the message "OPatch succeeded" is generated. While this could be construed as a complete patch installation, the documentation states that the post-installation steps still need to be executed. These final steps update the PL/SQL packages, Java Class files, etc., in the database and if these aren't installed then the server will remain vulnerable even though running "opatch lsinventory" indicates that the server is patched.
OPatch fails to update the inventory
Incorrect permissions, files to be patched are still in use, environment variables are wrong -- whatever the reason might be, OPatch can often fail to update the inventory. If information in the inventory is wrong then so too are any observations made about patch status and levels.
OPatch used to rollback patches
When a patch is rolled back, the changes aren't reflected in the Oracle inventory so even though the server appears to be patched it isn't.
He recommends manually verifying that patches were applied and are effective. Analyze the patch to see what files and objects are being replaced. Examine the same objects in the database. Note the differences then apply the patch. Once applied, query each component to see whether the server has the patched or unpatched version. Then, fix any problems that come to light.
"You're comparing the expected end state of the database server with the actual end state," Litchfield added. "If the two don't match the patch installation was not successful. By performing this patch verification you're able to isolate what didn't install properly and armed with this knowledge you can fix it."