Following Oracle Corp.'s release of several dozen software patchesSure, the latest patches cover "a lengthy laundry list" of security holes, said Patil, database administrator for Boston-based wireless communications provider American Tower Corp. But while users almost always complain of too few details and malfunctioning patches after an Oracle security update, Patil said his job is much easier since the Redwood Shores, Calif.-based database giant instituted a quarterly patching process last November.
"There's a lot of effort to pick through the advisory and see exactly what I need. You have to review the document very carefully," Patil said. "But at least with a quarterly process, you know when the next release is coming and you can schedule the deployment work well ahead of time. You can work out the manpower issues and all that. And when the patches come out, there's time to test things more carefully."
When Oracle Chief Security Officer Mary Ann Davidson announced the quarterly system last year, she said database administrators overwhelmingly asked for a schedule they could plan around, but not something as frequent as the monthly process the company originally conceived last year. To Patil, it was a relief.
"You're dealing with systems that are so interconnected," he said, "you don't want to go in there too often. It's much better to have a spread out process you can plan for."
But security experts say administrators shouldn't wait long to deploy these latest patches because they cover extensive security holes that could easily be attacked. They also warn that some known flaws went unfixed this time around and that at least one piece of exploit code is circulating.
Gaping glitches, words of advice
Caleb Sima, CTO of Atlanta-based Web application security firm SPI Dynamics Inc., said his company discovered one of the flaws, in Oracle Application Server 10g (10.1.2). "What we found was a buffer overflow in the server's management console," he said. "You could connect to the console and send a buffer overflow to an application, which could then allow you to take over the system."
His advice to administrators: don't make management consoles open and available on the Internet. "Make it internally available only," Sima suggested. "Those who have their management consoles available online are really increasing their risk of attack."
Another application security firm, Chicago-based Integrigy Corp., issued a similar warning regarding the Oracle E-Business Suite.
"Customers with Internet-facing implementations of the Oracle E-Business Suite are at most risk and should consider applying these patches quickly," it said in an advisory. "The Oracle E-Business Suite patches involved with this Critical Patch Update (CPU) are much more complex as compared to the previous CPUs, and will require additional functional testing in our opinion. In addition, the [patches] are not cumulative. Therefore, all the patches specified in this CPU and previous CPUs must be applied."
A new exploit and unfixed bugs
Amid the plethora of patches, experts warn that exploit code is in the wild. Pete Finnigan, an Oracle expert and author of Oracle Security Step By Step, warned in his blog Thursday:
"Someone has published an exploit for one of the bugs fixed in the… Oct 18 2005 security patch… It is not good that exploits are now in the public domain as anyone who has not patched is now vulnerable. All customers of Oracle should patch promptly."
Meanwhile, David Litchfield, managing director at U.K.-based Next Generation Security Software Ltd., said in a message posted to the BugTraq forum that he gave the latest CPU a "cursory examination" and found that it doesn't fix some of the vulnerabilities Oracle had told him would be fixed.
"Once again the patch is not sufficient," said Litchfield, who has offered similar criticisms of past Oracle patches. He said he would "conduct a full investigation of the patch over the coming few days and post some recommendations once complete."
Can speed kill?
While Oracle and others have recommended that users apply patches quickly, for a customer company with a large and complex network, is it reasonable to expect database managers to move quickly, or to even deploy the entire patch load?
"There are two philosophies about applying patches," Don Burleson, Oracle expert and CEO of Kittrell, N.C.-based BC Remote DBA, said in an e-mail exchange. "One school says that all patches should be applied because it may prevent a future problem. The other school is more conservative and adopts the philosophy of 'If it's not broken, don't fix it.'"
Your patching time may vary
At American Tower, which runs Oracle's E-Business Suite, Patil said the Oracle patch process can take a couple days or a couple weeks to complete, depending on how big the patch load is. As far as he's concerned, careful testing is paramount, even if it slows down the deployments.
"We go through the [patch detail] document very, very carefully and figure out which pieces we use," he said. "We check those pieces, get the patch numbers and download those that apply to our platform, which is HP Unix."
The next, and perhaps most important step is careful testing, Patil said. "In a production environment, even if the patch is only breaking something small, you have to be careful and make adjustments," he said.
A patch is tested at the technical level, then the business level, then at the management level, he said, adding, "Then I put my stamp on it." How has this quarter's cycle gone so far? When asked Thursday, Patil said, "We should at least have everything set at the tech level by tomorrow [last Friday]."