Tommi - Fotolia
When Lenovo recently discovered its System Update program -- used to automatically patch many of its notebook computers -- had critical security flaws, it patched its patching process. But the three flaws Lenovo fixed were significant, and they resurfaced fears about whether automatic system updates offer enough security. This comes as more and more systems, including the forthcoming Windows 10, are switching to hands-free updates.
Josh Corman, founder of I Am The Cavalry, said IT is just going to have to swallow its concerns. The history of broken or malicious updates, including a fake Firefox update in 2012, can't leave administrators gun-shy when it comes to updates.
"If you are going to run software, it is going to fail and you need to have a way to respond to that failure," Corman said. "There was a time where people would not apply patches when they came out because they could break your system. So, you might get hacked, but you might also brick your server in production."
Corman said that type of thinking needs to end because although automatic update services do introduce some remote attack surface, the benefits far exceed the risks.
"Many of those mistakes that people have made in the past we've learned from and grown as an industry to figure out how to do remote updates in a stable way and a secure way," Corman said. "It does not mean everyone will do it that way, but I'd rather encourage better implementation than avoiding any implementation."
Delivering secure updates
Implementation appears to be the key, because Corman and other experts agreed there are risks associated with automatic update systems, including the potential for man-in-the-middle attacks or updates being hijacked by malware, but many of those risks can be traced back to improper implementation of an update process, not to the updates themselves.
"There's no problem with the cryptography, the basic mechanism. I don't think there is anybody who has broken that. The problem is typically in the implementation of these things," said Wolfgang Kandek, CTO of Qualys. "I think cryptographers are talking about quantum computing potentially being able to break the cryptography and take over the process, but as far as we know, nobody has the technology at this point in time."
Corman went beyond the cryptography and broke down the proper architecture for delivering updates by saying it needs to be stable, so the update doesn't cause device failure; secure, meaning not passing updates in the clear or without digital signatures; and hygienic, meaning the software has been checked for authenticity and quality before being signed or delivered.
"If you can provide authentic updates over a secure channel in a stable and repeatable manner, then that's the path forward because you're going to need to update," Corman said, "and the inability to do so is not an option. It may be difficult to do these things, but it is nonetheless necessary."
Kandek agreed that creating a secure process for delivering updates can be difficult and suggested that one way software vendors can make things easier is to rely on centralized update mechanisms as much as possible, like the Windows or Mac app stores, or even the Chrome Updater system, which Google has made open source.
"This is difficult to do well," Kandek said. "If you want to cover all of the bases, it's a pretty complicated problem and it's not really worth investing into that as a company. Just use something that's out there."
Corman agreed the need for a stable centralized update mechanism is critical, but also warned that because there are so many different technology platforms, especially with industrial control systems, it is unlikely there will ever be a single update process for everyone.
Mitigating compatibility risk
Another way small enterprises could reduce issues of dependency and the potential for an update to break products, Kandek said, is by moving to the cloud. Kandek explained that more cloud services means endpoints become more generic and easier to update without consequences.
This won't be an option for everyone, so Ben Jun, CTO at Chosen Plaintext, said the aim of software developers and vendors should be to increase transparency when updates are pushed.
"An administrator wants to know, in a language they can understand, what a patch is going to do. Unfortunately, where we are today is that developers can't even tell you what the patch does, which is why we have some of the problems we're seeing," Jun said. "They think it's solving one problem, but they've added unintentional side effects. If you look at the code that triggers the flaw, it's a very small amount of code, and it's just as easy to introduce a 10-byte bug into a piece of code as it is to remove one."
Jun said this is a difficult problem to avoid, especially because security updates are often bundled with other updates that can just as easily break code, but the answer may come from hardware.
"Ultimately, I think we're going to get to a place where the hardware, the platform itself, can set limits on what the code can do," Jun said, "and there will be a fairly transparent way that an update applier can use to monitor what they're doing to the system itself."
Jun noted compatibility is one of the main reasons why administrators might delay installing a patch and is also a big reason why Microsoft began the tradition of Patch Tuesday.
"Patch Tuesday didn't come because Microsoft wanted to do it on a Tuesday. Patch Tuesday came because their user base essentially asked that they didn't get updates at random intervals that would suddenly make their networks go black," Jun said. "For Microsoft to remove the Patch Tuesday concept, it means they have a lot more confidence in the stability of the rollout process and systems aren't going to go down in the middle of the night when they push updates."
Corman said he hopes the Microsoft update rollout process for Windows 10 is more stable, because some companies don't fully understand the business value of providing a stable and secure update process. He described a car company that knew the cost and brand impact of doing a recall, but not the flip side where mobile app developers can push updates every day without a negative impact.
"The burden of security people is not to scare people; our challenge is to make the safer thing the easier thing," Corman said. "Since complexity is the enemy of security, it's also the enemy of stability, sustainability and ultimately affects your products and customer satisfaction. I think something like an update has security value, but also has profit margin value and value in maintainability and serviceability."
Learn how to mitigate risks from Windows automatic updates.