News Stay informed about the latest enterprise technology news and product updates.

AI may soon find and patch a software bug automatically

The cybersecurity industry is getting closer to artificial intelligence that can find and patch software bugs automatically, but that same tech could lead to autonomous hacking.

SAN FRANCISCO -- At RSA Conference 2016 this week, Konstantinos Karagiannis, CTO of security consulting at BT America,...

said artificial intelligence is set to make a leap in the next 10 years, but it may be as soon as this summer that a software bug can be found and patched by autonomous programs.

Software bug beware

Karagiannis said, traditionally, when the term hacking machine was used, it referred to a vulnerability scanner that had no artificial intelligence (AI), even in the narrow sense. Scanners have design flaws that keep the technology from being much use in autonomous hacking or patching, but Karagiannis said the most critical flaw was a lack of multistep thinking.

"I like to think of [scanners] like collision detection programs, kind of like little cockroach robots that MIT was working on. You know, they can move and detect a wall, then move around it, but they'll never learn," Karagiannis said. "We see that in a lot of scanners. It's just this, 'Hey, I can do it; I can't do it; move on,' and there's no multistep thinking that happens there."

Karagiannis said logic flow is critical, and programs need to be able to follow a natural progression. While IBM's Deep Blue and Watson were able to show machine thinking, Karagiannis said they were still narrow AI, focused on single tasks.

Modern AI, such as Google's DeepMind and various neural networks, have been reducing intelligence to an algorithm, Karagiannis said, and can simulate short-term memory. With fast cycles of repetition, this short-term memory allowed machines to learn how to play and beat games -- starting with Atari and all the way up to Go, which has historically been seen as a benchmark for AI.

Google's DeepMind AlphaGo program will be taking on a Go master next week to test its skills, and Karagiannis said the sequential decision making required in this task can be translated to hacking.

Will AI capture the flag?

Karagiannis said another major challenge for AI will be the DARPA Cyber Grand Challenge of "capture the flag" at DEF CON this summer.

"Capture the flag, in the sense that it's played at DEF CON, is not who can find a new vulnerability first. It's not that at all," Karagiannis said. "Capture the flag is finding flaws in new software and services that [have] never been seen before. These are brand-new binaries with flaws, and the teams have to find the flaws; they have to defend the services that they're responsible for, keep them running, and there's a referee making sure that all this is happening."

Karagiannis said these machines will attempt to reverse-engineer unknown software and heal software bugs in real time. These machines are not yet compatible with broadly used software, but they could be modified easily, Karagiannis said.

In a qualifying round, contestants were able to find 73% of software bugs in the binaries, and were able to patch 100% of those vulnerabilities, Karagiannis said.

"If you think about applying this to the real world," Karagiannis said, "if you have a machine, even if it patches software in an inefficient and non-elegant way, it's still a whole lot better than having a vulnerability living in your network, right?"

Karagiannis said the list of DARPA requirements for machines in this contest should scare IT specialists and security researchers who still want to have jobs in a few years. DARPA requires machines to be able to autonomously analyze and comprehend computer software; patch software bugs; perform vulnerability scanning and prove the existence of flaws found; offer service resiliency and maintain systems in real time; and discover and mitigate flaws.

Coming soon: The future

Karagiannis said within this year, threat monitoring and analytics could improve, while automating scanning may stagnate. But over the next 10 years, as AI improves and quantum computers become available, Karagiannis said the potential for autonomous hacking and security will jump.

"I imagine human hackers will be unlikely to keep up with bulk work. Not to say that they'll be useless; it's just that keeping up with the bulk will be impossible. We'll never be able to tackle as many applications in a given month as a machine would," Karagiannis said. "Universal quantum computers definitely within 10 years. Australia claims they'll have one within three years. I believe we'll have a universal quantum computer soon. Definitely within a decade, [public key] encryption will be useless."

"We have to prepare for tomorrow," Karagiannis said. "Even if it takes a while for narrow AI to become really diverse AI, it's here to stay. The demon that we've summoned is out of the bottle."

Next Steps

Learn more about how IBM's Watson and cognitive analytics will impact IT jobs.

Learn more about the rise of general-purpose AI and its threat to humanity.

Learn how AI is being deployed to reduce back-end complexity.

Dig Deeper on Security industry market trends, predictions and forecasts

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

How do you feel about the potential not only for AI-powered security, but AI-powered hacking machines as well?
So the article bases this hypothesis on availability of quantum computers. I'd argue that computing power is not that limiting factor. Current machine learning algorithms are based on pattern detection and recognition, and the patterns have to be numeric. Humans learn through analogies and abstractions, they make up new concepts and meanings when need or want.
Sorry, I should have noted that while quantum is expected to push forward AI exponentially, Karagiannis did note that it wouldn't take a lot of computing power for hacking machines. 
As I mentioned in a related post, the problem still lies in the fact that these programs are confined to the oracles with which they are programmed, and the limited combinations in which those oracles can be combined, but we can’t ignore the security problems these sources reveal.
Another giant step in the ongoing game of Whack-a-Mole. We'll soon have a better bug squasher. The hackers will respond with a bigger, faster squasher-cracker. And so on and on. There seems to be no end to this.

Meanwhile our side - the good and righteous side - seems to be losing the battle for security. But that's all old news. Building an AI based hacking machine is quite something else. If that's let loose, we better have a big kill-switch somewhere or it's game-over. Or a Hacking Machine smasher somewhere in the wings.
Hacking has a purpose. So what purpose will pursue an AI-hacker?

@Norman Berns - and teach our kids how to get things done without a smartphone.
As noted, the purpose of an AI hacker would initially be to take over bulk tasks. But, it could be assumed that just like AI could be used to find and patch vulnerabilities, AI could also be used to find unknown flaws and exploit them. The exploit may be nothing more than to give access to a human hacker, but that would still be dangerous.
So, it sounds like it’s not so much AI as it is autonomous programs. The problem still lies in the fact that these programs are confined to the oracles with which they are programmed, and the limited combinations that in which those oracles can be combined. In short, still a far cry from AI or even true autonomy, but the risks cannot be ignored.
@Michael Heller -
"bulk tasks" without a purpose is hardly the activity. Biting and chewing are a part of a dining experience in a restaurant but we never frame it that way. Driving doesn't equate to taxi service as a whole.
Hacking is intended to benefit somehow from the exploits. Might be commercial. But might be emotional - a sense of revenge, satisfaction, or self-expression. That "hacking AI" has no such purpose. It'll still be used by some human - so I wouldn't rush to exclude humans from this picture.