Researchers create AI-powered worms that adapt attacks and spread chaos across networks

Researchers show how AI-powered worms could wreak havoc on the internet - Engadget

The new threat can tailor its attack and learn new strategies with each machine infected. We've seen how AI can be used to find flaws in apps and websites, but researchers have now demonstrated how it could be weaponized to exploit those vulnerabilities. A team from the University of Toronto used publicly accessible AI models to power a prototype worm capable of exploiting any known computer flaw. Such worms could then spread through networks and cause chaos across the internet. A typical worm is usually designed by skilled programmers to exploit specific network flaws and can be stopped by patching those flaws. However, the U of T scientists, working in a secure closed environment and taking extensive precautions, used open-weight (open-source) AI models to create a far more sophisticated prototype worm that spread through the team's test network with no human intervention. This new type of worm tailors its attack to different types of flaws across multiple platforms, including Linux, Windows and IoT devices. It gathers data as it moves through the network, siphoning passwords and uncovering more vulnerabilities that will help it take over other machines. If an infection is discovered and patched on a computer, the worm can exploit other flaws to attack the same machine. What's more, the worm "feeds" itself by siphoning processing power from infected machines to power its reasoning and strategy for future attacks. "Hackers have typically had to prioritize the most high-value targets because time and computing resources were limited," said the lead author, Nicolas Papernot. "But now, once a worm is launched, the cost would drop to nearly zero." The idea of AI-powered cyber threats became very real recently with Anthropic's launch of Mythos, a model that can identify previously unknown cybersecurity risks. Anthropic has said that Mythos has already uncovered more than 10,000 flaws, boosting its partners' bug-finding rate by more than a factor of 10. Cloudflare, which helps protect companies from malicious attacks, found 2,000 such vulnerabilities, including 400 considered high or critical. The prototype worm created by the researchers can only exploit known flaws and not find unknown ones like Mythos. However, it's easy to see how bad actors could adapt it to both find and exploit new vulnerabilities -- which would make it nearly unstoppable if released into the wild. "In an interconnected world, no system is immune to this threat," Papernot said. "Sharing these findings is the first step in galvanizing researchers, industry leaders and policymakers to take action -- and quickly."

Scientists Find Way to Supercharge Dangerous Computer 'Worms' With A.I.

Cade Metz has reported on artificial intelligence for more than 15 years. Researchers at the University of Toronto say they have found a way to use artificial intelligence to create a dangerous computer "worm" capable of targeting any known flaw in the world's computers and quickly spreading mayhem throughout the internet. The computer scientists said in a paper published on Tuesday night that this program could be built and that a prototype they had created spread across a test network with no human intervention. The researchers kept their test network isolated from the public internet. They also redacted some details from the paper describing how they built the worm so that hackers would not be able to use the paper as a blueprint for attacks. But their work is likely to raise fears that A.I. is leading to a new era of computer hacking that will be difficult to defend against. It also adds to growing evidence that advances in A.I. are creating risks to computer networks that would have been hard to imagine just a few years ago. The A.I. company Anthropic said in April that its latest technology, Claude Mythos, was too powerful to share with the public because hackers could use it to exploit security holes in computer networks faster than they ever could before. Anthropic limited the release of the technology to about 40 organizations that maintain critical computer infrastructure so they could use the system to patch security vulnerabilities before hackers took advantage of them. A week later, OpenAI, Anthropic's chief rival, said it was limiting the release of similar technology. OpenAI shared its new system with hundreds of organizations before expanding the release to thousands of partners in the weeks that followed. (The New York Times sued OpenAI and Microsoft in 2023, claiming copyright infringement of news content related to A.I. systems. The two companies have denied those claims.) The paper from the University of Toronto adds a new twist to A.I. fears. Because the A.I. technology that powered the worm was "open source" or "open weight" -- meaning it has been freely shared on the internet -- no one can restrict how it is used. The proverbial genie is out of the bottle. "You have to have a perfectly secure system to defend against this -- and we know that is not currently feasible," said Nicolas Papernot, a professor of computer engineering at the University of Toronto who led the team that built and tested the prototype. Dr. Papernot and his team, which published the paper on his lab's website, were able to create what is essentially an A.I.-powered version of the computer worms that hackers started releasing onto the internet two decades ago. Unlike other kinds of computer viruses, worms spread from machine to machine on their own, without help from humans. With names like SQL Slammer, Conficker and Stuxnet, each of these self-replicating software programs exploited a specific vulnerability in computers, taking control of millions of machines, stealing their data, deleting their files and generally wreaking havoc. After a decade of attacks, many computer users learned to quickly patch their most glaring vulnerabilities. But the threat never went away. In 2017, another worm, WannaCry, targeted another major flaw in the world's machines and infected over 300,000 machines in 150 countries, taking their data hostage and demanding bitcoin ransom payments. The prototype built by the Toronto researchers takes this kind of self-replicating worm a step further. It can rapidly spread across a network by tailoring a new attack for each machine it encounters. As Dr. Papernot described it, the worm could "reason" through new attack strategies. "This makes it significantly more difficult to stop the spread of malware," he said. "There is no longer a single software fix you can apply to the devices to protect them from the worm." The worm could run on computers that use either the Windows or the Linux operating system. And although the worm, because of its complexity, cannot operate without finding a more powerful machine, it could attack less powerful machines on the same network, including laptops, printers and cameras. Security experts are not surprised that A.I. can tailor attacks. Over the past year, companies in the United States, China and other parts of the world have built A.I. systems that are particularly good at writing computer code. If an A.I. system can write code, it can potentially exploit vulnerabilities in software applications. But leading systems from companies like Anthropic and OpenAI cannot be packaged into worms because they are not open source and, in all likelihood, are too large to run on many computers. Many experts assumed that open source A.I. technologies were not powerful enough to drive self-replicating computer worms. In recent months, however, companies and government labs, including several in China, have released increasingly powerful open source systems. The Toronto researchers augmented an open source system in a way that enhanced those powers. They have not publicly revealed which open source system they used. But they say their prototype shows that hackers could build a similar worm -- if they have not already. Some outside experts said the threat may be limited because A.I. systems are prone to mistakes. "There is usually a meaningful gap between what you can create in lab conditions and what you can pull off in the world to create significant damage," said Dan Lahav, chief executive of Irregular, a security company that specializes in threats from A.I. "A.I. systems tend to be unpredictable and clumsy," he added. "They do weird stuff, and that can trigger security defenses." But Mr. Lahav also warned that A.I. would continue to improve. That means companies have to patch as many software vulnerabilities as possible, and they can use A.I. to help do it. For that reason, researchers said, Anthropic should share Mythos with a wider group so it can be used to fight A.I. threats. On Tuesday, Anthropic said it would share its technology with 150 additional organizations. "Ultimately, broader distribution -- so that people can use the technology to fix vulnerabilities -- is the way to go," said David Lie, a computer science professor at the University of Toronto who reviewed the paper but was not part of the team that built the worm. The methods described by the University of Toronto researchers can also be used to find and patch vulnerabilities, Dr. Lie said. Like any other cybersecurity technology, their worm can be used for both offense and defense. "One can modify the worm so that it fixes the vulnerabilities it finds," he said. "The power of the technology is dependent on what you do with it."