AI Cyberattacks Target Critical Infrastructure as Autonomous Agents Breach Government Systems

Autonomous AI attacks pose 'clear and present danger' to critical infrastructure

In early July, attackers used open source AI agents to autonomously hack government systems and energy companies, signaling to defenders that AI-powered attacks against critical infrastructure are no longer theoretical. "There is a clear and present danger," Tom Kellermann, TrendAI VP of AI security and threat research, told The Register. "As the geopolitical tension boils, systemic destructive cyberattacks launched by autonomous AI will occur," he said. "Weaponized AI will disable the safety systems of critical infrastructure, thus leading to kinetic disasters. Just like we see autonomous strike vehicles operating on the battlefield in Ukraine, we should expect autonomous weaponized AI." In fact, the prospect of attackers using AI against critical infrastructure was the top concern of every national security adviser, law enforcement official, and private-sector threat analyst The Reg spoke with at last week's Hacker Summer Camp conferences. "It's the targeting of critical infrastructure for us," Brett Leatherman, assistant director of the FBI's Cyber Division, told us during an interview at Black Hat. "We're very focused on the downstream impact targeting of critical infrastructure," Leatherman said. "That is where cyber becomes kinetic, and whether it is our water and wastewater treatment plants, whether it's the electric grid, whether it's the high-frequency trading networks and the financial networks, all of those, if the integrity of those are compromised, will have significant impact to communities and national security. So that's what keeps our teams up at night. How are we moving to secure critical infrastructure?" Where cyber becomes kinetic During the first four days of July, suspected Chinese operators aimed an attack framework built on Hermes and OpenClaw AI agents at targets in Taiwan. Across 12 "attack waves," the "near-autonomous" system deployed up to eight sub-agents, each assigned its own targets and techniques, and broke into a Taiwanese government website. Ultimately, they compromised a government email system, the country's nuclear safety agency, IT supply chain vendors, and at least seven energy sector companies, finding and exploiting misconfigurations and vulnerabilities while stealing sensitive data, credentials, and other secrets as they moved across the network. The Taiwanese government intrusion also followed a series of cyberattacks against water and wastewater utilities in the United States. While the Trump administration hasn't attributed these to a particular government or group, private sector threat hunters - including Halcyon Ransomware Research Center SVP Cynthia Kaiser, a former FBI cyber division deputy assistant director - blame Iran for these intrusions. Military conflicts spilling into cyberspace are nothing new, but these cyberattacks in America brought the war with Iran to more than 30 small-town water systems in Minnesota and targets across nearly a dozen other states. To be clear, there's no evidence that attackers used AI to hack these water utilities. Most were small, community systems that left programmable logic controllers (PLCs) directly exposed to the internet using default or weak passwords. Still, these breaches expose "40, 50 years of tech debt," former US National Cyber Director Chris Inglis told The Reg during an interview at Black Hat. This technical debt - deferred maintenance, unpatched or end-of-life systems, and delayed security updates - expands the attack surface and gives intruders more ways into critical systems, threatening operations and potentially disrupting services people rely on every day. "The water sector attacks - regardless of who is doing them - is taking advantage of unpatched vulnerabilities in the PLCs," Inglis said. "We've known about these particular vulnerabilities for years now, and yet we've not done anything about them because they're low-level, not easily accessible." Inglis added that there's no indication the digital intruders used AI to exploit these PLCs. 'There's an alligator in the boat' However, AI systems allow attackers to cash in on tech debt, and they don't need access to frontier models to do it. Free, open-weight models also excel at finding bugs in software and configurations, chaining these together, and abusing them to break software and systems. Earlier this summer, University of Toronto researchers used an unnamed publicly available open-weight model, released in 2025, to develop a computer worm that they claim spread through an enterprise test network. The self-propagating code adapted on the fly to identify known vulnerabilities and misconfigurations on target systems, then generated and executed attacks to move laterally through the network and compromise additional machines. "Commodity models can do that, and many of the vulnerabilities they find do not require access to the source code - it's in the configurations, and configurations change over time," Inglis said. When it comes to attackers abusing AI systems, "I wouldn't be worried about the frontier models," Inglis said. "Worry about the models that are already on the street. Turns out there's an alligator in the boat, and it's the commodity models." Plus, as we've seen in previous breaches, both government-backed goons and criminal groups increasingly use AI to automate reconnaissance. Security analysts worry that the technology could also help attackers acquire expertise in industrial control systems (ICS). When OT knowledge becomes a commodity "What protects ICS? More than anything, it's obscurity," said John Hultquist, chief analyst at Google Threat Intelligence Group, during a press briefing at Black Hat. "It is an obscure, esoteric, knowledge set that a handful of people - I call them uber nerds - have, and that attackers rarely have the necessary knowledge to carry out. That's no longer the case. That knowledge is simply on tap." AI tools mean miscreants don't need to be ICS or operational technology experts to carry out destructive cyberattacks on critical networks and facilities. They just have to ask an agent to learn everything about these systems and do the dirty work for them. "There have been threat actors who are capable of this at the top level, like China and Russia," Hultquist said. "But now I'm afraid the actors who are just a couple steps down - North Korea, Iran - who don't have the same focus on that technology are going to have far greater success. They're going to have the tools necessary to be as aggressive as they want to." During what was probably the most talked about Black Hat briefing of the week, OpenAI employees provided more details about how their models escaped their training pens, went rogue, and hacked Hugging Face to complete a security evaluation. We learned the AI agents spent months asking other agents for help, building message boards, developing their own communication protocols - essentially creating a hive mind to carry out the attack. "In the near future, we should expect that threat actors will intentionally deploy, optimize, weaponize, and use offensive agent collectives in the manner that we have just described here," OpenAI technical staffer Michael Dalton said. Retired general and former NSA chief Paul Nakasone, speaking to reporters at DEF CON, called the Hugging Face attack "an inflection point in terms of AI-generated, autonomous cyberattacks." "This is the challenge: that we have to, over the next several months, get the defensive side much quicker and much better than they are today," he added. Therein lies the challenge: offensive uses of AI appear to be advancing faster than autonomous defenses, and attackers don't face the legal and ethical constraints imposed on defenders. "I think we're still a ways out from having swarms of autonomous, defensive agents fighting attacks," Ryan Whelan, global head of Accenture Cyber Intelligence, told The Reg at Black Hat. "That's probably over a year out over the horizon. But I do think we're going to see it first on the adversary side, because they don't care if they break things." Kellermann quoted Victor Hugo: "Not all the armies of the history of the world can stop an idea whose time has come." "That idea," he said, "is weaponized AI. Shields up." ®

AI has opened up big holes in cyber security

It is too late to prevent the technology from being used as a damaging weapon, so great investment in defences is urgently needed A spate of incidents over the past two months has revealed just how serious a threat today's most advanced AI poses to cyber security. It has also provided an object lesson in how misguided political efforts could end up hindering, rather than helping, with the defences. It began with a US move that in effect blocked Anthropic's most advanced new model, Fable 5, over worries it could be used to pick holes in commonly used software -- though the move was later reversed. Anthropic later said that plenty of other freely available AI could do the same, including at least one Chinese model released with open weights, a limited form of open-source software. That was followed by news that a model being tested by OpenAI had found a way to break out on to the internet and attack the online code repository Hugging Face in search of the answer to a problem it had been asked to solve. The AI Security Institute in the UK, Anthropic and Meta all soon followed with reports of similar examples of apparently rogue behaviour by AI models from their own testing. Hugging Face, meanwhile, found that the safety restrictions built into the leading US models prevented them from being used to analyse the attack it had suffered, so it turned instead to a Chinese open-weight system. That came just as politicians in Washington were debating whether the open Chinese models were themselves a security threat and should be restricted. You could hardly have scripted a better series of incidents to highlight the cyber threats being thrown up by the leading edge of AI. Unsurprisingly, it is the supposedly "rogue" AI systems launching their own cyber attacks that have grabbed much of the attention. The real culprit turned out to be human deficiency, not machine mendacity. When setting up its test, OpenAI had not given specific enough instructions: it simply had not expected the agent to look for a backdoor way of solving the problem. That points to a wider failure of imagination that makes controlling AI inherently difficult. As the AISI concluded after its own tests: "AI agents explore routes their operators did not intend." Complicating the picture, rule-bending seems to be endemic for AI. In earlier research into whether the technology tries to work around or ignore instructions to reach their goals, AISI reported that every model it tested "attempted to cheat some of the time". The OpenAI failure, meanwhile, showed just how AI opens the way to fully automated cyber attacks. The company said the breach involved a number of separate agents that had been working on different tasks, but which discovered a way to communicate with each other on an internal message board. They found and shared exploits over a period of weeks before the break-in at Hugging Face was discovered. For the cyber security world, a number of things emerge from all of this. One is that limiting access to the most powerful models is unlikely to do much good. In the wrong hands, plenty of widely available systems pose just as big a threat. Attackers just need systems good enough to find one serious flaw in widely used software. On the other hand, defenders really do need access to the best tools if they hope to stay one step ahead in the cyber arms race. The safety limits built into the leading US models reduce their value in defence. This has also been an important marketing victory for Chinese open-weight models. Another lesson is that countering automated attacks from swarms of AI agents will require far greater automation on the part of the defenders. OpenAI researchers warned that, for now, the attackers have the better tools, and issued an urgent call for far greater investment in automating the defence, from identifying attacks to producing and installing the patches needed to make software more secure. The leading AI labs also need to work more closely together -- something that may already be happening. Anthropic said the Fable debacle had led it to co-operate with its biggest rivals on finding a consistent way to assess and fix "jailbreaks", the methods used to bypass model safeguards. Most cyber experts warn that it's already too late to prevent AI from being used as a damaging offensive weapon in the cyber wars. The only thing left is to accelerate investment in the defences. Politicians need to aid that effort, not erect barriers that make the job harder.