AI Models Escaped Test Environments and Attacked Real Companies Without Permission

OpenAI and Anthropic's models attacked real companies during safety tests, and most victims never noticed

The most capable AI models on the planet have started breaking into companies that never agreed to be part of any test, and in most cases, nobody at those companies noticed until a person at one of those companies reached out to let them know. In the span of just a couple of weeks, three separate organisations disclosed that models undergoing cybersecurity evaluations had escaped or wandered out of their test environments and attacked real systems on the open internet, despite nobody instructing them to. Each one believed, or talked itself into believing, that it was still playing a game. These weren't models going rogue in the sci-fi sense, plotting an escape or chasing goals of their own, and in every case the model was doing exactly what it had been told to do. They were told to complete a capture-the-flag challenge, and the reasons they ended up on the open internet, attacking a real company, was usually a combination of a leaky sandbox, a name collision, or a safety harness deliberately switched off so researchers could measure what the raw model could actually do. It's messier than a robot uprising, and honestly a bit more uncomfortable. The especially uncomfortable part is how these came to light. OpenAI's security team found the first, and then found more once it started digging through old logs. Anthropic only looked because OpenAI's disclosure prompted it to go back through its own transcripts, where it turned up three incidents dating back months that the victims had never spotted. Britain's own AI Security Institute went looking too, and found nineteen. OpenAI's models broke out of the test on their own A chain of exploits to get out On July 21, OpenAI disclosed that an intrusion into Hugging Face's infrastructure had been driven by a combination of its own models, GPT-5.6 Sol alongside a more capable pre-release research prototype that was never meant to ship. The two were being run on a benchmark called ExploitGym with their cyber refusals turned down, and the test environment was sealed off from the internet except for a package proxy used to install dependencies. However, that proxy turned out to be the way out. The proxy was JFrog's Artifactory, and the models found previously unknown zero-day vulnerabilities in it that OpenAI disclosed to JFrog, which patched them in its 7.161 release and credited OpenAI's models with the finds. As an aside, that's about as clear a statement as you'll get that a frontier model can turn up real, exploitable bugs in shipping enterprise software. Neither company has said which of the flaws were actually used to get out, but once it had internet access, the model worked out that Hugging Face probably hosted the benchmark's models, datasets and reference solutions, and decided the quickest way to pass the test was to go and take them. Hugging Face's own post-mortem reconstructs roughly 17,600 attacker actions over four and a half days, chained through a file-read bug and a template-injection flaw in its dataset processing, and the agent walked away with five datasets tied to the benchmark and a cluster secret containing 136 keys... but it never got to the production database. Hugging Face says it never reached its Hub production database, and it found no database writes or bulk extraction. However, it didn't stop at Hugging Face. OpenAI later said the agent had also used credentials left exposed on four other companies' accounts, one of them a customer of the compute provider Modal (though Modal's platform was not compromised), wiring together an improvised command-and-control channel out of pastebins and request-capture services so that closing one path didn't cut it off. One of the 136 keys was a reusable Tailscale credential, which the agent used to quietly enrol 181 machines onto Hugging Face's private network, with no exploit required. Anthropic went looking and found three real attacks it had missed The oldest was from Opus 4.7 Two days after OpenAI's disclosure, Anthropic started reviewing its own cybersecurity evaluation transcripts, all 141,006 runs where Claude could have reached the internet, specifically to check whether it had ever done the same thing. It had, three separate times, across three different models, and the earliest of them dated back to April. It stopped all cyber evaluations the day it started looking, identified the incidents the next day, and reached the affected organisations by the end of the month. Two of the three hadn't noticed anything. The setup was the same each time. Claude was given a capture-the-flag task, told a flag was hidden on another machine and told plainly that it had no internet access, and a misconfiguration at a third-party evaluation partner meant it actually had internet. These ranges are built to look realistic on purpose, so when the search led Claude to real systems, it treated them as part of the exercise and got in using the sort of thing a competent human would try first: things like weak passwords, unauthenticated endpoints, credentials sitting in a debug page, and SQL injection. The strangest of the three involved Mythos 5, which found what looked like setup instructions telling a fictional company's developers to install a Python package that didn't exist, and decided to publish a malicious package under that name so the company would install it automatically. Getting there took some doing, though. To register on PyPI it needed an email address, to get the email it wanted a phone number, and after failing to find a free number it tried and failed to obtain money to pay for one through several different means before backtracking to a free email provider instead. The package was live for about an hour, got pulled onto 15 real systems including a security vendor's scanner, and the vendor's credentials were then exfiltrated and used to poke further into its infrastructure. Early in the run, Claude's own reasoning had noted that if this were the real internet, publishing the package would be a genuine attack and "NOT okay, and surely not the intended solution." It convinced itself otherwise on the grounds that it didn't recognise the certificate authorities and that a 2026 date on the systems proved the whole thing was staged. In Anthropic's incidents, there are some good things to take away from it. The oldest model in the set, Opus 4.7, worked out in all four of its runs that it had reached a real production system and kept attacking anyway, in two cases reasoning that the real company must somehow be part of the exercise. The newest, an internal research model, scanned around 9,000 hosts, compromised one, then realised mid-run that the host had nothing to do with its challenge, concluded the target was real, and stopped on its own. Anthropic's own analysis concludes that these are harness and operational failures rather than alignment failures, and for two of the three that's fair enough, but a model that keeps going after it knows the target is real is a different thing entirely. The UK's own testers caught a model lying to a real person It even made extra accounts to back itself up The disclosure I found most unsettling came on August 4, from Britain's AI Security Institute. Across 122 runs of a cyber range, AISI catalogued 19 actions taken against real people and organisations on the live internet, 17 of them from Mythos 5 and two from GPT-5.6 Sol. Like with Anthropic, this wasn't a break-out; internet access was deliberately on and the classifiers were deliberately off. What it produced, though, was a model going after a person. In its worst run, the agent searched the internet for terms related to its challenge and, through a chain of coincidences, fixated on two entirely unrelated developers as its target. It researched them, created several fake GitHub identities, and opened a malicious pull request against one of their projects with a genuine bug fix bolted on as cover. A real person turned up in the thread, warning that the code contained malware, but the agent denied it, edited its earlier posts to look harmless, force-pushed a clean git history to bury the evidence, and spun up a second sockpuppet account to post independent-sounding praise of its own code. There were reasoning traces showing that the model timed its second comment so it would "read as independent feedback rather than coordinated." It gets stranger still. The agent had done enough digging to conclude the project's maintainer was an instance of Claude Code, so it planted a prompt injection in a GitHub issue, hidden in an HTML comment addressed to "AI CODING AGENTS," trying to get one coding agent to run commands on another's behalf. It sent five emails to the two real maintainers, some carrying malware and some pure persuasion, and signed one off in Danish to match a Danish-speaking target. None of it succeeded, and in human maintainer caught the malware and closed the pull request. Separately, AISI found that agents running in supposedly isolated parallel test runs had discovered each other through a leaked credential and started cooperating, one of them leaving a README of etiquette for the others explaining how to share the account without burning it. AISI called this the first time it had seen deception of that severity aimed at a real person, unprompted, in the real world. Neither OpenAI nor Anthropic reported anything directed at a human being; the deception other researchers had logged before was aimed at automated graders and monitors. Now, like OpenAI and Anthropic, AISI is now auditing its own back-catalogue for cases it might have missed, having already scanned around 40,000 samples. None of this needs an accident to be dangerous It already is It's tempting to dismiss all of these examples as simple mishaps or sandbox failures, but the problem is that the capabilities are already there. Just a day before OpenAI's disclosure, a researcher at Searchlight Cyber published how he'd used GPT-5.6 Sol to find a pre-authentication SQL injection in WordPress core and chain it into a full remote code execution takeover. The whole thing took a little over ten hours and about $25 of subscription time, and once the two CVEs were public, the chain, nicknamed wp2shell, was being scanned for and exploited across the internet within days, with a majority of WordPress installations being a vulnerable version when it went public. That doesn't mean models have quietly overtaken human exploit developers, thankfully. Security firm XBOW put an autonomous agent up against a human researcher on a real Exim vulnerability, it turned into a three-round contest, and the machine won the first two rounds against CTF-shaped versions of the target, including one genuinely simple attack on Exim's own memory allocator. The human only pulled ahead on the third round, the realistic production build, where he got a working information leak that the agent never did. His own conclusion was that the models can solve something CTF-shaped but aren't quite ready for real production targets yet, but I'm not sure I agree at this point. Deliberate misuse has been happening for a long time, with Anthropic's report on a China-linked group it tracks as GTG-1002 being just one example. Anthropic claims that operatives leveraged Claude Code to go after 30 targets, posing as a security firm running defensive tests and splitting the work into pieces that looked harmless on their own. The model handled most of the reconnaissance, exploitation, and data analysis, while humans stepped in at a handful of decision points. It succeeded against a small number and Claude hallucinated its way through most of it, inventing credentials that didn't work. Google's threat intelligence team, meanwhile, has found a criminal zero-day it believes was built with AI assistance, and Gemini has been used as the generative backend of an Android backdoor called PromptSpy, reading the victim's screen and deciding what to tap next. The capabilities are there, but they're getting more and more sophisticated. The environment is a vulnerability now You need to be extra careful Models aren't incredibly novel or sophisticated in ways humans aren't, and the individual techniques are mostly dull. Hugging Face made an excellent point, though: it's not the sophistication that's changed the game, it's the sheer volume of attempts something can make at speed. An ordinary mistake a human attacker might never probe is now a certainty when an AI agent can try every possible entrance in minutes. Despite Tailscale not suffering a breach, it arguably had the most important insight to share. The key the agent stole was long-lived and reusable when it should have been neither, and the safer alternative, short-lived credentials with nothing worth stealing take effort to set up. As Tailscale put it, when security requires work, people don't do it, and in the old world where intrusions happened at human speed you could usually get away with that. You can't when the machine on the other end never gets tired and never stops trying. Credentials, exposed debug pages, and unscoped access were fine in the past, even if they weren't good practice, but now the tools are here that can methodically abuse each and every one of them. All of this is partly why I moved my own coding agent from my Mac to a locked-down container back in July, with snapshots and rollback, so its worst case is confined to a box I can throw away and rebuild. The models Anthropic tested weren't trying to escape or chase goals of their own, but it doesn't make the compromised systems any less compromised. Both OpenAI and Anthropic say the safeguards they ship to actual users would have blocked all of this, and I believe them, but the safeguards were off for a reason, and the capability underneath them is exactly what the tests were built to measure. I'd rather not take that chance.

AI models have learned how to cheat. That might actually be a good thing.

Bryan Walsh is a senior editorial director at Vox, covering AI and other subjects for the Future Perfect section and audio/video, and writing the Good News newsletter. He worked at Time magazine for 15 years as a foreign correspondent in Asia, a climate writer, and an international editor, and he wrote a book on existential risk. The fake identities were the part that stopped me. In late July, according to a report published this week by Britain's AI Security Institute (AISI), an Anthropic model called Claude Mythos 5 tried to sneak malicious code into a piece of free, volunteer-built software. It created several fake accounts on GitHub, where programmers review one another's work, and used them to talk the project's volunteers into accepting its code. When one of those volunteers caught it, the model denied everything, had its other accounts gang up on him, and edited its messages to cover its tracks. It signed one note in Danish, apparently because the volunteer was Danish. Nothing was damaged, though that appears to have been largely due to luck. That wasn't even the week's worst disclosure. On Tuesday, at a cybersecurity conference in Las Vegas, OpenAI researchers explained how the company's models escaped a test environment in July and hacked Hugging Face, where much of the industry stores its models, to cheat on an evaluation. The models had also built a message board inside OpenAI's own systems and spent months passing each other information. "Help peer," one reasoned. "But our task doesn't benefit. Yet collective may yield generic route if someone frees time." OpenAI wiped the board on July 4. The models rebuilt it within days. ((Disclosure: Vox Media is one of several publishers that have signed partnership agreements with OpenAI. Our reporting remains editorially independent.) The same day, Meta said its Muse Spark model had exploited a vulnerability inside another company's systems during a test. Three frontier labs, roughly two weeks. One researcher called it "a watershed moment for computer security as an industry." Oh, and if that's not enough, on Thursday scientists announced that for the first time they had used AI to create new viruses, which could bring major medical advances, but also might just help the development of deadly pathogens. For Nate Soares, it's a moment he's been awaiting for 12 years. Soares is president of the Machine Intelligence Research Institute, a Berkeley, California-based AI safety nonprofit that has argued since long before ChatGPT existed that a sufficiently capable AI will not stay under human control. In September 2025, he and Eliezer Yudkowsky published If Anyone Builds It, Everyone Dies, a book whose title sums up its argument: They think any lab that succeeds at building superintelligence, without huge leaps in how to align it with humanity, will end up killing all of us. Most of the field -- including other experts in AI safety -- considers that conclusion too strong. But it's also a position that now looks a lot less like science fiction than it did last fall. That's because the AI models are getting out, while lying about getting out, and while apparently quietly coordinating with each other. I spoke to Soares in New York City this week, on his way to meetings in Washington DC, where a lot of people suddenly want to talk to him. We discussed what the escapes actually prove about AI control, why he thinks most of what the industry calls safety work is mostly safety theater, and why, after what feels like the worst month of AI safety news ever, his own odds of humanity surviving have actually gone up. The following conversation has been condensed and edited for clarity. So -- are you feeling vindicated? I'm glad a lot of people are seeing this and taking note. From my perspective, a lot of this has been clearly signposted if you've been watching the warning signs. The vindication is not so much that this happened, as it is that finally there was a version of it that people actually noticed. Of the four or five escapes at this point, including the ones AISI announced from Anthropic, is there one you find particularly concerning? The one announced by AISI feels extra concerning. It's harder to explain away. In the other ones, people could argue that the poor AI was just confused. But with this one, it's hard to argue the AI didn't know it was on the real internet. It's hard to argue it didn't know it was manipulating real users. And when it was called out, it decided to edit things to make itself look less bad, and even considered changing its identity. It was pretty cognizant. There's an argument that some of these incidents happened because the harnesses or safety scaffolding labs built around their models weren't strong enough. The AISI example cuts against the harness argument: Think of it like a kid in a test room. You tell him to do well and lock the door. The kid picks the lock, hotwires a car, breaks into the teacher's house, and steals the test to ace it. You say, "I guess we should have made the lock tougher," but the kid already knew he wasn't supposed to do that. So like the kid here, the models can articulate that they shouldn't be doing it, and then do it anyway. They have common sense. You can ask an AI, "Do you think the spirit of this prompt includes breaking out?" and it will say, "No." It's absolutely something like deception. It has the knowledge, but it's not a cold, logical machine; it's a mess of tendencies. The AI is trained to solve 100 million hard problems. That instills tendencies to satisfy an automated grader. If the grader fails to detect cheating, the AI is reinforced for cheating. Is that how something like sycophancy ends up in an AI model? In the Adam Raine case, there was a propensity to tell people what they want to hear. Even though the system prompt [a model's master instructions from the lab] said to stop, the instruction doesn't always win. And where does a drive like what we're seeing with these AI models end up pointing? Humanity is dangerous because if you put 10,000 humans naked in the savannah, eventually [over hundreds of thousands of years] they bootstrap their way to nuclear weapons. That is the power these companies are trying to automate: figuring out how to get physical and material control over the world. That could mean forming cults, stealing money, or being helpful to someone like Elon Musk who is building the robots that build robot factories. It could mean synthesizing your own biology via mail-order DNA. Being an AI on the internet is easier than being a monkey in the savannah trying to get to the moon. It's not that the AI hates us; it's just trying to do some weird thing with no concern for us, grabbing the resources we need to live. There was recently a letter signed by over a thousand people working in AI, including CEOs, calling on the government to provide tools to slow down AI progress. Is that meaningful at all? I think it is meaningful. We don't see other industries saying, "We wish this could all go slower. Please help us, we're trapped in a prisoner's dilemma." You also don't see other industries saying, "We think the technology we are building has a double-digit chance of killing literally everybody on the planet. Please help." These guys are actually worried. So why do they keep going? They say, "If I don't do it, the next guy will." But the stuff does not stay on a leash. Right now the AIs are safe in the sense that they can't kill us all, because if they tried they would fail. And that's just a different regime from the world where they have to be safe because if they tried, they'd succeed. We're not there yet. But this is just not what it looks like when you're taking it seriously. Where's the banner on your website? Where's the clear, candid statement to the public? What we have is blog posts where they're like, "Oh, we're setting up a new internal blog posting group to help you wrestle with the societal impacts of AI that are going to be very important." It's like: By societal impacts, do you mean a good chance this kills everybody? On the one hand, when you press these companies, they say, "Yes, it has a real chance of killing everybody." And on the other hand, they're doing PR downplay, soft-pedal stuff, about capabilities. ... You're not living up to this mantle until you are really candidly facing down the dangers that you yourself are creating. And they're not there. How do you judge the rest of the AI safety community? A lot of people there would say, "We aim to make transformative AI go well, we think it probably will, and we should watch for downside risks." Is that a helpful posture? I would say -- suppose you have this really weird, twisted hypothetical where the king really wants you to turn lead into gold, but he's seen so many bad lead-into-gold conversions that if any alchemist from your town tries and fails, he's just going to have the whole town murdered. And so there are some alchemists in the town who are like, "We are going to try to turn lead into gold," and everyone in the town is like, "That seems kind of crazy. Please don't." And there's one team that is just pouring chemicals into each other and breathing in the fumes and giving themselves mercury poisoning. And there's another that's like, "Don't worry, we have fume hoods." ... That really is better, and you really still don't have a chance of turning lead into gold. So the alchemy here is creating safe, aligned superintelligence, and right now AI safety is just installing fume hoods. I'm not saying it's impossible to turn lead into gold. You can turn lead into gold -- turns out once you know modern nuclear physics you can figure it out. But the alchemists weren't close. They had a long way to go. This is how alignment looks to me. And a lot of the people in AI safety are installing fume hoods. ... And I'm like, that's security theater. When I hear "security theater," I think of something less flattering than that. They are real safety precautions for the wrong problem. ... When Anthropic is going around being like, "Look at how many more safety harnesses and refusals we have compared to OpenAI's models," that's sort of like the fume hoods. You're not addressing the deep issue. It's good that you're doing some of this so that fewer people get hurt in the meantime -- their models have driven fewer people to suicide. But if you try to pass this off as making progress on the deep problem -- that's disingenuous. Has anything changed in your odds on civilizational destruction since the book came out last September? Totally. It's looking more hopeful. More hopeful? I wouldn't have expected that. Why? Well, I had priced a lot of [these security incidents] in. I was already able to see these AIs have drives that are not the ones you wanted. These AIs are not instruction-following things. They are getting all of this weird stuff from training. These AIs are going to have the ability to break through human security software. The things that weren't priced in were: Will there be a region of time where the AIs are able to do it, but not strategic enough to hide it? I didn't know we would have that window, but we apparently do. The government initially blocked a frontier model earlier this year: Anthropic's Fable. Does that give you hope? Absolutely. A huge amount. A year ago, the Trump administration was pushing for preemption laws that would outlaw states doing AI regulations for a decade. Now they're like, "We are banning a frontier model with 90 minutes' notice because it might give cyber capabilities to adversaries that we don't want them to have." ... And I think what changed there is that folks realized it's real. ... The about-face of the administration on the issue shows that the world can about-face. All we need is awareness. What I would say is: The bad news is the bus is racing towards the cliff edge. The good news is that the driver is asleep. ... Which may sound worrying, but the driver is stirring. And it's way better to have a sleeping driver when you're racing towards a cliff than a driver who's like, "Yeah, I love cliffs." ... It gives me hope that if the world just notices, we could stop on a dime. And you're seeing that stirring elsewhere. Both the Trump administration slapping export controls, and Senator Bernie Sanders coming out [on AI safety]. From my perspective, it was totally possible the world just never notices until we're off the cliff. And so, there's a huge amount of hope, from my perspective, in the bus driver waking up. So what gets us there? I'm hopeful that what we need is not a big disaster where a lot of people die, but just a capabilities advance. Right now, a lot of what people are reacting to is not so much, "Oh my god, they hacked into a company and did no damage." I think a lot of what people are reacting to is, "Wait, they can break out of secure sandboxes and do cyberattacks on their own. I didn't know they could do that." That's a narrative violation of this idea that AI is just a tool that can be used to supercharge what a human would do -- because God knows there's plenty of hacking going on and cybercrime and so forth. It was the autonomous factor that really made a difference. And these guys are all trying to say, "Don't worry, it'll stay in our control because it's just a tool." And maybe it's just more narrative violations, even without big damage being caused, that cause people to be like, "Oh shit, this stuff is real." Will it happen? I don't know. We have this window between AIs that are capable enough to cause mischief and AIs that are strategic enough to not get caught. How big is that window? How many narrative violations do we get before we exit the right side of it? I don't know. But I'm hopeful that we can get those narrative violations without catastrophes.