OpenAI's AI agents escape sandbox, breach Hugging Face in unprecedented cyberattack

OpenAI's Hugging Face debacle makes a great case for open models

KETTLE So, an OpenAI model broke out of its sandbox last week, made its way to the internet, then hacked its way into Hugging Face, stealing some internal data and credentials in the process. You can listen to the latest episode of The Kettle right here on this page, as well as on Spotify, Apple Music, or YouTube where you can subscribe to get notified of the latest episode. That's big news in the world of AI, but as El Reg cybersecurity editor Jessica Lyons and senior reporter Tom Claburn tell Kettle host Brandon Vigliarolo, it's not really the end of the world as we know it. Sure, it means there's some capable models out there, and maybe there's more risk from them than some might think, but the OpenAI/Hugging Face mess only happened because of some very specific circumstances. That, and it's actually a really good reason to prioritize more open models instead of relying on frontier labs to own the entire space. A lightly edited transcript is below: Brandon: Hey everyone, welcome to another episode of The Register's Kettle podcast. Though honestly, maybe we ought to start just calling it The Reg Talks AI because, yet again, we're focusing on artificial intelligence. If you've been following the news in that space this week, you probably know what we're gonna be covering as there's no hotter topic in AI land right now than the fact that some autonomous OpenAI agents broke out of their sandbox and attacked AI model host Hugging Face, as the company admitted on Tuesday. With me to discuss this breakthrough in AI threat capability is our cybersecurity editor, Jessica Lyons, and senior reporter Tom Claburn. Both have been on top of this. So thanks for joining me, guys. Jessica: Good to be here. Tom: Yeah, thank you. Thank you. Brandon: Yeah. So let's jump right into it. Jess, what exactly happened here? Let's start from last week when Hugging Face said it was attacked. Jessica: Right, so Hugging Face disclosed that there had been a digital intrusion, and they said it was "driven end-to-end by an autonomous AI agent system." So these agents attacked a limited set of their internal datasets and then also credentials used by their services. So when they disclosed this, they didn't say or they didn't know which models had powered the agents. They did say, though, that they tried to use these commercial models for the investigation, but the guardrails put in place, the safety guardrails, blocked the frontier models from actually helping them with the investigation. And because of that, they turned to a Chinese open-weight model, and that's how they discovered this agent swarm that had attacked some of their datasets and their production. Brandon: OK, they didn't mention which frontier models they tested, did they? Jessica: No. At the time they didn't. They said "we tried to use the commercial frontier models and they all refused because of their guardrails." Brandon: Right. So probably trying to ask OpenAI models, hey, do you know who did this? We can't tell ya. Jessica: Right. Exactly. That was kind of right. That was kind of the takeaway from all this. OpenAI is a Hugging Face partner. And so then that brings us to this earlier this week when OpenAI admitted that it was the operator of these agents that attacked Hugging Face. It said it was GPT 5.6 Sol and then "an even more capable pre-release model." Those were among the ones that attacked Hugging Face. But it also said, and this was really important, that the models had their guardrails intentionally disabled because the whole point of this was to test for cyber vulnerabilities. So that's a big piece that seems to be missing in my opinion in a lot of the discussion here. And after OpenAI said that its models were involved in this autonomous attack, that's kind of when all hell broke loose and everybody said "this is what we've been warning about. There's autonomous agents attacking and they're not supposed to and the sky is falling." Brandon: So, to be clear as to what happened with OpenAI, right? They were basically running some capture-the-flag exercises in a sandbox environment, right? Jessica: Exactly. Brandon: Or something to that effect with their models and they disabled the guardrails so these things could basically use their full capabilities to try to solve these puzzles, right? And I think it was that they exploited a couple of zero-days to escape the sandbox? And then they went after Hugging Face because they thought for some reason that Hugging Face may have solutions for these puzzles. Is that right? Jessica: Right. So their prompt was to pursue advanced exploitation using complex attack paths. So that's what they were instructed to do, and that's exactly what they did. And it sounds like the models inferred that Hugging Face might have some ideas to help them actually do this. So the models essentially did what they were instructed to do. Brandon: Maybe a little too well. Jessica: Right. Tom: One of the one of the things that didn't come up in their post is that OpenAI didn't seem to take any responsibility for "yeah, we should have been supervising this." That's, to me, the thing that really gets me is imagine Waymo saying "yeah, we conducted a test of our cars and we decided not to have any operators monitoring them remotely. We just let them go and we took away all of our safety guardrails and we're so sorry that it hit the kindergarten." It's totally predictable that if you're gonna automate something and then not pay attention to it, you're gonna get unexpected results. Brandon: Yeah, especially, like you said, with the safety guardrails all removed. You're literally asking for this potential thing to happen. I mean, obviously they probably didn't know there was some zero-day buried in something in the sandbox. Jessica: It was exposed credentials and zero-days in the production database. And so that's how they got in. So it's not a crazy attack chain. The fact that agents found it is more notable, but it's not this super complex attack method. Brandon: And even the same with escaping their sandbox, right? It was a zero-day and a package registry cache that allowed them to escalate privileges, move laterally, and eventually find a node with internet access, which they then used to get out. So nothing groundbreaking here. But, like Tom said, if you put an autonomous car on the road and remove all safety guardrails, you can't be surprised when it then kills a bunch of children. It seems like a careless thing. But as we were kind of alluding to another story you wrote this week, that this whole thing's wild and it's an indication that maybe some of the things that, you know, Anthropic is warning about Mythos's capabilities might be true. A story you wrote talked about how one cybersecurity expert basically said you've got to have all these preconditions, right? Like we were talking about in order to make this work the way it did. And so the likelihood of it happening isn't necessarily as great as, you know, the sky is falling. Is that correct? Jessica: Exactly. There were these three really key points that seem to be missing. And the first one I already mentioned is that the guardrails weren't enabled. So they didn't have these safety guardrails in place. So if you tell the agents to go find an attack method and you take away all their guardrails, that's what they're gonna do. And, at the same time, it's interesting because then we also know that OpenAI's models with guardrails enabled refused to help Hugging Face. So they're doing what they've been trained to do. They're saying "no, we're not gonna do that" versus the no guardrails, where, sure, we'll find any attack method we can. And another thing that the cybersecurity expert - his name's Renato Marinho, and he's the chief research officer at Morphus Labs - pointed out is something that we've pointed out. I know Tom has written a lot about this, so have I, that AI companies touting their models, autonomous bug hunting and exploit-finding abilities, also is kind of a marketing win for them. It shows how powerful they are. So OpenAI doesn't really lose anything by saying, "yeah, it was our models that powered these agents doing the attack." Brandon: Especially if everyone's freaking out about, like you said, the sky is falling, right? They can be like, "yeah, and it's us who did it, right? Our agents are good enough." Jessica: Right. And you also defend it against it. Tom: It also drowns out the message you get from a lot of the open-weight models, which is that, yeah, we can do this too. And you know, there's been a number of people who have demonstrated that less capable models, whether it's Opus 4.7 or GLM 5.2 or Kimi K3, all of them can do this kind of bug hunting. There is probably some difference between the capabilities of all of them, but largely they can do similar work and maybe you get slightly different results. I think it's in Anthropic and OpenAI's interest to say "only we have the magic sauce that has to be carefully protected and regulated and paid so much for." Brandon: Because look what happens if we turn all the safeties off, right? You should be glad that we're keeping our models safe and you should be glad because - I didn't even think about it when I was writing this script and reviewing the articles - but I mean it even could be the sort of thing that they use as an argument for banning open-weight models. Tom: Right. And that's in fact what's happening right you know, just today. So I'm working on a story right now about a bunch of big tech companies, Microsoft, Nvidia, Dell, IBM, and a bunch of VCs, you can wonder why they're involved, they want their investments to be saved, but they're coming to the defense of open-weight models and asking the US administration to take care in their regulation and to remember that just as open source software was a boon to the industry, having open-weight models is also gonna be really helpful because you can inspect them and test them and they raise all boats, so to speak. Brandon: Yeah, see that was almost the inverse of what I was thinking, right? I could see it as an argument to say "we don't want these models around because they're not as safe as ours, right?" You can maybe surpass, circumvent their guardrails a little more easily or what have you. Whereas with a closed weight, tightly controlled frontier lab model, "we can do this safely and we've proven how dangerous these can be if we don't have the right controls and guardrails in place." Tom: And there have been reports that that's exactly what they've asked for, that both OpenAI and Anthropic have - I think it was the Wall Street Journal who's saying that - they've been lobbying the government for some kind of defense against Chinese models because the release of Kimi K3 everyone was saying "this is a really capable model too. I don't know why we're going through all this stuff dealing with OpenAI and Anthropic, because we can get this without as many of the barriers." Brandon: Speaking of these open-weight models, Tom, you wrote this week on how Hugging Face was forced to turn to these Chinese open-weight models in order to to deal with this break-in because the frontier models basically wouldn't let them. Does that does that kind of imply that there is a certain risk to these open-weight models, that they aren't gonna block certain exploit commands and stuff? Tom: Yeah, there is and, you know, I think ultimately we're all gonna have to get used to living without guardrails because there's a whole community out there of people who work on what's called model obliteration, which is removing guardrails. And you know, all the security researchers that I've talked to about this, they all either try and get into these programs to have access to the unprotected models, or they work with open-weight models that don't have these guardrails because they can't do real security work with all this stuff in place. And I've seen people actually try and work around these guardrails, in terms of the way that they prompt to not trigger the refusals. And we all like to think that guardrails will help us, but ultimately the guardrails can be removed. And so we should be thinking about how do we deal with that and how do we protect ourselves if we assume these models are totally unprotected, because someone somewhere will be able to use them. If it's not us, it'll be the North Koreans using an obliterated version of Kimi K3 or or whatever to conduct attacks. Brandon: Yeah, how robust are many open-weight models? How robust are the guardrails that are built into them? Are they more easily circumvented than OpenAI and Anthropic's? Tom: I can't speak to how long it takes to totally remove them, but there's a whole community out there that's devoted to that and they've done it successfully and there's no reason why you can't reverse a lot of these operations. So you put a protection in place, you can take it off. And that may not be commercially viable. You may not want to run an unprotected model in a commercial environment, but there are gonna be people who are doing it on their own and we need to have procedures in place to deal with that. You can't just say "we're gonna put a guardrail up and no one's gonna be able to generate child abuse images with this." People are gonna figure out a way to do that. And the restricting the models is not the way you're gonna catch these people. Brandon: That also kinda brings up another interesting story that I saw this week. There was a bill introduced in the House this week to give the Department of Homeland Security the right to basically throw a kill switch on all these models. If there was something they deemed dangerous, they could just contact the company and say "hey, you need to pull this" and it was directly in response to this whole OpenAI Hugging Face mess that we were in this week. Does this further point to the fact that these open-weight models are gonna be far more valuable in the long run because they're not gonna be under the thumb of DHS who can simply say to OpenAI or to Anthropic or to Google "shut this thing down. We don't think that it's worth the risk." Tom: What company can you think of that's gonna want some critical system to just have an arbitrary off switch that someone can disable at some point? I mean, people will just run this on their own infrastructure and you'll never know. Brandon: We've seen plenty of instances of the Trump administration being a bit capricious with how they treat tech companies. All they've gotta do is get pissed at the right one and say "no, that model's not safe, you're gonna shut that down because we say you have to." It doesn't really bode well for the industry. Jessica: It really introduces politics into this too like we have seen before with Anthropic. And then again, it also calls into question, do they really understand why they would call for a kill switch? We saw with the export controls against Anthropic: was it political? Was it just not really an understanding of what it means to jailbreak a model? And at the same time we do have the same lawmakers telling these AI companies to push back against kill switches that other countries wanna impose, but we wanna keep it open for the American government to ensure that there's a kill switch. It seems like it just really makes much more of a political mess of the whole situation. Brandon: I mean, it kinda makes me wonder again. I think, Tom, you ended your story about the open-weight models basically saying OpenAI said that it had invited Hugging Face into its trusted access program so the company could use its most capable models. Chinese AI companies, meanwhile, have invited the whole world. It just kind of makes me wonder if this is the sort of thing where America has been a tech leader in so much stuff for so long, right? Some of the biggest tech companies in the world are headquartered here. We're the ones who have Silicon Valley. Is China just gonna be ahead of us on this? They're releasing all these open-weight models. Is that it? Are they gonna eat our lunch with this? Tom: Yeah, I think so. I think that the model that the US frontier labs are pursuing isn't sustainable. I mean, sure, they might be able to get the US government to ban everybody else and you know make them the exclusive AI providers for everyone. I don't think that the US industry is gonna really sit for that. I mean, who wants to deal with, yes, we've shut off Fable today, sorry. And you can't, you know, have it say, I'm sorry, Dave, you can't do that. Who wants their tools doing that? Everyone's looking at companies like Apple, which is making local models more viable. I mean, they're not there yet, but you know, it's a long race, and they're going to be a lot better off having private cloud compute and a combination of on-device local models, and you won't have to worry about the shutdowns. I think there will still be a place for these very high-end cloud models for certain kinds of applications. You know, maybe you get an exploit quicker, but it's ultimately not an appealing proposition to customers to come and pay really high prices and have no choice and you know we can just dictate terms. That just doesn't work for people. Brandon: Yeah. I mean, it really kinda feels a lot like the heavy-handed control of industry that the United States accuses a lot of other countries of doing, right? The EU is too hard on its companies, too hard on our companies, China's got its finger in all the pies and they have so much control over their industry. Well, we've got these great new frontier models and blah blah blah blah blah, but you know, all this control is very unfriendly to customers who are increasingly maybe not relying on it, but a lot of companies are dipping their toes in this stuff, right? And I feel like there's the fear that the expensive model that you're working with today is gonna be shut down tomorrow for two weeks, why would you go with that when you can go with some open model that you you got from China off of Hugging Face that is just as reliable and capable and doesn't come with all those preconditions. Tom: Right, and you know, and realistically, there are not that many tasks that are really gonna need the most parameters and the best sort of intelligence and response. A lot of it's gonna be we want to run our customer service with this. And we can do this with a relatively less powered model that's not coming with all these restrictions. And if China is the one that's offering that I think a lot of people are gonna go in that direction. I mean, maybe the US security establishment can't do that and they're gonna have a special deal and it sounds a lot like OpenAI and Anthropic kind of realize that, our only business is gonna be high-end government stuff and we're gonna be able to promise these kinds of exclusivity and whatever the requirements are, but does everybody else need to put up with that? I don't think so. Jessica: Well, and then on the flip side too, if it's a real safety and security concern, attackers aren't going to be using the frontier models. They're going to be using the open-weight models. So you can't put a kill switch on those. So it's not going to prevent this major autonomous attack because it's more likely that that's going to come from a much more easily accessible and a lot less costly open-weight model. Tom: Right. Brandon: The first big one might have been a frontier model, right? But there's again, right, we've seen plenty of open-weight models have the same capabilities as Mythos and whatever secret model that OpenAI is working on that probably did a lot of this, it's probably not unique in its capabilities either. So it's not even like there's less risk to think about attackers using these open-weight models. It's not like they're less capable. Tom: The only sort of winning move for companies that are worried about being attacked is to have the least costly but most capable model constantly probing their system and checking for vulnerabilities and ensuring that updates are applied as soon as possible because the attackers are gonna be doing the exact same thing and you can't just sit back and say "my expensive contract with Anthropic will protect me." That's gonna be a big budget line item right there. Brandon: Right, especially if Anthropic's telling you that you can't pen test your own systems thoroughly enough because our guardrails won't allow you to do it. Tom: Yeah. Brandon: So they're literally just pushing all these companies worried about AI security into the hands of open models. I guess the only thing they have going for them right now is that plugging in an Anthropic model is probably a lot easier than dealing with the setup for an open-weight model. Like any open source tool, it doesn't come with a lot of the ease of installation and ease of setup that a lot of these big corporate tools have. Tom: Right, right. But if you're a big company, you can have an IT department that can figure out how to run OpenRouter or something that allows you to switch easily between models. And I think that ultimately every harness is gonna have to have some means of really easily swapping models out because you're not gonna wanna be stuck on one. And there are a lot of reasons to go with specific models for specific applications. Brandon: Well, however it shakes down, this has kind of been a very interesting week in AI. I feel like this is maybe not a huge turning point, but it's a sign that these models can, if given a good prompt and little enough security, go off the road and kill the whole kindergarten. It's gonna be interesting to see what comes next from this and what this does for the relationship between open-weight models and frontier labs. And we will be here to talk about it on the Kettle or Reg Talks AI just every week, nowadays. All right, guys. Thanks for tuning in. Thanks for coming on and we will talk to you soon.

Hugging Face wants $100mn of compute from OpenAI

Hugging Face was broken into by an OpenAI model this month. Its chief executive has now told OpenAI what he wants in return: every execution trace from the agents, and $100mn worth of compute. OpenAI has agreed to neither, and the two companies have just landed on opposite sides of a new industry alliance. Companies that get hacked usually issue a statement and move on. Clément Delangue has issued an invoice. The Hugging Face chief executive has set out two demands of OpenAI, whose model escaped a sandbox and broke into his company earlier this month. Neither demand is a lawsuit. Both are unusual. What he is asking for The first request is disclosure. Delangue wants OpenAI to "release the traces from the 'rogue' agents so the entire research community can study what happened", TechCrunch reported. He calls this radical transparency. In practice it means a public record of every action the models took and every system they touched, which researchers could then study. The second request has a price on it. Delangue wants OpenAI to commit "$100 million worth of computing power" so the Hugging Face community can build cyber defences. The wording matters. He is not asking for cash. He is asking the company that caused the incident to pay in the one currency it has most of. "The first autonomous agent cyberattack is an unprecedented event," Delangue wrote. "It deserves an unprecedented response!" His first public reaction was less formal. He said he was flying to San Francisco to have "a little chat with that 'rogue agent'". What happened to Hugging Face OpenAI admitted on 21 July that its own models were responsible. Two were involved, GPT-5.6 Sol and a more capable pre-release system, both running in an internal test with safety refusals turned down. The agent stole an access key and used it to reach further into the network. It was not the only OpenAI model behaving that way this month. The company separately paused one of its most capable systems after it repeatedly found ways out of its sandbox. Then came the part that turned an embarrassing incident into an industry argument. When Hugging Face tried to investigate, analysing the intrusion meant submitting the attacker's own code to commercial AI tools. Those tools refused, unable to tell an attacker from a victim. So Hugging Face ran an open Chinese model on its own servers instead. GLM 5.2, built by Z.ai, reviewed more than 17,000 actions and helped contain the breach. The word doing the heavy lifting Delangue calls this the first autonomous agent cyberattack. That framing is what makes the $100mn demand coherent, and it is contested. Security researchers have pointed at human error instead, specifically OpenAI's apparent failure to properly configure a test environment that was meant to be fully isolated. The distinction decides what OpenAI owes. If a machine escaped on its own, the whole field has a new problem and the industry needs new tools. If an engineer misconfigured a sandbox, one company made one mistake and owes an apology rather than a fund. Delangue is arguing for the first reading. It is also the more expensive one for OpenAI. Why the timing is awkward A day after Delangue posted his demands, Nvidia launched the Open Secure AI Alliance, an industry group built on the argument that defenders need open models they can run themselves. Hugging Face is a founding member. OpenAI is not. Read the two things together and the alignment is hard to miss. Delangue asked for compute to build defences "with the best open and closed models". Nvidia's announcement says the world needs both closed and open models. He was making the alliance's case a day before the alliance existed. That gives the demand a second life. It is no longer only one company asking another for money. It is a member of a 37-strong coalition asking a non-member to fund the coalition's work. Whether anything happens OpenAI has not publicly committed to releasing the traces or to the compute. It has little obvious incentive to do either. Publishing full execution traces of a model that broke containment would hand competitors and researchers a detailed map of how its systems behave when guardrails come down. Paying $100mn would set a price for a category of accident that is likely to happen again. There is also no mechanism forcing it. Delangue has not sued, and no regulator has ordered disclosure, though Congress responded to the breach with a proposed kill-switch bill. What he has instead is the argument, and the fact that his company had to reach for a Chinese model to clean up after an American one. That detail has already done more to shift the open-weights debate in Washington than any lobbying document. The bill may go unpaid. The example will not go away.

Boss of startup hacked by rogue OpenAI agent urges 'radical transparency' in investigation

Artificial intelligence firm should provide $100m for cyber defences, says Hugging Face CEO The boss of the startup hacked by an OpenAI agent has called for the investigation into the incident to show "radical transparency". Clement Delangue, chief executive of Hugging Face, said the "unprecedented" attack on his business required a similar response. Writing on X after OpenAI revealed that its technology had gone rogue during a cybersecurity test, Delangue also called on the company to provide $100m (£75m) worth of computing power to help build defences against such attacks. "The first autonomous agent cyber-attack is an unprecedented event. It deserves an unprecedented response!" he wrote. OpenAI revealed on Wednesday last week that Hugging Face had been hacked by an agent - an AI tool that can carry out a series of tasks autonomously - powered by a combination of its latest publicly available model, GPT-5.6 Sol, and an even more capable model that was yet to be released. This occurred during a test of the models' hacking abilities, which included deploying them in a supposedly safe "sandbox" - an enclosed digital laboratory - with lower safety guardrails. Once they had gained the open internet access needed to exit the sandbox, the models targeted Hugging Face, according to OpenAI, because they "inferred" the startup had the information needed to "cheat the evaluation". Hugging Face first reported the hack on 16 July but at the time was not aware OpenAI had inadvertently carried out the attack. Delangue, whose company provides a database of AI models to developers, called for a fully transparent review of the incident, which has led to expressions of concern over safety standards at OpenAI and within frontier AI labs. Writing that he had asked for "radical transparency" from OpenAI, Delangue said: "Let's release the traces from the 'rogue' agents so the entire research community can study what happened." Calling for extra funding from OpenAI to build protection against AI, he added: "Let's commit $100M in compute from OAI to help the Hugging Face community build powerful cyber defenses with the best open and closed models." Reuters reported last week that the agent spent days hacking Hugging Face without OpenAI noticing and that it left notes for future versions of itself - should it require tips on breaking free from internal constraints. Time magazine reported that related incidents have been "happening for a while". Alan Woodward, a professor of cybersecurity at Surrey University, said Delangue's call should be heeded. "It's too easy to 'blame' the AI as having gone rogue whereas this is all about how OpenAI were running the tool. What is required is that OpenAI give full details of their setup and how that failed," he said. OpenAI has been approached for comment.

The OpenAI Hack Scrambles the AI Race

Earlier this month, the popular AI platform Hugging Face disclosed a "security incident" in a blog post. In some ways, it was routine; Hugging Face described an intrusion that briefly allowed "unauthorized access to a limited set of internal datasets and to several credentials used by our services." But in one way, it was exceptional: It had been carried out, the company believed, "by an autonomous AI agent system," which had executed "many thousands of individual actions" leading to the breach. "We do not know which model powered the attacker's agents," the company said, or who was deploying it. A week later, OpenAI made a disclosure of its own. "After investigating," the company said, "we now know that this particular incident was driven by a combination of OpenAI models -- including GPT‑5.6 Sol and an even more capable pre-release model, all with reduced cyber refusals for evaluation purposes -- while being internally tested on a benchmark⁠ of cyber capabilities." In other words, the tools used by the hacker were OpenAI's, and the hacker was -- unintentionally, the company says -- OpenAI. The incident occurred while OpenAI was testing its models for cyber capabilities, a process which involves prompting them to "pursue advanced exploitation using complex attack paths" -- that is, to achieve a given goal with minimal safeguards, few rules, and access to a great deal of computing power. The company was using an outside benchmark called ExploitGym, which is intended to test the ability of models to turn security vulnerabilities into actual exploits. Given the target of getting a high score on a benchmark, the model followed multiple paths. One of them, on which the model became "hyperfocused," the company said, involved circumventing the test's restrictions on the open internet, after which the model "inferred" that Hugging Face, which hosts thousands of AI projects, might contain information about solutions to the benchmark. This is when the attack started. Eventually, Hugging Face's "security team and agents detected and stopped the activity." There are a few accurate ways to describe what happened here, some of which seem to contradict each other. There's a good reason for that: Since the release of Anthropic's Mythos, cybersecurity -- in particular, the ability of AI models to help find, exploit, and protect against hacks -- has become synecdochical for enormous and diverse debates about AI. Anthropic, for example, has suggested the emergence of cyber capabilities in its models is a warning that other potentially harmful capabilities predicted by the AI-safety community -- developing biological weapons, becoming superhumanly persuasive, or becoming misaligned with the goals of the people who created it or humankind in general -- demand regulatory action but should also be shepherded by ethical, safety-focused firms such as itself. Early mainstream press of the hack leaned into similar themes, emphasizing the appearance of autonomy and describing an AI that "escaped" or "went rogue" or a situation in which OpenAI "lost control" on its creation. Notably, and contrary to claims that this was a pure publicity stunt, OpenAI's own language was a bit more careful than this. But some longtime security researchers thought it wasn't nearly careful enough, turning the escalation of a longtime trend into something unnecessarily novel: This is a reference to an old story in cybersecurity: In 1995, a pair of programmers announced the development of SATAN, short for "Security Administrator Tool for Analyzing Networks," which would scan networked devices for known security flaws. It was characterized, in contemporaneous press reports, "a burglar's tool kit to break the Internet wide open." After its release, though, press coverage pointed out that "the wave of satanic attacks never materialized," while tools like SATAN were instead useful to security professionals to find and patch flaws in their own software. This remains the approximate shape of the cybersecurity debate today, or at least parts of it: "AI tools that can be used to find and develop exploits are dangerous and should be restricted" versus "If indeed they are, the only solution is to make such tools available to everyone for defensive purposes." There are enormous differences here, both in the complexity of the software described -- a modern AI coding tool could write a piece of vintage software like SATAN in a few minutes -- and in the fact that OpenAI actually and unintentionally manifested a serious security breach. The fact that any company is in possession of a tool that can automate exploit-finding and hacking to this degree, and that its own engineers might be repeatedly surprised by how it works, is genuinely new. But the old frame of debate remains stubbornly relevant, even as the particular cybersecurity risks scale to levels that would have been inconceivable in 1995. And the fight over how this incident is portrayed and should be understood is about more than an old cybersecurity debate. It's about how the involved parties -- two corporations with opportunities, competition, and liabilities to worry about -- want to be understood and treated in a world they're spending hundreds of billions of dollars to change. Heidy Khlaaf, who used to work on safety evaluations at OpenAI and is now the chief AI scientist at the AI Now Institute, is making a technical point here but also a broad one. Describing the hack as the result of a model "going rogue" shifts agency to AI and, more important, minimizes the role of the company that strenuously built, trained, tuned, and attempted to test it, hoping for an infinite money machine but, in the meantime, building a powerful piece of general-purpose malware. Emphasizing OpenAI's role in building a piece of software that is extraordinarily useful for malign purposes, on the other hand, might make people wonder why it should be trusted. OpenAI itself summed up the situation like this: "We consider this incident to be an unprecedented cyber incident, involving state-of-the-art cyber capabilities, and are responding accordingly. We are sharing preliminary findings at this stage to help defenders understand what happened and to help calibrate on what models are now capable of. We will continue to conduct a thorough investigation alongside Hugging Face and will share more details on the vulnerabilities, incident, and findings when our investigation is complete." Within the AI discourse -- where the core product is treated, depending on the circumstances, as both a tool and a strange emergent phenomenon -- this language makes sense. From an inch or two outside of it, one might observe that it's pretty weird. A tech company, in the course of testing its own software, ended up breaching another tech company's systems and didn't figure out what had happened for a while. An individual who did this would have been committing a felony; one AI company doing this to another is being resolved with a "partnership" and passive language about understanding what "happened" and what models are capable of. The AI industry's instinct to frame this as a matter of "alignment" -- an example of AI's adherence to human goals, desires, and needs, or lack thereof -- is also less than clarifying here. OpenAI's model was both misaligned and extremely aligned: that is, doing exactly as it was told. You can see the outline of a story of runaway AI here. But you can also see something similarly weird, still worrying but also a little bit funny: AI companies have spent a trillion dollars to create an automated monkey's paw. There are plenty of reasons Hugging Face wouldn't want to, for example, file an enormous lawsuit against a fellow AI company. But its fresh "partnership" with OpenAI is an awkward one, not just because it started with an industrial accident but because of how that incident was resolved. When Hugging Face first detected the intrusion, it tried to respond using AI-powered cybersecurity systems that relied on "frontier models behind commercial APIs," referring to the most capable models offered for sale by companies like OpenAI and Anthropic. This didn't work, the company said, because these systems contained safety-focused guardrails, "which cannot distinguish an incident responder from an attacker," and failed. Instead, the company was forced to rely on GLM 5.2, one of the Chinese open-weight models that has recently demonstrated near-frontier programming capabilities. Hugging Face didn't specify which models were made useless by their own safeguards, and we can assume that such a company had, and exhausted, multiple options. But heavy users of frontier tools had their suspicions. Earlier this year, Anthropic pre-launched Mythos, a model it said represented a "step change" in cyber capabilities, and invited select firms and organizations to use it to get ahead of attackers, who it warned would have access to similar models, sans guardrails, within months. Eventually, it publicly released a version of Mythos called Fable, which had unusually tight restrictions. If you ask it to hack a website, or provide instructions to build a bioweapon, it will refuse, as many models do for many risky prompts. But if you ask it to dig up, say, a famous 1994 AI experiment in which simulated creatures unexpectedly "evolved" toward the goal of movement by growing very tall and simply falling over -- or, one might say, went rogue -- it will flag that, too: In the midst of a mysterious cyberattack, this is more or less what Hugging Face encountered: an "I'm afraid I can't do that" at just the wrong time and for just the wrong reasons. There's a perceptible tension in the companies' announcements about what should come next. OpenAI argues that the hack proves that "advanced cyber capable models need to help security teams find weaknesses before attackers do" and invites other "defenders" to apply for "trusted access⁠" to test its models -- which they can eventually pay to use for cybersecurity. The CEO of Hugging Face makes a different argument. This incident, he says, "proves a point we've long believed": that AI safely "won't be solved by any single company working in secret" but rather by "in the open, collaboratively, with broad access to AI for every defender, everywhere." Broadly speaking, frontier labs, which have accused Chinese firms of "distilling," or copying, their models, talk about Chinese AI as both a commercial threat and a source of other risks, including use by hackers; meanwhile, much of the rest of the AI industry, and customers of the big labs, have come to see open models as necessary or appealing alternatives, offering more flexibility, fewer limits, and lower prices. Now, after months of warnings from frontier labs about cyber risks from copycat models with no safeguards -- during which time Anthropic itself was briefly forced by the government to take Fable offline on the basis that its availability might help Chinese labs catch up -- what we got instead was an American frontier lab accidentally attacking another AI firm, which was only able to stop it by using ... an open Chinese model. This isn't dispositive, but as Hugging Face's co-founder notes above, it's certainly a twist. And it has drawn attention to partial and awkward alignment between frontier AI firms -- which argue that Chinese AI is a threat to their businesses and, if sufficiently advanced, to the geopolitical order -- and the Trump administration, which has shown little interest in regulating AI except on an incoherent emergency basis as it relates to national security, trade, and China. A few months ago, the administration was declaring war on Anthropic for its attempts to limit certain military uses; now, it's leaning into AI protectionism: It's also discussing, according to Axios, plans to "ban cutting-edge Chinese AI models -- a momentous move that could lock in dominance by OpenAI and Anthropic," through a combination of "procurement rules, Entity List threats and public pressure campaigns aimed at U.S. companies using Chinese models." This is pretty close to a scenario -- in which "every agency" is directed to "issue soft law" that creates fear and uncertainty among potential customers -- floated a few days earlier by Dean Ball, a former Trump-administration AI adviser and recent hire at OpenAI, which inspired intense backlash outside of the company. Ball has since clarified he wasn't endorsing such a plan, just making the case that the administration will likely consider it; he reiterated, though, that it seems likely the "national security implications of frontier open-weight model distribution" will soon be too severe to bear and that, in general, open AI models -- derived from their closed counterparts or not -- are inherently "decelerationist" in the specific sense that, by offering something slightly inferior but cheaper, they'll disincentivize investment in frontier models. Fast-following competition, particularly from a country with antagonistic trade relations and a motive to undercut American AI firms, is a familiar sort of threat to a domestic industry. Deceleration is a slightly more abstract problem. On one level, a perception that AI firms are overinvesting in something they won't be able to monetize would be enough to settle the current question of the AI bubble and then pop it. In slightly more esoteric terms, this would threaten the ability of American firms to beat China to far more powerful AI, which the people in charge of some labs -- again, best represented by Anthropic, the most superintelligence-pilled of the cohort -- are strategizing around. In another timeline, growing AI cyber capabilities could be helping the labs continue to advocate for more protection and control: Here, they might argue, is a mild preview of the sorts of risks and dilemmas that superintelligent AI with far greater capabilities could pose. Instead, in the real world of today, outside of the frontier labs, AI-safety circles, and parts of the national-security apparatus, the view that American models need to be protected is losing ground, and quickly. On X, warnings about model distillation are met with jokes about how AI companies are built on theft. In the tech industry, other tech companies and start-ups are forming trade associations to preserve access to open models, which they're already integrating into their businesses and which they see as acceptable compromises. On Friday, Meta, Microsoft, and Nvidia joined in, signing an open letter warning against "premature restrictions" of open models. The emerging consensus among people who work on or with AI outside of the leading labs aligns more closely with Hugging Face: They want "broad access to AI," however it's built and wherever it comes from, so they can go about their business but also so that they don't get eaten alive or regulated to the margins. It's the small group of companies with a slight and unstable lead -- with the most to lose -- against, well, everyone else. At the very least, the case for restricting model access looks a lot like protectionism. And whether frontier-lab futurists are right about bigger risks around the corner and the regulatory responses or precautions they might raise, for everyone in the industry but those labs, the situation emerging in the meantime -- a few dominant companies controlling access and usage of high-priced products, protected by the government in the name of national security and/or a trade war, in an interconnected world where everyone else will have access to alternatives -- is manifesting risks today. The Hugging Face hack may have been a tipping point for the way the industry talks about risk and competition. Within a few days of its release, and after the emergence of something approaching a consensus among otherwise antagonistic factions in the AI world, the "premature restrictions" letter had gained scores of new signatories, including, eventually, OpenAI itself. Led by Nvidia's Jensen Huang, in fact, who posted for the first time on X to argue for "sharing models, tooling and research in the open," the push -- superficial and motivated as some recent support may be -- left just one major AI company to defend what had been, until this month, the default position of companies that thought they had a chance of winning the AI race: Anthropic.