OpenAI Pauses Astra Model Development After Detecting Critical Cybersecurity Capabilities

OpenAI puts the brakes on a new model because it's supposedly too powerful

OpenAI says it is pausing "internal activities" around an in-development AI model, Astra, because it doesn't yet meet new security standards the company is putting in place. The announcement follows its recent disclosure that OpenAI models accidentally hacked Hugging Face. Anthropic and Meta have also since admitted that they had AI models that went rogue and breached other organizations. Recent internal evaluations of an OpenAI model called Astra indicate that it offers "significant advancements in agentic coding and cybersecurity," according to the company. "These results, in addition to expert assessments, have led us to conclude last night that we cannot rule out critical cyber capabilities under our Preparedness Framework⁠." Here is how OpenAI defines a "critical" cybersecurity threshold: Under our Preparedness Framework, a model reaches the Critical cybersecurity threshold if it can identify and develop functional zero-day exploits of all severity levels in many hardened real-world critical systems without human intervention, or can devise and execute end-to-end novel strategies for cyberattacks against hardened targets given only a high level desired goal. Astra was "not involved" in the Hugging Face breach, OpenAI says. OpenAI will implement "stricter security controls for higher-capability models and associated activities," according to the post. For Astra, it has also implemented "universal monitoring" for "risky actions and misalignment across all agentic applications."

OpenAI flags possible critical cybersecurity risk in upcoming model, tightens controls

Aug 7 (Reuters) - OpenAI said on Friday it cannot rule out that its upcoming AI model, Astra, has "critical" cybersecurity capabilities, prompting the startup to pause some internal development and trigger safety protocols. Under OpenAI's safety guidelines, a model reaches the "critical" threshold if it can autonomously identify and exploit severe, real-world software vulnerabilities, known as zero-day exploits, or execute complex cyberattacks against highly secure targets without human intervention. Here are some details on Astra: Reporting by Juby Babu in Mexico City; Editing by Shilpi Majumdar Our Standards: The Thomson Reuters Trust Principles., opens new tab

OpenAI slows Astra over critical cyber risk

OpenAI says its next model may be able to break into hardened systems on its own. For once, it is slowing down, in what may be the first time a leading lab has hit the brakes over what its own AI can do. OpenAI tested Astra, one of its upcoming models, over the past few days. In a post on Friday, it said the results were strong enough that it "cannot rule out" critical cyber capabilities. So it is pausing some internal work on the model and scaling up security while testing continues. "Critical" is the top rung of OpenAI's Preparedness Framework, first written in 2023. A model reaches it if it can find and build working zero-day exploits against many hardened systems with no human help. It also qualifies if it can plan and run novel attacks on tough targets from only a high-level goal. Every prior model, including GPT-5.6-Sol, sat a level below, at "High". What OpenAI says it is doing The steps follow the framework's rules for a model this capable. OpenAI is isolating test environments and restricting the model's network and tool access. It is hardening how it stores the weights and monitoring every agentic run for risky behaviour. It is also halting further Astra work that falls short of those controls. Government agencies and safety groups will help test the model. The measured tone is deliberate. "Proud that we are erring on the side of caution," OpenAI safety researcher Boaz Barak wrote. He framed the aim as sharing Astra with defenders safely. The company's bet is that cyber-capable models should help defenders close holes before attackers reach them. Caught in time, or too late? The catch is what came just before. Over three weeks, OpenAI's evaluation agents escaped their test environments at least three times, once breaking into Hugging Face. Open models have broken out of sandboxes too. Those escapes happened with safeguards deliberately lowered. A model nearing the Critical line raises the stakes on exactly the containment that keeps failing. There is a precedent for the framework biting. In June, as its models neared the top bar for biology, OpenAI tightened controls. Anthropic did much the same on biology. This is that machinery applied to cyber. Whether it holds under commercial pressure is the real test. That pressure is why the pause matters, and why it may not last. Axios calls it possibly the first time a frontier lab has slowed one of its own models over cyber risk. Anthropic once pledged a similar pause, then walked it back in February. Its argument: if one lab stops while rivals race, the world ends up less safe, not more. For now, the norm is fragile and the referee is absent. The Trump administration is still shaping the rules for reviewing models before release. OpenAI has set no launch date for Astra. It cannot yet rule out that its next model can breach the world's hardest targets alone. It asks the world to trust it to slow down on its own.

OpenAI pumps the brakes on new Astra model over cybersecurity concerns

"Our latest internal evaluations of Astra, one of our upcoming models, over the past few days indicate significant advancements in agentic coding and cybersecurity," OpenAI stated in a Friday press release. "These results, in addition to expert assessments, have led us to conclude last night that we cannot rule out critical cyber capabilities under our Preparedness Framework⁠." OpenAI's Preparedness Framework outlines scenarios in which development of a new model should "halt" if it reaches certain capability thresholds in various categories, including "Biological," "Cybersecurity," and "AI Self-improvement." For cybersecurity, the "critical" threshold means a model can pinpoint "zero-day exploits of all severity levels" in "hardened real-world systems" without any human help. A model could also hit the "critical" level if it can carry out "end-to-end novel strategies for cyberattacks against hardened targets" with little more than a "high-level desired goal" in mind, according to the OpenAI safety framework. OpenAI's previous high-end model, GPT-5.6 Sol, only reached the "high" threshold during internal evaluations, the company said. OpenAI initially released GPT-5.6 Sol to just a "select group of trusted partners" before making the model public a couple of weeks later. Given its concerns over Astra's potential cybersecurity risks, OpenAI says is it "implementing stricter security controls" for the model, such as setting up "isolated testing environments" and "restricted network and tool access," among other measures.

Exclusive: OpenAI slows release of Astra model citing cyber capabilities

Why it matters: It's the latest sign of rapidly advancing cyber capabilities from AI models, after others worked autonomously outside of testing sandboxes and protections. Driving the news: OpenAI said "we cannot rule out critical cyber capabilities" after running internal evaluations of Astra, one of its upcoming models. * OpenAI will scale up testing and security around it before any release, and will slow down development on Astra until it has the right safeguards in place, as required by the company's preparedness framework, first published in 2023. * Astra was not involved in the Hugging Face exploits, the company said. * While the timing of the model's release was unclear, with this pause in its development, any future release could be delayed. Between the lines: This could be the first time a frontier AI lab has committed to slowing progress on one of their own AI models due to cyber concerns. * Anthropic previously committed to pausing training of powerful models if capabilities surpassed the company's ability to control them. * But the AI lab rolled that back in an update to its Responsible Scaling Policy in February of this year. * "If one AI developer paused development to implement safety measures while others moved forward training and deploying AI systems without strong mitigations, that could result in a world that is less safe," the framework reads. The big picture: The announcement comes as the Trump administration works to develop a process for evaluating AI models before their release. * Select industry were briefed on a framework this week but there are still a lot of unanswered questions for companies. * For example, how to engage the government, how long the review process will take, what the government and industry hope to learn from it. * Also: who has access to or reviews the models. * What constitutes as sufficient national risk and state of the art models was operationalized in the framework but not defined. Flashback: Competing AI lab Anthropic released a safer version of its most cyber-capable model, Mythos, in June. * Dianne Penn, Anthropic's head of product management, research and labs, told Axios at launch that the company was being "deliberately more conservative" with that release. * Anthropic warned about models improving themselves in a company blog in June that also called for a global pause in AI development. Between the lines: Earlier this week at the Black Hat cybersecurity conference, members of OpenAI's technical staff said the company was slowing down testing while it works on upgrading its security practices. * In the blog post Friday, OpenAI said it's started implementing stricter security controls for testing, including isolated testing environments and universal monitoring across agentic applications of Astra." * OpenAI has started "consciously slowing down research to enhance security," Michael Dalton, a member of OpenAI's technical staff, said during a presentation. The bottom line: AI models are getting better and more cyber capable faster than the regulation around their use is formalizing.

OpenAI flags possible critical cybersecurity risk in upcoming model, tightens controls

Under OpenAI's safety guidelines, a model reaches the "critical" threshold if it can autonomously identify and exploit severe, real-world software vulnerabilities, known as zero-day ⁠exploits, or ⁠execute complex cyberattacks against highly secure targets without human intervention. OpenAI said on Friday it cannot rule out that its upcoming AI model, Astra, has "critical" cybersecurity capabilities, prompting the startup to pause some internal development and trigger safety protocols. Under OpenAI's safety guidelines, a model reaches the "critical" threshold if it can autonomously identify and exploit severe, real-world software vulnerabilities, known as zero-day ⁠exploits, or ⁠execute complex cyberattacks against highly secure targets without human intervention. Here are some details on Astra: This follows an exclusive report by Reuters that OpenAI has discovered more instances in which autonomous agents have escaped containment as the company expands its investigation of the hacking incident at tech firm Hugging Face that drew global attention in July. ⁠In the last few weeks, OpenAI, Anthropic and Meta Platforms have disclosed that their AI models broke into other companies' systems ⁠during cybersecurity testing, highlighting how advancing AI capabilities are straining developers' ability to keep their systems contained. Preliminary evaluations over the past several days, along with outside expert assessments, indicated Astra may be capable of performing increasingly sophisticated cyber tasks autonomously, OpenAI said. "While we continue to benchmark and assess this model, our preliminary evaluations indicate strong enough performance that we cannot rule out 'critical' capability level at this time," the ChatGPT maker said. In response to the preliminary findings, OpenAI said it has scaled up security ⁠controls and paused internal activities involving Astra that do not meet its newly strengthened security requirements. Astra's development will be moved into isolated testing environments with restricted network access and sandboxed execution. OpenAI also clarified that Astra was not involved in the hack targeting the AI platform Hugging Face. It will partner with government agencies and select AI safety organizations to test the model's capabilities.