Open-Weight AI Models Match Frontier Capabilities But Refuse Zero Harmful Requests, SaferAI Finds

Open-weight AI models are catching up to the frontier. The safety gap remains.

As policymakers debate how to govern increasingly powerful AI systems like OpenAI's GPT-5.6 Sol and Anthropic's Mythos, a Chinese open-weight model has narrowed the gap with the industry's leaders. GLM-5.2, the open-weight AI model from China's Z.ai, is only a few months behind OpenAI's GPT-5.5 and Anthropic's Claude Opus 4.7 on cyber and bio capabilities, according to a new report from AI safety nonprofit SaferAI. But the divide between frontier capabilities and safety practices is growing. According to SaferAI's evaluation, which the nonprofit ran via Z.ai's public API, GLM-5.2 refused none of the offensive cyber or dual-use biology tasks it was given. By comparison, Claude Opus 4.7 "refused so consistently that SaferAI could not complete CyberGym on it at all." (CyberGym is a benchmark that evaluates cybersecurity capabilities. OpenAI used it in the evaluation that preceded last month's Hugging Face breach.) It's a stark reminder of what some critics have warned for years: that open-weight AI models could put highly capable AI into the hands of potential attackers, with no way to police how they use the technology once they download the weights. With open-weight models rapidly approaching the capabilities of the world's leading AI systems, the debate is moving from whether they can compete to how society manages risks once they are released. "The frontier of capability is not the frontier of risk, and so we do have to take into account the state of the mitigations as well to assess the risk properly," Henry Papadatos, executive director of SaferAI, told TechCrunch. While Z.ai could apply safety measures to its hosted API, those protections become unenforceable once someone runs the weights on their own hardware, where they can remove or modify any safeguards, fine-tune the models, or change system prompts. Frontier developers like OpenAI and Anthropic tend to rely on safeguards like classifiers, refusal training, and API-level controls to limit dangerous cyber and biological assistance. Those measures are far from foolproof: jailbreaks routinely bypass protections on deployed models. Far.ai, an AI safety nonprofit, found hundreds of universal jailbreaks -- defined as reusable keys that succeed on most harmful requests -- in frontier models like xAI's Grok 4.5 and Google DeepMind's Gemini 3.1 Pro. According to the report, jailbreaks succeed when attackers combine multiple manipulation techniques -- including roleplaying, authority impersonation, fake conversation history, and follow-up prompts -- to amplify weak points in a model's defenses. But the safeguards in place for closed models don't work at all on open-weight models, which are designed to run on any infrastructure with any set of safeguards -- or lack thereof. "The objective should clearly be that the good capabilities -- the safe ones -- are accessible to anyone, and then we try to remove the bad ones, even in an open source fashion," Papadatos said. One technique Papadatos noted could help is called "pre-training data filtering," which is when an AI company removes offensive cybersecurity information from their training data and then trains the model on the curated dataset. Some research suggests this can reduce hazardous biological knowledge without harming overall model performance. However, for cybersecurity, data filtering is much less practical. It's difficult to train a general model that excels at coding but isn't also a good hacker. Because coding has become AI's biggest moneymaker, developers face pressure to keep improving those capabilities even as they search for ways to limit misuse. Because of that, frontier developers have increasingly relied on other mitigations instead. One approach has been to selectively restrict the kinds of cybersecurity assistance models will provide. Anthropic's Opus 5, for example, can search for vulnerabilities in uncompiled source code, but not compiled software, per the model's system card. The reasoning is that this makes it harder to use Opus 5 for offensive purposes. Others include rigorous pre-deployment safety evaluations, publishing risk assessments, and withholding model weights if a system is perceived as too dangerous. In GLM-5.2's case, SaferAI says Z.ai didn't publish a safety framework, pre-deployment testing commitments, or risk assessment for the model. TechCrunch has asked Z.ai whether it conducted internal or third-party frontier safety evaluations before release, but did not receive a response. Chinese leaders have increasingly acknowledged the risks of advanced AI. At the World AI Conference last month, Chinese President Xi Jinping emphasized the importance of open-weight models, while also stressing the necessity of ensuring AI remains a tool under strict human control. Graham Webster, who studies Chinese AI policy at the Stanford Cyber Policy Center, told TechCrunch that China has robust regulations governing AI, but those rules have historically focused on politically sensitive content, misinformation, and social stability rather than catastrophic AI risks like offensive cyber capabilities and biological misuse. "U.S. AI thinkers are, in general, more concerned with this existential catastrophic [idea] than the Chinese community," Webster said, adding that many Chinese policy researchers believe that if there's truly going to be a novel frontier risk, American companies will likely encounter it first. "The Chinese system has confidence that they control the use of these technologies inside China," Webster continued. "Being online in China is something you do attributed to your real name, and companies can be held accountable, users can be held accountable." Webster mused that the same mechanism that model providers use for refusing to engage on certain political topics can potentially be tweaked to make sure models refuse to complete offensive cyber attacks or won't deliver adverse biological engineering outcomes. He added that because Chinese companies tend to coordinate with regulators behind the scenes, it can be tough to know what internal testing they're conducting before release. Advocates of open-weight AI argue that releasing the weights is important for cybersecurity because it allows companies to defend themselves against attacks -- Hugging Face relied on GLM-5.2 to defend itself against OpenAI's breach -- and because it allows them to better prepare for future threats if they know what's coming. "The same systems that helped stop an AI-powered cyberattack can now help defend against millions of cyberattacks every day, while helping us identify and fix vulnerabilities before attackers exploit them," Clem Delangue, CEO of Hugging Face, said this week in a social media post. Papadatos said that benefit is often overstated, and doesn't mean "we should open-source dangerous capabilities." "The main point in my mind is that we shouldn't just accept that dangerous capabilities are easily accessible by anyone anywhere," he said, stressing that he believes the industry should be striving for only making the "good capabilities" easily accessible. By default attackers adopt new tools faster than defenders do. For example, a ransomware group can change its methods in a week. A hospital cannot."

Open-weight AI caught the frontier. Safety didn't

Open-weight AI models are now only a few months behind the industry's best on raw capability. On safety they are far behind, and once the weights are downloaded, no lab can enforce a guardrail. A new evaluation of a leading Chinese open model shows exactly how wide the gap has grown. Open-weight AI models have nearly caught the frontier on capability. On safety, they have not. And once the weights are public, no lab can enforce a guardrail. A new evaluation of China's leading open model makes the gap concrete. GLM-5.2, the open-weight model from China's Z.ai, is only a few months behind OpenAI's GPT-5.5 and Anthropic's Claude Opus 4.7 on cyber and bio tasks, TechCrunch reported, citing the safety nonprofit SaferAI. But it refused none of the offensive-cyber or dual-use-biology tasks it was set. Claude Opus 4.7, by contrast, refused so consistently that SaferAI could not finish the cyber benchmark on it at all. "The frontier of capability is not the frontier of risk," SaferAI's Henry Papadatos said, so the safeguards matter as much as the model. Z.ai can guard its own hosted service. Those protections vanish the moment someone runs the weights on their own hardware, where any safeguard can be stripped out. It is the risk critics of open models have warned about for years. Closed models are not airtight either. The nonprofit Far.ai found hundreds of universal jailbreaks in xAI's Grok 4.5 and Google's Gemini 3.1 Pro. The difference is that a closed lab can patch a jailbroken model. Open weights cannot be recalled once they are out. Bolting safety back on So the industry is trying to add guardrails from the outside. In the same week, Mistral released Shieldstral, a small open-weight classifier that screens text and images against plain-language rules and, it says, matches models seven times its size. Cisco released Antares, open-weight models that hunt for vulnerabilities buried in code. Open-weight tools, built for open-weight risk. Openness cuts both ways, its defenders argue. Hugging Face used GLM-5.2 to help defend itself during OpenAI's breach. Its chief, Clem Delangue, says the systems that stop one attack can fend off millions more. Papadatos calls that overstated. The industry "shouldn't open-source dangerous capabilities," he says, and attackers move faster than defenders: a ransomware crew changes tactics in a week, a hospital cannot. A governance blind spot The rules do not fit the problem. The White House's new voluntary framework reviews certain closed frontier models for cyber risk. But it reportedly does not cover open-source models so far. Anthropic, once focused on IP theft, has shifted to naming safety as its main worry about open weights. Z.ai published no safety framework for GLM-5.2, and did not answer TechCrunch's questions. China is not ignoring the risk, but it aims elsewhere. Xi Jinping has backed open weights while stressing "human control" over AI. Its rules, Stanford's Graham Webster notes, target political content and social stability more than catastrophic cyber or bio misuse. The capability race is nearly settled: open is close behind and far cheaper. The safety race is not, and AI is already learning to attack as well as defend. The hard part is making sure only the defence is easy to download.