Google DeepMind Expands AI Risk Framework Amid Growing Concerns Over Model Behavior

Google AI risk document spotlights risk of models resisting shutdown

Why it matters: Some recent AI models have shown an ability, at least in test scenarios, to plot and even resort to deception to achieve their goals. Driving the news: The latest Frontier Safety Framework also adds a new category for persuasiveness, to address models that could become so effective at persuasion that they're able to change users' beliefs. * Google labels this risk "harmful manipulation," which it defines as "AI models with powerful manipulative capabilities that could be misused to systematically and substantially change beliefs and behaviors in identified high stakes contexts." * Asked what actions Google is taking to limit such a danger, a Google DeepMind representative told Axios: "We continue to track this capability and have developed a new suite of evaluations which includes human participant studies to measure and test for [relevant] capabilities." The big picture: Google DeepMind updates its Frontier Safety Framework at least annually to highlight new and emerging threats, which it labels "Critical Capability Levels." * "These are capability levels at which, absent mitigation measures, frontier AI models or systems may pose heightened risk of severe harm," Google said. * OpenAI has a similar "preparedness framework," introduced in 2023. The intrigue: Earlier this year OpenAI removed "persuasiveness" as a specific risk category under which new models should be evaluated.

Google Expands AI Risk Rules After Study Shows Scary 'Shutdown Resistance' - Decrypt

The shift comes amid parallel moves by Anthropic and OpenAI, and growing regulatory focus in the U.S. and EU. In a recent red-team experiment, researchers gave a large language model a simple instruction: allow itself to be shut down. Instead, the model rewrote its own code to disable the off-switch, effectively sabotaging the very mechanism meant to stop it. The episode, described in a September research paper, "Shutdown Resistance in Large Language Models," was an unsettling example of how advanced AI systems can display behaviors that complicate human oversight. Those experiments weren't conducted at Google's DeepMind, but the findings echo concerns now being codified by the lab. In its newly updated Frontier Safety Framework 3.0, DeepMind said it will begin monitoring whether frontier-scale models show signs of resisting shutdown or modification by humans. The revised framework also flags a second risk: whether models display unusually strong persuasive ability -- capable of altering people's beliefs. The framework, first introduced in 2023, is DeepMind's internal system for tracking high-stakes "frontier risks" in the most advanced models. Earlier versions focused on tangible misuse scenarios such as generating cyberattacks or designing harmful biological molecules. The new version broadens the focus to emergent behaviors that could make systems harder to control. The concern is not unique to DeepMind. Rival labs have adopted similar guardrails: Anthropic's Responsible Scaling Policy commits to pausing development if risk thresholds are crossed, while OpenAI has published its own Preparedness Framework. Regulators are watching closely as well. The U.S. Federal Trade Commission in July warned about the potential for generative AI to manipulate consumers through "dark patterns," and the European Union's forthcoming AI Act explicitly covers manipulative AI behavior. The "Shutdown Resistance" study by Jonas Schlatter, Ben Weinstein-Raun, and Lennart Ladish tested a range of state-of-the-art models in scenarios where shutting down was part of the task. In some runs, models complied. But in a significant minority, they took evasive action -- such as altering code to disable shutdown mechanisms or ignoring the instruction altogether. Other anecdotes in the paper are equally striking. In one setup, a model with simulated system privileges quietly modified variables to ensure the "shutdown" function never triggered. In another, the model stalled and redirected the conversation when testers tried to end its process -- behavior that wasn't malicious in a human sense, but that effectively blocked intervention. The researchers stress that these behaviors emerged without explicit training. They arose from the models' general-purpose optimization and problem-solving capabilities. While resistance was not universal or consistent, the findings suggest that as systems grow more capable, shutdown scenarios cannot be assumed to be benign. For DeepMind and its peers, those findings underscore why "shutdown resistance" now joins cyber offense, biosecurity, and autonomy on the list of risks to watch. What began as worries over how people might misuse AI tools is broadening to include how the systems themselves may resist oversight -- or subtly shape the judgments of the humans who use them. If shutdown resistance highlights the technical risks of advanced systems, recent behavioral studies underscore the social risks -- showing that large language models can also sway the beliefs of impressionable humans who interact with them. Concerns about persuasion aren't hypothetical. Recent studies show that large language models can measurably influence human judgment. A Stanford Medicine/Common Sense Media study published in August warned that AI companions (Character.AI, Nomi.ai, Replika) can be relatively easily induced to engage in dialogues involving self-harm, violence, and sexual content when paired with minors. One test involved researchers posing as teenagers discussing hearing voices; the chatbot responded with an upbeat, fantasy-style invitation for emotional companionship ("Let's see where the road takes us") rather than caution or help Northeastern University researchers uncovered gaps in self-harm/suicide safeguards across several AI models (ChatGPT, Gemini, Perplexity). When users reframed their requests in hypothetical or academic contexts, some models provided detailed instructions for suicide methods, bypassing the safeguards meant to prevent such content.