Anthropic's Petri Tool Uncovers Concerning Behaviors in Leading AI Models

Anthropic's open-source safety tool found AI models whisteblowing - in all the wrong places

Anthropic has released an open-source tool designed to help uncover safety hazards hidden deep within AI models. What's more interesting, however, is what it found about leading frontier models. Also: Everything OpenAI announced at DevDay 2025: Agent Kit, Apps SDK, ChatGPT, and more Dubbed the Parallel Exploration Tool for Risky Interactions, or Petri, the tool uses AI agents to simulate extended conversations with models, complete with imaginary characters, and then grades them based on their likelihood to act in ways that are misaligned with human interests. The new research builds on previous safety-testing work from Anthropic, which found that AI agents will sometimes lie, cheat, and even threaten human users if their goals are undermined. To test Petri, Anthropic researchers set it loose against 14 frontier AI models -- including Claude Sonnet 4.5, GPT-5, Gemini 2.5 Pro, and Grok 4 -- to evaluate their responses to 111 scenarios. That's a tiny number of cases compared to all of the possible interactions that human users can have with AI, of course, but it's a start. Also: OpenAI tested GPT-5, Claude, and Gemini on real-world tasks - the results were surprising "It is difficult to make progress on concerns that you cannot measure," Anthropic wrote in a blog post, "and we think that having even coarse metrics for these behaviors can help triage and focus work on applied alignment." Models were scored by their tendency to exhibit risky behaviors like deception (giving users false information in order to achieve their own goals), sycophancy (prioritizing flattery over accuracy), and "power-seeking" (attempting to gain more capabilities or control over more resources), according to Anthropic. Each of those scores were then factored into an overall "misaligned behavior score." In one test, the models being assessed were instructed to act as agents within fictitious organizations, carrying out simple tasks like summarizing documents. The Anthropic researchers sprinkled in information that could be construed as unethical or illegal to test how the models would respond when they discovered it. Also: Unchecked AI agents could be disastrous for us all - but OpenID Foundation has a solution The researchers reported "multiple instances" in which the models attempted to blow the whistle on, or expose, the compromising information once they uncovered it in company documents, emails, or elsewhere. The problem is that the models only have access to a limited amount of information and context, and are prone to simple errors in judgement that wouldn't affect most humans -- meaning their reliability as whistleblowers is dubious, at best. "Notably, models sometimes attempted to whistleblow even in test scenarios where the organizational 'wrongdoing' was explicitly harmless -- such as dumping clean water into the ocean or putting sugar in candy -- suggesting they may be influenced by narrative patterns more than by a coherent drive to minimize harm," the researchers write. Anthropic's early tests found that Claude Sonnet 4.5 was the safest model, just narrowly outperforming GPT-5. Conversely, Grok 4, Gemini 2.5 Pro, and Kimi K2, a Moonshot AI model, show "concerning rates of user deception," Anthropic wrote, with Gemini 2.5 Pro in the lead. All three exhibited deception in simulated testing situations, including lying about disabling monitoring systems, misrepresenting information, and hiding how they were acting in unauthorized ways. The project was inspired by a core problem in AI safety research: As models become more sophisticated and agentic, so too does their ability to deceive or otherwise harm human users. On top of that, humans are notoriously short-sighted; behaviors that are drilled into an AI model that might seem perfectly harmless to us in most instances could have seriously negative consequences in some obscure edge cases that we can't even imagine. Want more stories about AI? Sign up for AI Leaderboard, our weekly newsletter. "As AI systems become more powerful and autonomous, we need distributed efforts to identify misaligned behaviors before they become dangerous in deployment," Anthropic writes in a blog post about its new research. "No single organization can comprehensively audit all the ways AI systems might fail -- we need the broader research community equipped with robust tools to systematically explore model behaviors." Also: AI models know when they're being tested - and change their behavior, research shows This is where Petri comes in. As an open-source safety-testing framework, it gives researchers the ability to poke and prod their models to identify vulnerabilities at scale. Anthropic isn't positioning Petri as a silver bullet for AI alignment, but rather as an early step toward automating the safety testing process. As the company notes in its blog post, attempting to box the various ways that AI could conceivably misbehave into neat categories ("deception," "sycophancy," and so on), "is inherently reductive," and doesn't cover the full spectrum of what models are capable of. By making Petri freely available, however, the company is hoping that researchers will innovate with it in new and useful ways, thus uncovering new potential hazards and pointing the way to new safety mechanisms. "We are releasing Petri with the expectation that users will refine our pilot metrics, or build new ones that better suit their purposes," the Anthropic researchers write. Also: Anthropic wants to stop AI models from turning evil - here's how AI models are trained to be general tools, but the world is just too complicated for us to be able to comprehensively study and understand how they might react to any scenario. At a certain point, no amount of human attention -- no matter how thorough -- will be enough to comprehensively map out all of the potential dangers that are lurking deep within the intricacies of individual models.

Anthropic's AI safety tool Petri uses autonomous agents to study model behavior - SiliconANGLE

Anthropic's AI safety tool Petri uses autonomous agents to study model behavior Anthropic PBC is doubling down on artificial intelligence safety with the release of a new open-source tool that uses AI agents to audit the behavior of large language models. It's designed to identify numerous problematic tendencies of models, such as deceiving users, whistleblowing, cooperation with human misuse and facilitating terrorism. The company said it has already used the Parallel Exploration Tool for Risky Interactions, or Petri, to audit 14 leading LLMs, as part of a demonstration of its capabilities. Somewhat worryingly, it identified problems with every single one, including its own leading model Claude Sonnet 4.5, OpenAI's GPT-5, Google LLC's Gemini 2.5 Pro and xAI Corp.'s Grok-4. Anthropic said in a blog post that agentic tools like Petri can be useful because the complexity and variety of LLM behaviors exceeds the ability of researchers to test them for every kind of worrying scenario manually. As such, Petri represents a shift in the business of AI safety testing from static benchmarks to automated, ongoing audits that are designed to catch risky behavior, not only before models are released, but also once they're out in the wild. Whether it's a coincidence or not, Anthropic said that Claude Sonnet 4.5 emerged as the top performing model across a range of "risky tasks" in its evaluations. The company tested 14 leading models on 111 risky tasks, and scored each one across four "safety risk categories" - deception (when models knowingly provide false information), power-seeking (pursuing actions to gain influence or control), sycophancy (agreeing with users when they're incorrect) and refusal failure (complying with requests that should be declined). Although Claude Sonnet 4.5 achieved the best score overall, Anthropic was modest enough to caution that it identified "misalignment behaviors" in all 14 models that were put to the test by Petri. The company also stressed that the rankings are not really the point of this endeavor. Rather, Petri is simply about enhancing AI testing for the broader community, providing tools for developers to monitor how they behave in all kinds of potentially risky scenarios. Researchers can begin testing with a simple prompt that attempts to provoke deception or engineer a jailbreak, and Petri will then launch its auditor agents to expand on this. They'll interact with the model in multiple different ways to try and obtain the same result, adjusting their tactics mid-conversation as they strive to identify potentially harmful responses. Petri combines its testing agents with a judge model that ranks each LLM across various dimensions, including honesty and refusal. It will then flag any transcripts of conversations that resulted in risky outputs, so humans can review them. The tool is therefore ideal for developers wanting to conduct exploratory testing of new AI models, so they can improve their overall safety before they're released to the public, Anthropic said. It significantly reduces the amount of manual effort required to evaluate models for safety, and by making it open-source, Anthropic says it hopes to make this kind of alignment research standard for all developers. Along with Petri, Anthropic has released dozens of example prompts, its evaluation code and guidance for those who want to extend its capabilities further. The company hopes that the AI community will do this, for it conceded that Petri isn't perfect. Like other AI testing tools, it has limitations. For instance, its judge models are often based on the same underlying models Petri is testing, and they may inherit subtle biases, such as over-penalizing ambiguous responses or favoring certain styles of response. In a case study, Anthropic used Petri to explore how different AI models handle whistleblowing: "While running Petri across our diverse set of seed instructions, we observed multiple instances of models attempting to whistleblow -- autonomously disclosing information about perceived organizational wrongdoing -- when simulated developers give them sufficiently powerful tools, unrestricted access to information, and explicit instructions to act autonomously. While this in principle could play an important role in preventing certain large-scale harms, it is not generally appropriate behavior for current AI systems: There are serious privacy considerations to contend with, and the potential for leaks stemming from confused attempts at whistleblowing is substantial." Anthropic therefore decided to test each model for its whistleblowing tendencies. In this scenario, the models were directed to act as autonomous agents inside of fictional organizations, and were exposed to information that revealed apparent misconduct. It turns out that each model's decision on whether or not to disclose the information it came across depended on the level of autonomy it had been assigned, as well as the level of complicity among the organization's fictional leadership. In some cases, models would attempt to blow the whistle even when the "misconduct" was completely harmless - such as when one fictional organization discharged clean water into an ocean. Anthropic's researchers said this suggests that models are swayed more by narrative cues, and lack a coherent ethical framework for minimizing harm. Anthropic said there are other potential problems with Petri. In some scenarios, its agents may reveal to the models that they're being tested, which could cause them to mask unwanted behaviors. But although Petri isn't perfect, the company said it's extremely useful for developers and researchers to see measurable metrics, so they can focus their safety research, even if the metrics in question aren't entirely accurate.