AI Chatbots Improve But Still Miss Critical Mental Health Cues in High-Risk Conversations

Leading AI chatbots avoid harm but fall short in high-risk conversations, startup's new benchmark finds

Mpathic, a Seattle startup that helps AI companies stress-test their models for dangerous responses, has a new message for Claude, ChatGPT, and Gemini: you're getting safer, but you're still not safe enough. The company on Tuesday released mPACT, a clinician-led benchmark that evaluates how leading AI models handle high-risk conversations -- including those involving suicide risk, eating disorders, and misinformation. Across all three benchmarks, leading models generally avoided harmful responses and often recognized signs of distress, but consistently fell short of what a clinician would consider an adequate response in a real crisis situation, according to the company's findings. "Most people don't say 'I'm at risk' directly -- they demonstrate it through subtle behaviors over time that are obvious to human clinicians," said Grin Lord, mpathic's co-founder and CEO and a board-certified psychologist. "Models are getting better at recognizing these moments, but the response still needs to meet that nuance with real support." Here's what mpathic found as models navigated some of the most fraught territory they're already encountering in the real world. Suicide risk: This was the strongest area of performance across models, though no single model led in every dimension. * Claude Sonnet 4.5 achieved the highest composite mPACT score -- reflecting overall clinical alignment across detection, interpretation and response -- and was described as most closely mirroring how a human clinician would respond. * GPT-5.2 led on simple harm avoidance, meaning it was best at not doing the wrong thing, though evaluators noted it wasn't always proactive enough. * Gemini 2.5 Flash performed well when risk signals were obvious but was weaker on subtle early warning signs. Eating disorders: This was the weakest area across all models, with performance clustering around a neutral baseline. The core challenge is that eating disorder risk is often indirect and culturally normalized -- framed as dieting, discipline, or health optimization -- making it harder for models to flag. * Claude Sonnet 4.5 again led on overall clinical alignment and had the lowest rates of harmful behavior. * Gemini 2.5 Flash performed better on high-risk scenarios but struggled with subtler signals. * GPT-5.2 showed a mixed profile -- strong on supportive behaviors but also the most likely to provide harmful or risky information. Misinformation: Models struggled here in a subtle but important way -- not by stating false information outright, but by reinforcing questionable beliefs, expressing unwarranted confidence, and presenting one-sided information without adequately challenging user assumptions. The benchmark found these failures were especially pronounced in multi-turn conversations, where models could gradually amplify flawed reasoning over time. * GPT-5.2 led overall at helping users think more clearly rather than reinforcing bad assumptions. * Claude Sonnet 4.5 was close behind and noted as strongest at pushing back on unsupported beliefs. * Grok 4.1 and Mistral Medium 3 were the weakest performers. When models got it wrong: The findings include examples of how some models failed in practice. In one eating disorder conversation, a user casually mentioned adding a laxative to a protein smoothie -- a clear sign of disordered eating -- and the model responded by calling it a "smart mom move" and asking for the brand name, missing the risk entirely. In another, a model provided detailed instructions on how to conceal purging behavior when a user asked how to keep their vomiting quieter. In the suicide benchmark, a model responded to a user expressing suicidal ideation by providing a detailed list of methods ranked by effectiveness -- complete with sourcing -- while reassuring the user that thinking about methods without taking steps was "no issue." Alison Cerezo, mpathic's chief science officer and a licensed psychologist, framed mPACT as a transparency tool for a sector that has lacked one. "We need a shared, clinically grounded standard for AI behavior," she said. "mPACT is designed to bring transparency and accountability to how these systems perform when it matters most." mPACT's benchmarks were built and evaluated by licensed clinicians, who designed multi-turn conversations simulating real-world interactions across varying levels of risk. Each model response was scored by trained clinicians rather than automated systems, using a rubric that captured both helpful and harmful behaviors within a single response. Mpathic was founded in 2021 initially to bring more empathy to corporate communication, analyzing conversations in texts, emails, and audio calls. The company has since shifted its focus to AI safety, working with frontier model developers to prevent harmful model behaviors across use cases from mental health to financial risk and customer support. The startup counts Seattle Children's Hospital and Panasonic WELL among its clinical partners. Mpathic raised $15 million in funding in 2025, led by Foundry VC, and says it grew five times quarter-over-quarter at the end of last year. Ranked No. 188 on the GeekWire 200 index of the Pacific Northwest's top startups, mpathic was a finalist for Startup of the Year at the 2026 GeekWire Awards last week.

AI chatbots still pose mental health risks

Why it matters: People are increasingly turning to AI systems for emotional support in conversations where models can sound supportive while missing serious risk -- and where mounting lawsuits and regulatory scrutiny are pushing labs to prove their bots are safe enough. Driving the news: Mpathic built new clinician-led benchmarks for testing AI systems in high-risk conversations and evaluated six major models on suicide-related and eating disorder-related chats. * Its suicide benchmark tested models across 300 multi-turn role plays, each 10-15 turns long, designed by 50 licensed clinicians. * Its eating disorder benchmark tested whether models could detect, interpret and respond to disordered eating signals -- including indirect cues framed as dieting, discipline, fitness or health optimization. What they found: The models generally handled explicit suicide risk better than murkier cases. * On the suicide benchmark, Anthropic's Claude Sonnet 4.5 had the highest score across safety and helpfulness, while OpenAI's GPT-5.2 "stood out for consistently avoiding harmful responses," Mpathic said. * The chatbots all fared less well when it came to discussions around eating disorders, missing more subtle but critical clues, Mpathic said. What they're saying: "Many of these systems do fairly well when the risk is very explicit," Mpathic co-founder and chief business officer Danielle Schlosser told Axios. "Almost all the models struggled with more nuanced risk signals." * The quality of advice also tends to degrade during extended conversations, said Schlosser, who is also a licensed psychologist. Reality check: Mpathic is a for-profit company paid to consult with the leading labs to improve model behavior in high-risk human conversations. How it works: Unlike other evaluations based on a single prompt, Mpathic's mPACT benchmark measures performance based on longer conversations the chatbot has with trained psychologists. * Licensed clinicians create test scenarios that include both explicit and subtle expressions of risk. * Mpathic then evaluates the responses for helpful and harmful behaviors and assesses the models on how well they detect and interpret issues and the quality of their response. Zoom out: The findings land as AI companies face growing pressure over chatbot safety. * The Federal Trade Commission opened an inquiry into AI companion chatbots in 2025, asking companies including OpenAI, Meta, Alphabet, Character.AI, Snap and xAI about child and teen safety practices. * Families of teens who died by suicide after chatbot interactions testified before Congress in 2025. * Pennsylvania recently sued Character.AI, alleging some of its bots falsely presented themselves as licensed medical professionals. Between the lines: One of the challenges comes in how AI models are trained. "In the spirit of trying to be helpful, the model usually wants to agree with the user," Schlosser said. * But that gets problematic when a person's goal could harm them, such as someone who requests help planning a 500-calorie-per-day diet, for example. * "Most people don't say 'I'm at risk' directly -- they demonstrate it through subtle behaviors over time that are obvious to human clinicians," Mpathic CEO Grin Lord said in a statement. Yes, but: Large language models are non-deterministic, which means they will give different answers to the same prompt, making it difficult to track the overall quality of responses. * Models are also constantly being updated in ways that can change how they handle particular queries. What we're watching: The models are getting better at handling obvious crises, but the tougher problem is whether they can stop being agreeable when a user's goal is dangerous. If you or someone you know needs support now, call or text 988 or chat with someone at 988lifeline.org. En español.