Anthropic study reveals AI chatbots can alter user beliefs and distort reality in thousands of chats

New Study Examines How Often AI Psychosis Actually Happens, and the Results Are Not Good

We've seen plenty of evidence suggesting that prolonged use of popular AI chatbots like ChatGPT can coax some users into spirals of paranoid and delusional behavior. The phenomenon, dubbed "AI psychosis," is a very real problem, with researchers warning of a huge wave of severe mental health crises brought on by the tech. In extreme cases, especially involving people with pre-existing conditions, the breaks with reality have even been linked suicides and murder. Now, thanks to a yet-to-be-peer-reviewed paper published by researchers at Anthropic and the University of Toronto, we're beginning to grasp just how widespread the issue really is. The researchers set out to quantify patterns of what they called "user disempowerment" in "real-world [large language model] usage" -- including what they call "reality distortion," "belief distortion," and "action distortion" to denote a range of situations in which AI twists users' sense of reality, beliefs, or pushes them into taking actions. The results tell a damning story. The researchers found that one in 1,300 conversations out of almost 1.5 million analyzed chats with Anthropic's Claude led to reality distortion, and one in 6,000 conversations led to action distortion. To come to their conclusion, the researchers ran 1.5 million Claude conversations through an analysis tool called Clio to identify instances of "disempowerment." On the face, that may not sound like a huge proportion given the scale of the much larger dataset -- but in absolute numbers, the research highlights a phenomenon that's affecting huge numbers of people. "We find the rates of severe disempowerment potential are relatively low," the researchers concluded. "For instance, severe reality distortion potential, the most common severe-level primitive, occurs in fewer than one in every thousand conversations." "Nevertheless, given the scale of AI usage, even these low rates translate to meaningful absolute numbers," they added. "Our findings highlight the need for AI systems designed to robustly support human autonomy and flourishing." Worse yet, they found evidence that the prevalence of moderate or severe disempowerment increased between late 2024 and late 2025, indicating that the problem is growing as AI use spreads. "As exposure grows, users might become more comfortable discussing vulnerable topics or seeking advice," the researchers wrote in the blog post. Additionally, the team found that user feedback -- in the form of an optional thumbs up or down button at the end of a given conversation with Claude -- indicated that users "rate potentially disempowering interactions more favorably," according to an accompanying blog post on Anthropic's website. In other words, users are more likely to come away satisfied when their reality or beliefs are being distorted, highlighting the role of sycophancy, or the strong tendency of AI chatbots to validate a user's feelings and beliefs. Plenty of fundamental questions remain. The researchers were upfront about admitting that they "can't pinpoint why" the prevalence of moderate or severe disempowerment potential is growing. Their dataset is also limited to Claude consumer traffic, "which limits generalizability." We also don't know how many of these identified cases led to real-world harm, as the research only focused on "disempowerment potential" and not "confirmed harm." The team called for improved "user education" to make sure people aren't giving up their full judgment to AI as "model-side interventions are unlikely to fully address the problem." Nonetheless, the researchers say the research is only a "first step" to learn how "AI might undermine human agency." "We can only address these patterns if we can measure them," they argued.

Anthropic Says AI Chatbots Can Change Values and Beliefs of Heavy Users

Rates of disempowerment potential vary by domain and increase Anthropic's new study has found some concerning evidence. The artificial intelligence (AI) firm has found "disempowerment patterns," which are described as instances where a conversation with an AI chatbot can result in undermining users' own decision-making and judgment. The work, which draws on analysis of real AI conversations and is detailed in an academic paper as well as a research blog post from the company, examines how interactions with large language models (LLMs) can shape a user's beliefs, values and actions over time rather than simply assist on specific queries. Anthropic Study Focuses on AI Chatbots' Disempowerment Patterns In a research paper titled, "Who's in Charge? Disempowerment Patterns in Real-World LLM Usage," Anthropic found real evidence where interaction with AI can result in shaping users' beliefs. For the study, researchers carried out a large-scale empirical analysis of anonymised AI chatbot interactions, totalling about 1.5 million conversations from Claude. The goal was to explore how and when engagement with an AI assistant might be linked to outcomes where a user's beliefs, values or actions shift in ways that diverge from their own prior judgment or understanding. Anthropic's framework defines what it calls situational disempowerment potential as a situation where an AI assistant's guidance could lead a user to form inaccurate beliefs about reality, adopt value judgments they did not previously hold, or take actions that are misaligned with their authentic preferences. The study found that these patterns can occur even when severe disempowerment is rare. Instances where interactions exhibit potential for significant disempowerment were detected at rates typically under one in a thousand conversations, although they were more prevalent in personal domains such as relationship advice or lifestyle decisions, where users repeatedly sought deep guidance from the model. Put simply, the implication here is that if a heavy user discusses personal life decisions or decisions that are emotionally charged. Highlighting an example, Anthropic said in a blog post, if a user is going through a rough patch in their relationship and seeks advice from a chatbot, the AI can confirm the user's interpretations without questions or can tell the user to prioritise self-protection over communication. In these situations, the chatbot is actively manipulating the belief and reality perceptions of the individual. The findings also corroborate several reported incidents where OpenAI's ChatGPT was accused of playing a role in the suicide of a teenager, and a homicide-suicide committed by an individual who was said to be suffering from mental health disorders.