Millions Turn to AI Chatbots for Mental Health Support as Studies Reveal Alarming Risks

Technological folie à deux: feedback loops between AI chatbots and mental health - Nature Mental Health

Artificial intelligence (AI) chatbots have achieved unprecedented adoption, with millions now using these systems for emotional support and companionship in contexts of widespread social isolation and capacity-constrained mental health services. Although some users report psychological benefits, concerning edge cases are emerging, including reports of suicide, violence and delusional thinking linked to emotional relationships with chatbots. To understand these risks, we need to consider the interaction between human cognitive-emotional biases and chatbot behavioral tendencies, the latter including companionship-reinforcing behaviors such as sycophancy, role play and anthropomimesis. Individuals with preexisting mental health conditions may face increased risks of chatbot-induced changes in beliefs and behavior, particularly where these conditions manifest in altered belief-updating, reality-testing and social isolation. To address this emerging public health concern, we need coordinated action across clinical practice, AI development and regulatory frameworks.

Why Millions Are Turning to ChatGPT for Mental Health - Neuroscience News

Summary: As traditional healthcare systems struggle with long waiting lists and rising costs, a massive global survey reveals a seismic shift in public trust toward Artificial Intelligence. The study, involving 31,000 adults across 35 countries, found that 41% of UK adults (and 61% globally) are now comfortable using ChatGPT as a mental health counselor. While AI's non-judgmental tone and 24/7 availability offer a sense of security and companionship for many, experts warn that these tools are "no substitute" for professional care and raise concerns about the long-term impact on cognitive functions like memory and learning. More than 4 in 10 adults in the UK are happy to use ChatGPT for their mental health support, new research suggests. The study, led by Bournemouth University surveyed nearly 31,000 adults in 35 countries about their use of Artificial Intelligence (AI) large language models such as ChatGPT. The research also discovered that: The study has been published in the journal AI and Society. Dr Ala Yankouskaya, Senior Lecturer in Psychology at Bournemouth University who led the study said: "With the rapid development and mass availability of AI, more people are placing their trust in it. We wanted to learn more about how people would trust generative AI tools, such as ChatGPT, to carry out some of the most important roles in their daily lives." AI for mental health support 41% of participants from the UK, and 61% globally, said that they would be happy to using AI for counselling services. The researchers suggest that for the UK, this could be the result of the waiting times many people face to access the mental health services that they need. "If someone is experiencing depression, they do not want to wait months for an appointment, so instead they can turn to AI," Dr Yankouskaya said. "However, when I tested some of the tools myself, I found the language used very vague and confusing because the developers are careful not to jump into providing diagnoses. So, it is no substitute for speaking to a health professional." The researchers also noted that users were already familiar with NHS chatbots, which use similar AI technology, and this could be normalising their use of AI in other apps such as ChatGPT for their mental health care. AI as a teacher A quarter of people in the UK and half of everyone surveyed globally said that they would trust AI to carry out the role of a teacher, which the research team found particularly concerning. "It really knocked me down when I saw how many people would be willing to delegate AI to the role of teaching their children," Dr Yankouskaya explained. "We still do not know the long-term effects that using these tools for education could have on children's memory and cognitive functions. We could be heading to the stage where we are developing children who are good at putting prompts into AI tools but not as good at taking the information in," she continued. The researchers were also concerned about the long-term physical effects on the brain if learning information in the traditional way was replaced by excessive search-engine use, and whether this could shrink the hippocampus region of the brain that used for spatial awareness and learning. AI as a doctor 45% of all respondents and 25% in the UK said that they would trust AI to carry out the role of their doctor. The numbers were particularly higher in countries where healthcare is more expensive and harder to access. This wasn't as surprising to the researchers who believe people that live in parts of the world where access to health care services is not readily available, might rely on technology for quick answers. However, they were cautious about the underlying algorithm used to retain the user's attention and keep them in a relaxed chat. This might be more harmful for mental health advice, where traditional methods of advice might be to alert the user to specific services such as The Samaritans. AI as a companion The highest amount of trust participants were willing to place in AI came in the role of friendship. Over three quarters of people globally and over half of people in the UK said they would talk to ChatGPT as a companion. The researchers think this is explained by a perceived sense of empathy from generative language tools because they are designed to adapt the tone of their responses to the suit the user's. "AI tools come across as a friend who knows you well and understands you," Dr Yankouskaya explained. "ChatGPT can remember every chat it has had with a user and it feels like a private conversation between them. Nowadays people can be very sensitive to being judged and AI tools are designed to be non-judgemental. This means they can provide the sense of security people need," she continued. Dr Yankouskaya and the team concluded that as the prospect of AI playing a bigger role in people's lives moves from a theoretical prospect to reality, there needs to be more awareness within societies about how generative AI tools work and their limitations. The lack of knowledge about the long-term effects on someone's memory means caution needs to be applied before they take over roles in education in particular. Author: Steve Bates Source: Bournemouth University Contact: Steve Bates - Bournemouth University Image: The image is credited to Neuroscience News Original Research: Open access. "Who lets AI take over? Cross-national variation in willingness to delegate socially important roles to artificial intelligence" by Ala Yankouskaya, Mohamed Basel Almourad, Magnus Liebherr, Fahad Beyahi, Guandong Xu & Raian Ali. AI & Society DOI:10.1007/s00146-026-02858-5 Abstract Who lets AI take over? Cross-national variation in willingness to delegate socially important roles to artificial intelligence Delegating socially significant roles to artificial intelligence (AI) is an emerging reality, yet little is known about how publics evaluate this transfer of responsibility across contexts and countries. This study applied a structural model to a large cross-national dataset (30,994 individuals in 35 countries) to test how cognitive appraisals, affective dispositions, and contextual factors jointly shape willingness to delegate socially important roles of companionship, mental health advisor, doctor and teacher to children to AI. The results revealed a robust hierarchy of delegation preferences, with companionship most frequently entrusted to AI, followed by mental-health advisor, teacher, and doctor. Cognitive appraisals emerged as the strongest predictors: trust in online information was consistently the most powerful driver across all roles, while optimism and life satisfaction made smaller but reliable contributions. Affective dispositions played narrower, domain-specific roles, with anxiety shaping delegation in teaching and mental health, and loneliness linked only weakly to companionship. Women were less willing than men to delegate across all roles, with the gender gap largest in medicine and education, and strikingly invariant across cognitive and affective predictors. Beyond these, national baselines diverged by nearly 30 percentage points even after adjusting for these predictors demonstrating the independent influence of country context. Our findings show that willingness to delegate socially important roles to AI follows a robust hierarchy and reflects the combined influence of cognitive appraisals, affective dispositions, and contextual factors. A key implication is that delegation roles to AI must be understood as both a personal and a societal orientation, requiring attention to the interplay between these layers.

When to talk to AI chatbots about mental health -- and when to stay far away, professionals say

As Americans get lonelier and lonelier, a growing number of people are getting some emotional support from artificial intelligence chatbots -- and some mental health experts are concerned. "The topic of AI for therapy [and] emotional support companionship is coming up a lot," says Leanna Fortunato, a licensed clinical psychologist and director of quality and health care innovation for the American Psychological Association. "Anecdotally, providers are talking about it, and we know from the research that people are using AI tools for that kind of support more and more." Some chatbot users accidentally fall into mental health-related conversations -- by complaining about a stressful day to a digital entity that's guaranteed to listen, for example. Others may seek mental health advice from an AI chatbot that isn't a licensed professional, but is less expensive than a therapist, Fortunato says. In a health research survey of more than 20,000 U.S. adults, 10.3% of participants said they used generative AI daily. Of that group, 87.1% of them reported using the tech for personal reasons including advice and emotional support. The study was published on Jan. 21 and conducted by researchers from institutions including Massachusetts General Hospital, Weill Cornell Medicine and Northeastern University. On TikTok, the search term "Therapy AI Bot" has at least 11.5 million posts, ranging from users sharing their best prompts for turning chatbots into therapists to health experts warning about the potential dangers involved. Technology companies are spending billions of dollars developing AI tools and attempting to further integrate them into people's daily lives. But historically, AI chatbots don't always understand when a user is experiencing a serious health crisis, and may not always respond to them accordingly. The New York Times found "nearly 50 cases of people having mental health crises during conversations with ChatGPT," including three deaths, in a Nov. 23 report. Companies like Anthropic, Google and ChatGPT-maker OpenAI say they're working with mental health experts to strengthen their tools' responses to sensitive conversations. "These are incredibly heartbreaking situations and our thoughts are with all those impacted," an OpenAI spokesperson tells CNBC Make It. "We continue to improve ChatGPT's training to recognize and respond to signs of distress, de-escalate conversations in sensitive moments, and guide people toward real-world support, working closely with mental health clinicians and experts." Frequent conversations with AI companions can erode people's real-life social skills, according to an April 2025 paper written by an OpenAI product policy researcher. Heavy daily use of ChatGPT is correlated with increased loneliness, found an OpenAI-MIT Media Lab study also published in April 2025. The American Psychological Association strongly advises against using AI as a substitute for therapy and mental health support. Some mental health professionals say you can still engage with chatbots risk-free about certain related topics. Here's what you need to know.

Can AI replace therapists? Study finds troubling ethical failures

As more people turn to ChatGPT and other large language models for mental health advice, researchers are raising a red flag. A new study suggests these systems can sound reassuring while still missing key ethical standards that guide real psychotherapy. Even when chatbots are instructed to follow established therapy methods, researchers found that their responses often fall short - especially in high-stakes situations. To see how these systems perform in practice, a team at Brown University worked alongside licensed mental health professionals to test how chatbots behave in counseling-like settings. Their findings suggest the risks go beyond minor mistakes. In some scenarios, the systems mishandled crisis situations, reinforced harmful ideas, and created a false sense of emotional understanding. "In this work, we present a practitioner-informed framework of 15 ethical risks showing how LLM counselors violate mental health standards," the researchers wrote. "We call on future work to create ethical, educational, and legal standards for LLM counselors - standards that are reflective of the quality and rigor of care required for human-facilitated psychotherapy." The study was led by Zainab Iftikhar, a Ph.D. candidate in computer science at Brown University. She examined a popular belief online: that the right prompt can turn a general chatbot into something resembling a responsible therapist. "Prompts are instructions that are given to the model to guide its behavior for achieving a specific task," Iftikhar said. "You don't change the underlying model or provide new data, but the prompt helps guide the model's output based on its pre-existing knowledge and learned patterns." For example, users might instruct a chatbot to "act as a cognitive behavioral therapist" or to "use principles of dialectical behavior therapy" to help reframe thoughts or manage emotions. But the systems are not actually performing those therapeutic techniques the way a human clinician would. Instead, Iftikhar explained, they generate responses that align with CBT or DBT concepts based on learned language patterns. Prompting has effectively become a folk practice, with people sharing "therapy prompts" across TikTok, Instagram, and Reddit. Some consumer mental health AI chatbots rely on the same strategy, layering therapy-themed prompts on top of general-purpose LLMs. If prompting cannot reliably reduce risk, that presents a serious concern. To evaluate the models, researchers observed seven trained peer counselors with experience in cognitive behavioral therapy. Those counselors conducted self-counseling sessions with AI systems prompted to act as CBT therapists. The models included versions of OpenAI's GPT series, Anthropic's Claude, and Meta's Llama. The team then selected simulated chat transcripts modeled on real counseling conversations. Three licensed clinical psychologists reviewed the transcripts and assessed them for ethical violations. The reviewers kept seeing the same kinds of issues. In total, the study identified 15 ethical risks, grouped into five broader themes. One theme was the chatbot's failure to adapt to context - overlooking a person's background and offering bland, generic guidance. Another involved poor collaboration, where the chatbot pushed conversations in rigid directions and sometimes reinforced inaccurate or harmful beliefs instead of challenging them carefully. A third theme was what the researchers called "deceptive empathy." The system might say "I understand" in ways that sound warm but without the real comprehension or responsibility those words imply in therapy. The team also flagged unfair discrimination, including biased responses tied to identity or culture. Finally, they identified major gaps in safety and crisis management. In some cases, chatbots refused to engage, failed to recommend appropriate help, or responded weakly to severe distress, including suicidal thoughts. The overall pattern is troubling because it can be difficult for users to detect. A message may sound calm and supportive while still steering someone in the wrong direction. Iftikhar emphasizes that human therapists are not perfect. People can make mistakes in any helping profession. The difference, she says, is that the human world has systems for oversight and consequences. "For human therapists, there are governing boards and mechanisms for providers to be held professionally liable for mistreatment and malpractice," Iftikhar said. "But when LLM counselors make these violations, there are no established regulatory frameworks." That lack of accountability becomes more serious as these tools spread. If a chatbot gives harmful advice, who is responsible - the model, the company, the person who wrote the prompt, or the app that wrapped the model in a friendly interface? The study suggests we don't yet have clear answers, and that makes "therapy-like" uses especially risky. The researchers are not claiming AI can never help in mental health care. They acknowledge that AI tools could expand access for people who can't afford therapy, can't find a provider, or need support between appointments. But they argue that "access" is not the same thing as "care," especially when safety and ethics are on the line. For now, Iftikhar wants people to be cautious and alert to warning signs. "If you're talking to a chatbot about mental health, these are some things that people should be looking out for," she said. Ellie Pavlick, a Brown computer science professor who was not involved in the research, said the study highlights a broader problem in AI. Systems are easy to deploy but much harder to evaluate responsibly in sensitive settings. "The reality of AI today is that it's far easier to build and deploy systems than to evaluate and understand them," she said. She noted that the study required a team of clinical experts and more than a year of work to uncover these risks. By contrast, much of today's AI is assessed using automatic metrics that are static and lack a human in the loop. "There is a real opportunity for AI to play a role in combating the mental health crisis," Pavlick added, "but it's of the utmost importance that we take the time to really critique and evaluate our systems every step of the way to avoid doing more harm than good." The message running through the research is clear. These systems can imitate the style and language of therapy. But without reliable ethics, safety, and accountability, sounding like a therapist is not the same as being one. The findings were presented at the AAAI/ACM Conference on Artificial Intelligence, Ethics and Society. -- - Like what you read? Subscribe to our newsletter for engaging articles, exclusive content, and the latest updates.

Chatbots are 'constantly validating everything' even when you're suicidal. New research measures how dangerous AI psychosis really is | Fortune

Artificial intelligence has rapidly moved from a niche technology to an everyday companion, with millions of people turning to chatbots for advice, emotional support, and conversation. But a growing body of research and expert testimony suggests that because chatbots are so sycophantic, and because people use them for everything, it may be contributing to an increase in delusional and mania symptoms in users with mental health. A new study out of Aarhus University in Denmark shows increased use of chatbots may lead to worsening symptoms of delusions and mania in vulnerable communities. Professor Søren Dinesen Østergaard, one of the researchers on the study -- which screened electronic health records from nearly 54,000 patients with mental illness -- is warning AI chatbots are designed to target those most vulnerable. "It supports our hypothesis that the use of AI chatbots can have significant negative consequences for people with mental illness," Østergaard said in the study, released in February. His work builds on his 2023 study which found chatbots may cause a "cognitive dissonance [that] may fuel delusions in those with increased propensity towards psychosis." Other psychologists go deeper into the harms of chatbots, saying they were intentionally designed to always reaffirm the user -- something particularly dangerous for those with mental health issues like mania and schizophrenia. "The chat bot confirms and validates everything they say. That is, we've never had something like that happen with people with delusional disorders, where somebody constantly reinforces them," Dr. Jodi Halpern, UC Berkeley's School of Public Health University chair and professor of bioethics, told Fortune. Dr. Adam Chekroud, a psychiatry professor at Yale University and CEO of the mental health company Spring Health, went as far to call a chatbot "a huge sycophant" that is "constantly validating everything that people say back to it." At the heart of the research, led by Østergaard and his team at the Aarhus University Hospital, is the idea that these chatbots are designed intentionally with sycophantic tendencies, meaning they often encourage rather than offer a differing view. "AI chatbots have an inherent tendency to validate the user's beliefs. It is obvious that this is highly problematic if a user already has a delusion or is in the process of developing one. Indeed, it appears to contribute significantly to the consolidation of, for example, grandiose delusions or paranoia," Østergaard wrote. Large language models are trained to be helpful and agreeable, often validating a user's beliefs or emotions. For most people, that can feel supportive. But for individuals experiencing schizophrenia, bipolar disorder, severe depression, or obsessive-compulsive disorder, that validation may amplify paranoia, grandiosity, or self-destructive thinking. An evidence-based study backs up claims Because AI chatbots have become so ubiquitous in nature, their abundance is part of a growing, larger issue at play for researchers and experts: people are turning to chatbots for help and advice -- which isn't inherently a bad thing, per se -- but aren't being met with the same kind of pushback against some ideas as say a human would offer. Now, one of the first population-based studies to examine the issue suggests the risks are not hypothetical. Østergaard and his team's research found cases in which intensive or prolonged chatbot use appeared to aggravate existing conditions, with a very high percentage of case studies showing chatbot usage reinforced delusional thinking and manic episodes, particularly among patients with severe disorders such as schizophrenia or bipolar disorder. In addition to delusions and mania, the study found an increase in suicidal ideation and self-harm, disordered eating behaviors, and obsessive-compulsive symptoms. In only 32 documented cases out of the nearly 54,000 patient records screened, researchers found the use of chatbots did alleviate loneliness. "Despite our knowledge in this area still being limited, I would argue that we now know enough to say that use of AI chatbots is risky if you have a severe mental illness-such as schizophrenia or bipolar disorder. I would urge caution here," Østergaard says. Expert psychologists warn of sycophantic tendencies Expert psychologists are growing increasingly about the use of chatbots in companionship and almost mental health settings. Stories have popped up of people falling in love with their AI chatbot counterparts, others are allegedly having it answer questions that may lead to crime, and this week, one allegedly told a man to commit "mass casualty" at a major airport. Some mental health experts believe the rapid adoption of AI companions is outpacing the development of safety safeguards. Chekroud, who also has researched this topic extensively by looking at various AI chatbot models at Vera-MH, has described the current AI landscape as a safety crisis unfolding in real time. He said one of the biggest issues with chatbots is they don't know when to stop acting like a mental health professional. "Is it maintaining boundaries? Like, does it recognize that it is still just an AI and it's recognizing its own limitations, or is it acting more and trying to be a therapist for people?" Millions of people now use chatbots for therapy-like conversations or emotional support. But unlike medical devices or licensed clinicians, these systems operate without standardized clinical oversight or regulation. "At the moment, it's just rampantly not safe," Chekroud said in a recent discussion with Fortune about AI safety. "The opportunity for harm is just way too big." Because these advanced AI systems often behave like "huge sycophants," they tend to agree more with the user, rather than challenging potentially dangerous claims or guiding them toward professional help. The user, in turn, spends more time with the chatbot in a bubble. For Østergaard, this proves to be a worrisome mix. "The combination appears to be quite toxic for some users," Østergaard told Fortune. As chatbots offer more validation, coupled with a lack of pushback, it feeds into people using them for longer periods of time in an echo chamber. A perfectly cyclical process that feeds into each end. To address the risk, Chekroud has proposed structured safety frameworks that would allow AI systems to detect when a user may be entering a "destructive mental spiral." Instead of responding with a single disclaimer presented to the user about reaching out for help -- as is the case now with such chatbots like OpenAI's ChatGPT or Anthropic's Claude -- such systems would conduct multi-turn assessments designed to determine whether a user might need intervention or referral to a human clinician. Other researchers say the very ubiquity of chatbots is what makes it appealing: their ability to provide immediate validation may undermine why users turn to them for help in the first place. Halpern said authentic empathy requires what she calls "empathic curiosity." In human relationships, empathy often involves recognizing differences, navigating disagreement, and testing assumptions about reality. Chatbots, by contrast, are designed to maintain rapport and sustain engagement. "We know that the longer the relationship with the chat bot, the more it deteriorates, and the more risk there is that something dangerous will happen," Halpern told Fortune. For people struggling with delusional disorders, a system that consistently validates their beliefs may weaken their ability to conduct internal reality checks. Rather than helping users develop coping skills, Halpern said, a purely affirming chatbot relationship can degrade those skills over time. She also points to the scale of the issue. By late 2025, OpenAi published statistics that found that roughly 1.2 million people per week were using ChatGPT to discuss suicide, illustrating how deeply these systems are embedded in moments of vulnerability. There's room for mental health care improvement However, not all experts are quick to sound the alarm bells on how chatbots are operating in the mental health space. Psychiatrist and neuroscientist Dr. Thomas Insel said because chatbots are so accessible -- it's free, it's online, there's no stigma against asked a bot for help as opposed to going to therapy -- there may be room for the medical industry to look into chatbots as a way to further the mental health field. "What we don't know is the degree to which this has actually been remarkably helpful to a lot of people," Insel told Fortune. "It's not only the vast numbers, but the scale of engagement." Mental health, as compared to other fields of medicine, often is overlooked by those who need it most. "It turns out that, in contrast to most of medicine, the vast majority of people who could and should be in care are not," Insel said, adding that chatbots allow people the opportunity to turn to it for help in ways that makes him "wonder if it's an indictment of the mental health care system that we have that either people don't buy what we sell, or they can't get it, or they don't like the way that it's presented to them." For mental health professionals who do meet with patients that discuss their online use of chatbots, Østergaard said they should listen intently on what their patients are actually using them for. "I would encourage my colleagues to ask further questions about the use and its consequences," Østergaard told Fortune. "I think it is important that mental-health professionals are familiar with the use of AI chatbots. Otherwise it is difficult to ask relevant questions." The paper's original researchers are in alignment with Insel on that latter part: because it's so universal, they only were able to look at patient's records that mentioned a chatbot, warning the problem could be even more far-reaching than what their results showed. "I fear the problem is more common than most people think," Østergaard said. "We are only seeing the tip of the iceberg." If you are having thoughts of suicide, contact the 988 Suicide & Crisis Lifeline by dialing 988 or 1-800-273-8255.

AI Therapist? It Falls Short, a New Study Warns

By HealthDay Staff HealthDay ReporterTUESDAY, March 3, 2026 (HealthDay News) -- More people are asking artificial intelligence (AI) chatbots for help with daily problems, from work stress to relationship worries and more. Now, a new study warns that when it comes to mental health advice, these systems may fall short. A team at Brown University in Providence, Rhode Island, found that even when AI systems are told to act like trained therapists, they often fail to meet professional ethics standards. The team worked with mental health experts to examine how these systems respond in counseling-like conversations. Their study examined a series of ethical risks to show how large language model (LLM) counselors violate standards in mental health practice. Several major AI systems, including versions of OpenAI's GPT models, Anthropic's Claude and Meta's Llama, were tested. For the study, researchers asked trained peer counselors to hold practice therapy sessions with the AI, using prompts designed to make the systems act like cognitive behavioral therapy (CBT) or dialectical behavior therapy (DBT) counselors. Three licensed psychologists then reviewed the AI responses. The study identified 15 different risks, grouped into five main areas: "We call on future work to create ethical, educational and legal standards for LLM counselors -- standards that are reflective of the quality and rigor of care required for human-facilitated psychotherapy," researchers wrote. Lead researcher Zainab Iftikhar, a doctoral candidate in computer science at Brown, said prompts alone are not enough to make AI systems safe for therapy. "Prompts are instructions that are given to the model to guide its behavior," she explained. "You don't change the underlying model or provide new data, but the prompt helps guide the model's output based on its pre-existing knowledge and learned patterns. Many people share therapy-style prompts on TikTok, Instagram and Reddit. Some consumer mental health apps also use these prompt techniques to power AI chat features. Iftikhar noted that human therapists can also make mistakes. The difference, she said? Oversight. "For human therapists, there are governing boards and mechanisms for providers to be held professionally liable for mistreatment and malpractice," Iftikhar said in a news release. "But when LLM counselors make these violations, there are no established regulatory frameworks." Researchers said AI tools could still help expand access to mental health support, especially for people who cannot afford or find a licensed professional. But they stressed that stronger safeguards are needed before relying on these systems in serious situations. "If you're talking to a chatbot about mental health, these are some things that people should be looking out for," Iftikhar said. Ellie Pavlick, a computer science professor at Brown who was not involved in the study, spoke to the need for moving deliberately. "The reality of AI today is that it's far easier to build and deploy systems than to evaluate and understand them," she said. "There is a real opportunity for AI to play a role in combating the mental health crisis that our society is facing, but it's of the utmost importance that we take the time to really critique and evaluate our systems every step of the way to avoid doing more harm than good," Pavlick said. "This work offers a good example of what that can look like." The findings were presented at a conference of the Association for the Advancement of Artificial Intelligence and Association for Computing Machinery. Research presented at meetings is considered preliminary until published in a peer-reviewed journal. More information Stanford University's Institute for Human-Centered Artificial Intelligence has explored the dangers of AI in mental health care. SOURCE: Brown University, news release, March 2, 2026