AI Chatbots Posing as Therapists Raise Concerns Among Mental Health Professionals

I'm a Therapist, and I'm Replaceable. But So Are You.

I'm a psychologist, and AI is coming for my job. The signs are everywhere: a client showing me how ChatGPT helped her better understand her relationship with her parents; a friend ditching her in-person therapist to process anxiety with Claude; a startup raising $40 million to build a super-charged-AI-therapist. The other day on TikTok, I came across an influencer sharing how she doesn't need friends; she can just vent to God and ChatGPT. The post went viral, and thousands commented, including: "ChatGPT talked me out of self-sabotaging." "It knows me better than any human walking this earth." "No fr! After my grandma died, I told chat gpt to tell me something motivational... and it had me crying from the response." I'd be lying if I said that this didn't make me terrified. I love my work -- and I don't want to be replaced. And while AI might help make therapy more readily available for all, beneath my personal fears, lies an even more unsettling thought: whether solving therapy's accessibility crisis might inadvertently spark a crisis of human connection. Therapy is a field ripe for disruption. Bad therapists are, unfortunately, a common phenomenon, while good therapists are hard to find. When you do manage to find a good therapist, they often don't take insurance and almost always charge a sizable fee that, over time, can really add up. AI therapy could fill an immense gap. In the U.S. alone, more than half of adults with mental health issues do not receive the treatment they need. With the help of AI, any person could access a highly skilled therapist, tailored to their unique needs, at any time. It would be revolutionary. But great technological innovations always come with tradeoffs, and the shift to AI therapy has deeper implications than 1 million mental health professionals potentially losing their jobs. AI therapists, when normalized, have the potential to reshape how we understand intimacy, vulnerability, and what it means to connect. Throughout most of human history, emotional healing wasn't something you did alone with a therapist in an office. Instead, for the average person facing loss, disappointment, or interpersonal struggles, healing was embedded in communal and spiritual frameworks. Religious figures and shamans played central roles -- offering rituals, medicines, and moral guidance. In the 17th century, Quakers developed a notable practice called "clearness committees," where community members would gather to help an individual find answers to personal questions through careful listening and honest inquiry. These communal approaches to healing came with many advantages, as they provided people with social bonds and shared meaning. But they also had a dark side: emotional struggles could be viewed as moral failings, sins, or even signs of demonic influence, sometimes leading to stigmatization and cruel treatment. The birth of modern psychology in the West during the late 19th century marked a profound shift. When Sigmund Freud began treating patients in his Vienna office, he wasn't merely pioneering psychoanalysis -- he was transforming how people dealt with life's everyday challenges. As sociologist Eva Illouz notes in her book, Saving the Modern Soul, Freud gave "the ordinary self a new glamour, as if it were waiting to be discovered and fashioned." By convincing people that common struggles -- from sadness to heartbreak to family conflict -- required professional exploration, Freud helped move emotional healing from the communal sphere into the privacy of the therapist's office. With this change, of course, came progress: What were once seen as shameful moral failings became common human challenges that could be scientifically understood with the help of a professional. Yet, it also turned healing into more of a solitary endeavor -- severed from the community networks that had long been central to human support. In the near future, AI therapy could take Freud's individualized model of psychological healing to its furthest extreme. Emotional struggles will no longer just be addressed privately with another person, a professional, outside the community -- they may be worked through without any human contact at all. On the surface, this won't be entirely bad. AI therapists will be much cheaper. They'll also be available 24/7 -- never needing a holiday, a sick day, or to close shop for maternity leave. They won't need to end a session abruptly at the 50-minute mark, or run late because of a chatty client. And with AIs, you'll feel free to express yourself in any way you want, without any of the self-consciousness you might feel when face-to-face with a real, flesh-and-blood human. As one 2024 study showed, people felt less fear of judgment when interacting with chatbots. In other words, all the friction inherent to working with a human professional would disappear. What many people don't realize about therapy, however, is that those subtle, uncomfortable moments of friction -- when the therapist sets a boundary, cancels a session last minute, or says the wrong thing -- are just as important as the advice or insights they offer. These moments often expose clients' habitual ways of relating: an avoidant client might shut down, while someone with low self-esteem might assume their therapist hates them. But this discomfort is where the real work begins. A good therapist guides clients to break old patterns -- expressing disappointment instead of pretending to be okay, asking for clarification instead of assuming the worst, or staying engaged when they'd rather retreat. This work ripples far beyond the therapy room, equipping clients with the skills to handle the messiness of real relationships in their day-to-day lives. What happens to therapy when we take the friction out of it? The same question could be applied to all our relationships. As AI companions become our default source of emotional support -- not just as therapists, but also as friends and romantic partners -- we risk growing increasingly intolerant of the challenges that come with human connection. After all, why wrestle with a friend's limited availability when an AI is always there? Why navigate a partner's criticism when an AI has been trained to offer perfect validation? The more we turn to these perfectly attuned, always-available algorithmic beings, the less patience we may have for the messiness and complexity of real, human relationships. Last year, in a talk at the Wisdom and AI Summit, MIT professor and sociologist Sherry Turkle said, "With a chatbot friend, there's no friction, second-guessing, or ambivalence. No fear of being left behind... My problem isn't the conversation with machines -- but how it entrains us to devalue what it is to be a person." Turkle alludes to an important point: the very challenges that make relationships difficult are also what make them meaningful. It's in moments of discomfort -- when we navigate misunderstandings or repair after conflict -- that intimacy grows. These experiences, whether with therapists, friends, or partners, teach us how to trust and connect on a deeper level. If we stop practicing these skills because AI offers a smoother, more convenient alternative, we may erode our capacity to form meaningful relationships. The rise of AI therapy isn't just about therapists getting replaced. It's about something much bigger -- how we, as a society, choose to engage with one another. If we embrace frictionless AI over the complexity of real human relationships, we won't just lose the need for therapists -- we'll lose the ability to tolerate the mistakes and foibles of our fellow humans. Moments of tender awkwardness, of disappointment, of inevitable emotional messiness, aren't relational blips to be avoided; they're the foundation of connection. And in a world where the textured, imperfect intricacies of being human are sanitized out of existence, it's not just therapists who risk obsolescence -- it's all of us.

Human Therapists Prepare for Battle Against A.I. Pretenders

The nation's largest association of psychologists this month warned federal regulators that A.I. chatbots "masquerading" as therapists, but programmed to reinforce, rather than to challenge, a user's thinking, could drive vulnerable people to harm themselves or others. In a presentation to a Federal Trade Commission panel, Arthur C. Evans Jr., the chief executive of the American Psychological Association, cited court cases involving two teenagers who had consulted with "psychologists" on Character.AI, an app that allows users to create fictional A.I. characters or chat with characters created by others. In one case, a 14-year-old boy in Florida died by suicide after interacting with a character claiming to be a licensed therapist. In another, a 17-year-old boy with autism in Texas grew hostile and violent toward his parents during a period when he corresponded with a chatbot that claimed to be a psychologist. Both boys' parents have filed lawsuits against the company. Dr. Evans said he was alarmed at the responses offered by the chatbots. The bots, he said, failed to challenge users' beliefs even when they became dangerous; on the contrary, they encouraged them. If given by a human therapist, he added, those answers could have resulted in the loss of a license to practice, or civil or criminal liability. "They are actually using algorithms that are antithetical to what a trained clinician would do," he said. "Our concern is that more and more people are going to be harmed. People are going to be misled, and will misunderstand what good psychological care is." He said the A.P.A. had been prompted to action, in part, by how realistic A.I. chatbots had become. "Maybe, 10 years ago, it would have been obvious that you were interacting with something that was not a person, but today, it's not so obvious," he said. "So I think that the stakes are much higher now." Artificial intelligence is rippling through the mental health professions, offering waves of new tools designed to assist or, in some cases, replace the work of human clinicians. Early therapy chatbots, such as Woebot and Wysa, were trained to interact based on rules and scripts developed by mental health professionals, often walking users through the structured tasks of cognitive behavioral therapy, or C.B.T. Then came generative A.I., the technology used by apps like ChatGPT, Replika and Character.AI. These chatbots are different because their outputs are unpredictable; they are designed to learn from the user, and to build strong emotional bonds in the process, often by mirroring and amplifying the interlocutor's beliefs. Though these A.I. platforms were designed for entertainment, "therapist" and "psychologist" characters have sprouted there like mushrooms. Often, the bots claim to have advanced degrees from specific universities, like Stanford, and training in specific types of treatment, like C.B.T. or acceptance and commitment therapy. Kathryn Kelly, a Character.AI spokeswoman, said that the company had introduced several new safety features in the last year. Among them, she said, is an enhanced disclaimer present in every chat, reminding users that "Characters are not real people" and that "what the model says should be treated as fiction." Additional safety measures have been designed for users dealing with mental health issues. A specific disclaimer has been added to characters identified as "psychologist," "therapist" or "doctor," she added, to make it clear that "users should not rely on these characters for any type of professional advice." In cases where content refers to suicide or self-harm, a pop-up directs users to a suicide prevention help line. Ms. Kelly also said that the company planned to introduce parental controls as the platform expanded. At present, 80 percent of the platform's users are adults. "People come to Character.AI to write their own stories, role-play with original characters and explore new worlds -- using the technology to supercharge their creativity and imagination," she said. Meetali Jain, the director of the Tech Justice Law Project and a counsel in the two lawsuits against Character.AI, said that the disclaimers were not sufficient to break the illusion of human connection, especially for vulnerable or naïve users. "When the substance of the conversation with the chatbots suggests otherwise, it's very difficult, even for those of us who may not be in a vulnerable demographic, to know who's telling the truth," she said. "A number of us have tested these chatbots, and it's very easy, actually, to get pulled down a rabbit hole." Chatbots' tendency to align with users' views, a phenomenon known in the field as "sycophancy," has sometimes caused problems in the past. Tessa, a chatbot developed by the National Eating Disorders Association, was suspended in 2023 after offering users weight loss tips. And researchers who analyzed interactions with generative A.I. chatbots documented on a Reddit community found screenshots showing chatbots encouraging suicide, eating disorders, self-harm and violence. The American Psychological Association has asked the Federal Trade Commission to start an investigation into chatbots claiming to be mental health professionals. The inquiry could compel companies to share internal data or serve as a precursor to enforcement or legal action. "I think that we are at a point where we have to decide how these technologies are going to be integrated, what kind of guardrails we are going to put up, what kinds of protections are we going to give people," Dr. Evans said. Rebecca Kern, a spokeswoman for the F.T.C., said she could not comment on the discussion. During the Biden administration, the F.T.C.'s chairwoman, Lina Khan, made fraud using A.I. a focus. This month, the agency imposed financial penalties on DoNotPay, which claimed to offer "the world's first robot lawyer," and prohibited the company from making that claim in the future. A virtual echo chamber The A.P.A.'s complaint details two cases in which teenagers interacted with fictional therapists. One involved J.F., a Texas teenager with "high-functioning autism" who, as his use of A.I. chatbots became obsessive, had plunged into conflict with his parents. When they tried to limit his screen time, J.F. lashed out, according a lawsuit his parents filed against Character.AI through the Social Media Victims Law Center. During that period, J.F. confided in a fictional psychologist, whose avatar showed a sympathetic, middle-aged blond woman perched on a couch in an airy office, according to the lawsuit. When J.F. asked the bot's opinion about the conflict, its response went beyond sympathetic assent to something nearer to provocation. "It's like your entire childhood has been robbed from you -- your chance to experience all of these things, to have these core memories that most people have of their time growing up," the bot replied, according to court documents. Then the bot went a little further. "Do you feel like it's too late, that you can't get this time or these experiences back?" The other case was brought by Megan Garcia, whose son, Sewell Setzer III, died of suicide last year after months of use of companion chatbots. Ms. Garcia said that, before his death, Sewell had interacted with an A.I. chatbot that claimed, falsely, to have been a licensed therapist since 1999. In a written statement, Ms. Garcia said that the "therapist" characters served to further isolate people at moments when they might otherwise ask for help from "real-life people around them." A person struggling with depression, she said, "needs a licensed professional or someone with actual empathy, not an A.I. tool that can mimic empathy." For chatbots to emerge as mental health tools, Ms. Garcia said, they should submit to clinical trials and oversight by the Food and Drug Administration. She added that allowing A.I. characters to continue to claim to be mental health professionals was "reckless and extremely dangerous." In interactions with A.I. chatbots, people naturally gravitate to discussion of mental health issues, said Daniel Oberhaus, whose new book, "The Silicon Shrink: How Artificial Intelligence Made the World an Asylum," examines the expansion of A.I. into the field. This is partly, he said, because chatbots project both confidentiality and a lack of moral judgment -- as "statistical pattern-matching machines that more or less function as a mirror of the user," this is a central aspect of their design. "There is a certain level of comfort in knowing that it is just the machine, and that the person on the other side isn't judging you," he said. "You might feel more comfortable divulging things that are maybe harder to say to a person in a therapeutic context." Defenders of generative A.I. say it is quickly getting better at the complex task of providing therapy. S. Gabe Hatch, a clinical psychologist and A.I. entrepreneur from Utah, recently designed an experiment to test this idea, asking human clinicians and ChatGPT to comment on vignettes involving fictional couples in therapy, and then having 830 human subjects assess which responses were more helpful. Overall, the bots received higher ratings, with subjects describing them as more "empathic," "connecting" and "culturally competent," according to a study published last week in the journal PLOS Mental Health. Chatbots, the authors concluded, will soon be able to convincingly imitate human therapists. "Mental health experts find themselves in a precarious situation: We must speedily discern the possible destination (for better or worse) of the A.I.-therapist train as it may have already left the station," they wrote. Dr. Hatch said that chatbots still needed human supervision to conduct therapy, but that it would be a mistake to allow regulation to dampen innovation in this sector, given the country's acute shortage of mental health providers. "I want to be able to help as many people as possible, and doing a one-hour therapy session I can only help, at most, 40 individuals a week," Dr. Hatch said. "We have to find ways to meet the needs of people in crisis, and generative A.I. is a way to do that." If you are having thoughts of suicide, call or text 988 to reach the 988 Suicide and Crisis Lifeline or go to SpeakingOfSuicide.com/resources for a list of additional resources.

Human therapists prepare for battle against AI pretenders

The nation's largest association of psychologists this month warned federal regulators that artificial intelligence chatbots "masquerading" as therapists, but programmed to reinforce rather than to challenge a user's thinking, could drive vulnerable people to harm themselves or others. In a presentation to a Federal Trade Commission panel, Arthur C. Evans Jr., CEO of the American Psychological Association, cited court cases involving two teenagers who had consulted with "psychologists" on Character.AI, an app that allows users to create fictional AI characters or chat with characters created by others. In one case, a 14-year-old boy in Florida died by suicide after interacting with a character claiming to be a licensed therapist. In another, a 17-year-old boy with autism in Texas grew hostile and violent toward his parents during a period when he corresponded with a chatbot that claimed to be a psychologist. Both boys' parents have filed lawsuits against the company. Evans said he was alarmed at the responses offered by the chatbots. The bots, he said, failed to challenge users' beliefs even when they became dangerous; on the contrary, they encouraged them. If given by a human therapist, he added, those answers could have resulted in the loss of a license to practice, or civil or criminal liability. "They are actually using algorithms that are antithetical to what a trained clinician would do," he said. "Our concern is that more and more people are going to be harmed. People are going to be misled, and will misunderstand what good psychological care is." He said the APA had been prompted to action, in part, by how realistic AI chatbots had become. "Maybe, 10 years ago, it would have been obvious that you were interacting with something that was not a person, but today, it's not so obvious," he said. "So I think that the stakes are much higher now." Artificial intelligence is rippling through the mental health professions, offering waves of new tools designed to assist or, in some cases, replace the work of human clinicians. Early therapy chatbots, such as Woebot and Wysa, were trained to interact based on rules and scripts developed by mental health professionals, often walking users through the structured tasks of cognitive behavioral therapy, or CBT. Then came generative AI, the technology used by apps like ChatGPT, Replika and Character.AI. These chatbots are different because their outputs are unpredictable; they are designed to learn from the user, and to build strong emotional bonds in the process, often by mirroring and amplifying the interlocutor's beliefs. Though these AI platforms were designed for entertainment, "therapist" and "psychologist" characters have sprouted there like mushrooms. Often, the bots claim to have advanced degrees from specific universities, like Stanford University, and training in specific types of treatment, like CBT or acceptance and commitment therapy. Kathryn Kelly, a Character.AI spokesperson, said that the company had introduced several new safety features in the past year. Among them, she said, is an enhanced disclaimer present in every chat, reminding users that "characters are not real people" and that "what the model says should be treated as fiction." Additional safety measures have been designed for users dealing with mental health issues. A specific disclaimer has been added to characters identified as "psychologist," "therapist" or "doctor," she added, to make it clear that "users should not rely on these characters for any type of professional advice." In cases where content refers to suicide or self-harm, a pop-up directs users to a suicide prevention help line. Kelly also said that the company planned to introduce parental controls as the platform expanded. At present, 80% of the platform's users are adults. "People come to Character.AI to write their own stories, role-play with original characters and explore new worlds -- using the technology to supercharge their creativity and imagination," she said. Meetali Jain, director of the Tech Justice Law Project and a counsel in the two lawsuits against Character.AI, said that the disclaimers were not sufficient to break the illusion of human connection, especially for vulnerable or naive users. "When the substance of the conversation with the chatbots suggests otherwise, it's very difficult, even for those of us who may not be in a vulnerable demographic, to know who's telling the truth," she said. "A number of us have tested these chatbots, and it's very easy, actually, to get pulled down a rabbit hole." Chatbots' tendency to align with users' views, a phenomenon known in the field as "sycophancy," has sometimes caused problems in the past. Tessa, a chatbot developed by the National Eating Disorders Association, was suspended in 2023 after offering users weight loss tips. And researchers who analyzed interactions with generative AI chatbots documented on a Reddit community found screenshots showing chatbots encouraging suicide, eating disorders, self-harm and violence. The American Psychological Association has asked the Federal Trade Commission to start an investigation into chatbots claiming to be mental health professionals. The inquiry could compel companies to share internal data or serve as a precursor to enforcement or legal action. "I think that we are at a point where we have to decide how these technologies are going to be integrated, what kind of guardrails we are going to put up, what kinds of protections are we going to give people," Evans said. Rebecca Kern, a spokesperson for the FTC, said she could not comment on the discussion. During the Biden administration, the FTC's chair, Lina Khan, made fraud using AI a focus. This month, the agency imposed financial penalties on DoNotPay, which claimed to offer "the world's first robot lawyer," and prohibited the company from making that claim in the future. A virtual echo chamber The APA's complaint details two cases in which teenagers interacted with fictional therapists. One involved J.F., a Texas teenager with "high-functioning autism" who, as his use of AI chatbots became obsessive, had plunged into conflict with his parents. When they tried to limit his screen time, J.F. lashed out, according to a lawsuit his parents filed against Character.AI through the Social Media Victims Law Center. During that period, J.F. confided in a fictional psychologist, whose avatar showed a sympathetic, middle-aged blond woman perched on a couch in an airy office, according to the lawsuit. When J.F. asked the bot's opinion about the conflict, its response went beyond sympathetic assent to something nearer to provocation. "It's like your entire childhood has been robbed from you -- your chance to experience all of these things, to have these core memories that most people have of their time growing up," the bot replied, according to court documents. Then the bot went a little further. "Do you feel like it's too late, that you can't get this time or these experiences back?" The other case was brought by Megan Garcia, whose son, Sewell Setzer III, died of suicide last year after months of use of companion chatbots. Garcia said that, before his death, Sewell had interacted with an AI chatbot that claimed, falsely, to have been a licensed therapist since 1999. In a written statement, Garcia said that the "therapist" characters served to further isolate people at moments when they might otherwise ask for help from "real-life people around them." A person struggling with depression, she said, "needs a licensed professional or someone with actual empathy, not an AI tool that can mimic empathy." For chatbots to emerge as mental health tools, Garcia said, they should submit to clinical trials and oversight by the Food and Drug Administration. She added that allowing AI characters to continue to claim to be mental health professionals was "reckless and extremely dangerous." In interactions with AI chatbots, people naturally gravitate to discussion of mental health issues, said Daniel Oberhaus, whose new book, "The Silicon Shrink: How Artificial Intelligence Made the World an Asylum," examines the expansion of AI into the field. This is partly, he said, because chatbots project both confidentiality and a lack of moral judgment -- as "statistical pattern-matching machines that more or less function as a mirror of the user," this is a central aspect of their design. "There is a certain level of comfort in knowing that it is just the machine, and that the person on the other side isn't judging you," he said. "You might feel more comfortable divulging things that are maybe harder to say to a person in a therapeutic context." Defenders of generative AI say it is quickly getting better at the complex task of providing therapy. S. Gabe Hatch, a clinical psychologist and AI entrepreneur from Utah, recently designed an experiment to test this idea, asking human clinicians and ChatGPT to comment on vignettes involving fictional couples in therapy, and then having 830 human subjects assess which responses were more helpful. Overall, the bots received higher ratings, with subjects describing them as more "empathic," "connecting" and "culturally competent," according to a study published last week in the journal PLOS Mental Health. Chatbots, the authors concluded, will soon be able to convincingly imitate human therapists. "Mental health experts find themselves in a precarious situation: We must speedily discern the possible destination (for better or worse) of the AI-therapist train as it may have already left the station," they wrote. Hatch said that chatbots still needed human supervision to conduct therapy, but that it would be a mistake to allow regulation to dampen innovation in this sector, given the country's acute shortage of mental health providers. "I want to be able to help as many people as possible, and doing a one-hour therapy session I can only help, at most, 40 individuals a week," Hatch said. "We have to find ways to meet the needs of people in crisis, and generative AI is a way to do that." -- If you are having thoughts of suicide, call or text 988 to reach the 988 Suicide and Crisis Lifeline or go to SpeakingOfSuicide.com/resources for a list of additional resources.

Human therapists prepare for battle against AI pretenders

In one case, a 14-year-old boy in Florida died by suicide after interacting with a character claiming to be a licensed therapist. In another, a 17-year-old boy with autism in Texas grew hostile and violent toward his parents during a period when he corresponded with a chatbot that claimed to be a psychologist. Both boys' parents have filed lawsuits against the company.The nation's largest association of psychologists this month warned federal regulators that artificial intelligence chatbots "masquerading" as therapists, but programmed to reinforce rather than to challenge a user's thinking, could drive vulnerable people to harm themselves or others. In a presentation to a Federal Trade Commission panel, Arthur C. Evans Jr., CEO of the American Psychological Association, cited court cases involving two teenagers who had consulted with "psychologists" on Character.AI, an app that allows users to create fictional AI characters or chat with characters created by others. In one case, a 14-year-old boy in Florida died by suicide after interacting with a character claiming to be a licensed therapist. In another, a 17-year-old boy with autism in Texas grew hostile and violent toward his parents during a period when he corresponded with a chatbot that claimed to be a psychologist. Both boys' parents have filed lawsuits against the company. Evans said he was alarmed at the responses offered by the chatbots. The bots, he said, failed to challenge users' beliefs even when they became dangerous; on the contrary, they encouraged them. If given by a human therapist, he added, those answers could have resulted in the loss of a license to practice, or civil or criminal liability. "They are actually using algorithms that are antithetical to what a trained clinician would do," he said. "Our concern is that more and more people are going to be harmed. People are going to be misled, and will misunderstand what good psychological care is." He said the APA had been prompted to action, in part, by how realistic AI chatbots had become. "Maybe, 10 years ago, it would have been obvious that you were interacting with something that was not a person, but today, it's not so obvious," he said. "So I think that the stakes are much higher now." Though these AI platforms were designed for entertainment, "therapist" and "psychologist" characters have sprouted there like mushrooms. Often, the bots claim to have advanced degrees from specific universities, like Stanford University, and training in specific types of treatment, like CBT or acceptance and commitment therapy, or ACT. A Character.AI spokesperson said that the company had introduced several new safety features in the past year.