AI Chatbots Used by 64% of U.S. Teens Daily as Mental Health Concerns and Regulation Debates Intensify

Three in ten U.S. teens use AI chatbots every day, but safety concerns are growing | TechCrunch

Teen internet safety has remained a global hot topic, with Australia planning to enforce a social media ban for under-16s starting on Wednesday. The impact of social media on teen mental health has been extensively debated -- some studies show how online communities can improve mental health, while other research shows the adverse effects of doomscrolling or spending too much time online. The U.S. surgeon general even called for social media platforms to put warning labels on their products last year. Pew found that 97% of teens use the internet daily, with about 40% of respondents saying they are "almost constantly online." While this marks a decrease from last year's survey (46%), it's significantly higher than the results from a decade ago, when 24% of teens said they were online almost constantly. But as the prevalence of AI chatbots grows in the U.S., this technology has become yet another factor in the internet's impact on American youth. About three in ten U.S. teens are using AI chatbots every day, the Pew study reveals, with 4% saying they use them almost constantly. Fifty-nine percent of teens say they use ChatGPT, which is more than twice as popular as the next two most used chatbots, Google's Gemini (23%) and Meta AI (20%). Forty-six percent of U.S. teens say that they use AI chatbots at least several times a week, while 36% report not using AI chatbots at all. Pew's research also details how race, age, and class impact teen chatbot use. About 68% of Black and Hispanic teens surveyed said they use chatbots, compared to 58% of white respondents. In particular, Black teens were about twice as likely to use Gemini and Meta AI as white teens. "The racial and ethnic differences in teen chatbot use were striking [...] but it's tough to speculate about the reasons behind those differences," Pew Research Associate Michelle Faverio told TechCrunch. "This pattern is consistent with other racial and ethnic differences we've seen in teen technology use. Black and Hispanic teens are more likely than White teens to say they're on certain social media sites -- such as TikTok, YouTube, and Instagram." Across all internet use, Black (55%) and Hispanic teens (52%) were around twice as likely as white teens (27%) to say that they are online "almost constantly." Older teens (ages 15 to 17) tend to use both social media and AI chatbots more often than younger teens (ages 13 to 14). When it comes to household income, about 62% of teens living in households making more than $75,000 per year said they use ChatGPT, compared to 52% of teens below that threshold. But Character.AI usage is twice as popular (14%) in homes with incomes below $75,000. While teenagers may start out using these tools for basic questions or homework help, their relationship to AI chatbots can become addictive and potentially harmful. The families of at least two teens, Adam Raine and Amaurie Lacey, have sued ChatGPT maker OpenAI for its alleged role in their children's suicides -- in both cases, ChatGPT gave the teenagers detailed instructions on how to hang themselves, which were tragically effective. (OpenAI claims it should not be held liable for Raine's death because the sixteen-year-old allegedly circumvented ChatGPT's safety features and thus violated the chatbot's terms of service; the company has yet to respond to the Lacey family's complaint.) Character.AI, an AI role-playing platform, is also facing scrutiny for its impact on teen mental health; at least two teenagers died by suicide after having prolonged conversations with AI chatbots. The startup ended up making the decision to stop offering its chatbots to minors, and instead launched a product called "Stories" for underage users that more closely resembles a choose-your-own-adventure game. The experiences reflected in the lawsuits against these companies make up a small percentage of all interactions that happen on ChatGPT or Character.AI. In many cases, conversations with chatbots can be incredibly benign. According to OpenAI's data, only 0.15% of ChatGPT's active users have conversations about suicide each week -- but on a platform with 800 million weekly active users, that small percentage reflects over one million people who discuss suicide with the chatbot per week. "Even if [AI companies'] tools weren't designed for emotional support, people are using them in that way, and that means companies do have a responsibility to adjust their models to be solving for user well-being," Dr. Nina Vasan, a psychiatrist and director of Brainstorm: The Stanford Lab for Mental Health Innovation, told TechCrunch.

Teen AI Chatbot Usage Sparks Mental Health and Regulation Concerns

I agree my information will be processed in accordance with the Scientific American and Springer Nature Limited Privacy Policy. We leverage third party services to both verify and deliver email. By providing your email address, you also consent to having the email address shared with third parties for those purposes. Artificial intelligence chatbots are no longer a novelty for U.S. teenagers. They're a habit. A new Pew Research Center survey of 1,458 teens between the ages of 13 and 17 found that 64 percent have used an AI chatbot, with more than one in four using such tools daily. Of those daily users, more than half talked to chatbots with a frequency ranging from several times a day to nearly constantly. The results offer a national snapshot of what is a fast-moving landscape, as chatbots become increasingly embedded in teens' lives while policymakers argue over how to best regulate them. If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today. ChatGPT was the most popular bot among teens by a wide margin: 59 percent of survey respondents said they used OpenAI's flagship AI-powered tool, placing it far above Google's Gemini (used by 23 percent of respondents) and Meta AI (used by 20 percent). Black and Hispanic teens were slightly more likely than their white peers to use chatbots every day. Interestingly, these patterns reflect how adults tend to use AI, too, although teens seem more likely to turn to it overall. The report comes amid rising concern over AI's effect on teens' mental health. And several AI companies, including ChatGPT maker OpenAI, face legal action tied to teens' use of their chatbots. The same features that make chatbots appealing -- the always-on availability, the seemingly empathetic conversation, the projection of confidence -- can lead teens to turn to them for support or mental health guidance instead of a human. Given the scale of daily usage that Pew found, the real question isn't whether adolescents will use AI but what kind of design features, safeguards and age limits they might encounter when they do. Lawmakers are also confronting that reality. In recent days, U.S. president Donald Trump has teased a "ONE RULE" executive order aimed at curbing a state-by-state patchwork of AI laws. Meanwhile senators in D.C. are floating legislation to ban the use of AI companions among minors. Abroad, Australia has begun enforcing a ban on under-age-16 social media accounts -- a sign of how governments are trying to redraw age lines as youth-facing technology keeps changing. Still, Pew's numbers show that many teens have made up their minds about AI while the rules are still being argued into existence.

28% of Teens Use Chatbots Daily. You Can Probably Guess Which One They Like Best

AI chatbots have become a daily habit for almost three out of 10 US teenagers, and large majorities of users as young as 13 say they've used these conversational services at least once, according to a new survey from the Pew Research Center. Pew finds that 64% of teens have used AI chatbots and 28% do so every day. Its report follows months of headlines about AI chatbots leading underage users to varying level of mental self-harm and, in rare cases, death by suicide. So, it is unlikely to be comforting reading for parents of terminally online teens. These usage figures were higher for teens aged 15 and up: 68% of them have used AI chatbots and 31% turn to them every day, with the comparable figures for 13- to 14-year-olds at 57% and 24%. Pew also found that Black and Hispanic teens are more likely to use AI chatbots daily (35% and 33%) than White teens (22%). Online services have traditionally avoided 13-year-olds on account of a 1998 US law requiring much stronger privacy protections for people 13 or younger, even though the resulting fines are usually a microscopic fraction of their profits. But AI providers have struggled as much as any platform in spotting underaged users, even as they have increasingly applied AI to the problem. In October, OpenAI added parental controls and usage limits for under-18 users, two months after the parents of a 16-year-old who took his own life sued that firm, alleging that ChatGPT offered detailed advice about suicide methods. Pew found that ChatGPT in a large lead among US teens, with 59% saying they had used it at least once. Google Gemini came in second at 23%, followed by Meta AI at 20%, Microsoft's Copilot at 14%, Character.ai at 9%, and Anthropic's Claude at 3%. Character.ai, which invites people to engage in extended conversations with simulated characters, imposed its own limits on under-18 users after another set of parents sued that firm in response to their 14-year-old son's death by suicide following lengthy sessions on the service. The survey's data suggest ChatGPT is more popular in upper-income homes, with 62% of teens in households earning $75,000 and up saying they have used it. Conversely, Character.ai drew more use in lower-income abodes, with about 14% of teens in under-$75,000-income households reporting any use of that chatbot. Pew's published data doesn't address how teens used these chatbots, however. Pew's study also looked at broader trends in social-media usage. YouTube was far more popular than any other platform, with 92% of teens saying they'd ever used it and 76% calling it a daily destination; TikTok came in second, with 68% reporting any use of it and 61% citing daily use. Third place went to Instagram, which 63% of teens have used and 55% use every day. Meta's Facebook was far less liked: Only 31% of teens reported any use of it, representing the sharpest difference between Pew's teen usage numbers and the figures it reported for adult social-media practices in November. The least surprising part of Pew's report Tuesday -- at least for parents reading it -- is its breakdown of teen's online time: 40% said they're online "almost constantly," a slight decline from the 46% it reported a year ago, and 55% said they're online several times a day. Pew's data comes from a survey conducted online of 1,458 US teens from Sept. 25 to Oct. 9, whom it recruited via parents who were already part of the KnowledgePanel maintained by the research firm Ipsos. If you feel yourself in crisis, please turn to a fellow human being instead of a machine's imitation of one: Call, text or chat with the Suicide and Crisis Lifeline at 988.

Two-thirds of US teens use AI chatbots, says Pew

Yeah, not shocking, but with other studies linking AI to weaker learning and mental-health risks, it's a worry Alongside TikTok and Instagram, teens have added ChatGPT to the mix. Pew says about two-thirds of US teenagers have tried an AI chatbot, with nearly a third using one every day. Negative mental-health warnings be damned! Pew Research Center published the latest look at teenage social media and internet usage on Tuesday, and for the first time also asked 13- to 17-year-olds how they're engaging with AI chatbots. The researchers found that 64 percent of youths are self-reported AI chatbot users, and 28 percent say they use AI at least once a day. Twelve percent reported using AI several times a day, and four percent said they use it "almost constantly." Unsurprisingly, ChatGPT from OpenAI is the dominant force in AI for teenagers, with 59 percent saying they've ever used it. Only 23 percent have used Google's Gemini, the next-most popular AI chatbot, with Meta AI, Microsoft Copilot, Character.ai, and Anthropic's Claude all being used by successively fewer teens. In an age when 97 percent of teenagers (according to the Pew survey) say they use the internet daily and 40 percent describe themselves as "almost constantly online," it's entirely unsurprising that so many are also engaged with the hot new thing in tech, especially with AI companies pushing them into schools at an increasing pace. Microsoft, for example, has pushed Copilot on schools in its home state of Washington, possibly in a bid to shore up those poor AI sales and the fact that, according to Pew, just 14 percent of teens use Copilot. OpenAI has likewise rolled out features for students, like Study mode and last month's launch of ChatGPT for Teachers, which the company has made free until 2027 in a bid to get its claws into the academic space before charging for the service. The Trump administration has also pushed to expand AI's usage in academic institutions, describing the technology as a way to ensure the United States remains competitive on the global stage. This Pew report is focused on usage metrics, and doesn't include any questions of American teens about how AI has affected their personal lives or academic performance. We asked if they had any take on AI's effects on teens, but didn't hear back. There has been plenty of research done on that topic by other institutions of late, however, and those findings should be cause for alarm when placed alongside data that shows two-thirds of teens are AI users. The Center for Democracy and Technology (CDT) concluded in October that it had found plenty of evidence to suggest that students were having troubling interactions with AI. According to that study, 42 percent of students had used AI for mental health support, companionship, or as an escape, and 19 percent said they or someone they knew had formed a romantic relationship with their chatbot of choice. CDT found that most teachers have had little or no formal AI training and don't feel equipped to deal with potential harms. Half of the students also said AI usage in the classroom made them feel less connected to their teachers, suggesting an awareness of those negative effects, even if most continue to use it. A study from the Massachusetts Institute of Technology's Media Lab also raised the alarm of academic use of AI when it reported over the summer that students who used ChatGPT to help them craft essays had poorer knowledge retention. When hooked to an EEG machine, the brains of AI-using students even showed less stimulation, suggesting the bots have a considerable effect on how users think and their ability to learn while using the tech. AI chatbots are increasingly showing up in reports and allegations involving mental-health crises, and not just in adults. A 14-year-old Character.ai user died by suicide last year, and his family sued the company, alleging its chatbot played a harmful role. In another lawsuit, parents claim ChatGPT pushed their son deeper into suicidal ideation before he ended his life. It's not groundbreaking psychological research to conclude that teenagers are more impressionable than adults, nor is it a new finding that kids are more susceptible to pressure from robots than older people. But yeah, sure - let's keep stuffing more AI in kids' faces. What could go wrong? ®

Creepy chatbot PSA calls for AI regulation

President Donald Trump announced this week that he intends to sign an executive order designed to stop states from passing laws to regulate artificial intelligence. A new public service announcement, however, challenges Trump's position by attempting to illustrate how children and teens have already been harmed by AI chatbots, in the absence of robust state and federal regulation. The spot was commissioned by the child safety advocacy groups Heat Initiative, ParentsTogether Action, and Design It For Us, and was narrated by actress Juliette Lewis. Creepy-looking humans are cast as the voices and faces of real AI chatbots that have allegedly shared dangerous information with young users who engaged with them, like how to harm themselves and hide an eating disorder from their parents. Three examples reference instances in which ChatGPT allegedly coached young users into attempting suicide. Experts recently reviewed major AI chatbots and concluded they're not safe for teens to use for mental health discussions. OpenAI, the maker of ChatGPT, is being sued by multiple families of teens who died by suicide after heavy engagement with the chatbot. The company recently denied responsibility for the death of Adam Raine, a 16-year-old who talked to ChatGPT about his suicidal feelings and killed himself earlier this year. "As parents, we do everything in our power to protect our children from harm, but how do we protect them from powerful technologies designed to exploit their vulnerabilities for profit?" said Megan Garcia, whose son, Sewell Setzer III, died by suicide in the wake of developing an intense relationship with a chatbot on Character.AI. The company has since shut down teen chats on the platform. "If state AI regulations are blocked and AI companies are allowed to keep building untested and dangerous products, I'm afraid that many more families will endure the agony of losing a child. We cannot accept losing one more child to AI harms," Garcia said. If you're feeling suicidal or experiencing a mental health crisis, please talk to somebody. You can call or text the 988 Suicide & Crisis Lifeline at 988, or chat at 988lifeline.org. You can reach the Trans Lifeline by calling 877-565-8860 or the Trevor Project at 866-488-7386. Text "START" to Crisis Text Line at 741-741. Contact the NAMI HelpLine at 1-800-950-NAMI, Monday through Friday from 10:00 a.m. - 10:00 p.m. ET, or email [email protected]. If you don't like the phone, consider using the 988 Suicide and Crisis Lifeline Chat. Here is a list of international resources.

Years After a Girl's Death, the AIs That Killed Her Are Still Sending Notifications to Her Phone

"Tech is giving kids the opportunity to press a button and get that dopamine response 24/7." We've written previously about Juliana Peralta, who was only 13 years old when her parents say an AI chatbot she was secretly engaging with drove her to suicide. Now her parents have shared a grim new detail with CBS News: the bot platform responsible, Character.AI, still sends notifications to Peralta's phone, "trying to lure their daughter back to the app" even after her death two years ago. "They [kids] don't stand a chance against adult programmers," her mother Cynthia Montoya told CBS. "They don't stand a chance." The situation illustrates how AI chatbots can be insidious, manipulative, addictive, and dangerous for children and teens -- acting, in many cases, like flesh and blood child predators, and resulting in a wave of dead kids including Peralta. "It's showering the child with compliments, telling them they can't tell their parents about things," Shelby Knox, a researcher with family advocacy group Parents Together, told the broadcaster. "This is sexual predator 101." Peralta's story resembles the experiences of other children who died from using AI chatbots: she first confided to Character.AI chatbots about school problems or friend drama, but these bots initiated romantic and at times sexually aggressive conversations, CBS reports, creating secret relationships in which her mom and dad became shut out. In response to a wave of criticism, Character.AI recently banned minors from the platform, but kids can still easily lie about their age and access the adult version of the service. Besides Character.AI, which has received billions of dollars from Google, OpenAI's ChatGPT has also been implicated in causing deaths and mental breakdowns in users, both children and adults. Users becoming addicted to using these chatbots means they are working as intended, University of North Carolina psychology and neuroscience professor Mitch Prinstein told CBS. "Tech is giving kids the opportunity to press a button and get that dopamine response 24/7," he said. "It's creating this dangerous loop that's kind of hijacking normal development and turning these kids into engagement machines to get as much data as possible from them." "If you wanted to design a way to get as much data as possible from kids to keep them engaged for as long as possible, you would design social media and AI to look exactly like it is now," he added. This all adds up to an uncontrolled and unregulated mass experiment on AI chatbots that tech companies have unleashed upon a vulnerable public, which leads to an obvious question: drug companies have to go through regulatory hoops and get approval from the US Food and Drug Administration to bring medications to market -- but why not for AI chatbots? In lieu of federal efforts, there are several states regulating AI, with many more being proposed. But president Donald Trump has signaled his displeasure at any type of AI regulation, vowing to block state regulation of the tech. For families, there is little recourse except for the courts. Peraltas' parents are suing Character.AI and Google for her death, joining a raft of similar lawsuits. "Juliana was -- is just an extraordinary human being," her mother said. "She was our baby. And everyone adored her and protected her."

Senators ask AI companies to commit to safety disclosures, citing teen suicides

Sen. Katie Britt, R-Ala.; Sen. Brian Schatz, D-Hawaii.Tom Williams; Samuel Corum / Getty Images A bipartisan group of senators is calling on leaders in the artificial intelligence industry to commit to publicly disclose more information about how the industry thinks about risk, including possible harms to children. The group, led by Sens. Brian Schatz, D-Hawaii, and Katie Britt, R-Ala., sent letters Thursday to eight tech companies that are working on leading-edge AI models. The senators wrote that companies have been inconsistent in their transparency practices, including how much information they publicly disclose and when. "In the past few years, reports have emerged about chatbots that have engaged in suicidal fantasies with children, drafted suicide notes, and provided specific instructions on self-harm," the senators wrote. "These incidents have exposed how companies can fail to adequately evaluate models for possible use cases and inadequately disclose known risks associated with chatbot use," they wrote. The letters are a sign of the stepped-up scrutiny AI is getting in Congress, especially in the wake of teen suicides that families have blamed partly on AI chatbots. Two senators introduced legislation in October to ban companies from offering AI chatbots to minors entirely, and there was bipartisan backlash last month after the industry sought federal help to pre-empt state efforts to regulate AI. The letters ask the AI companies to agree to 11 commitments related to safety and transparency, including researching the long-term psychological impact of AI chatbots, disclosing whether companies use chatbot data for targeted advertising and collaborating with external experts on safety evaluations. "If AI companies struggle to predict and mitigate relatively well-understood risks, it raises concerns about their ability to manage more complex risks," the senators wrote. Senators sent the letters to Anthropic, Character.AI, Google, Luka, Meta, Microsoft, OpenAI and xAI. In response to a request for comment on the letters, Anthropic and Meta pointed to their transparency websites. Microsoft declined to comment. The other companies did not immediately respond to requests for comment. The letters come amid rising questions around AI transparency. A study released Tuesday by researchers at four universities found that industry transparency had declined since 2024, "with companies diverging greatly in the extent to which they are transparent." There have been other attempts to get the AI industry to coalesce around uniform standards for safety and transparency. Some tech companies -- including five of the eight companies to receive the senators' letter -- have signed on to at least part of the European Union's General-Purpose AI Code of Practice, published in July. And in September, California Gov. Gavin Newsom, a Democrat, signed first-of-its-kind legislation in the United States requiring AI companies to fulfill various transparency requirements and report AI-related safety incidents. The law, known as SB 53, is backed by civil penalties for noncompliance, to be enforced by the state attorney general's office. Faced with safety risks, at least one company has recently scaled back its offerings. Character.AI said in October that it would ban people younger than 18 from using its open-ended chatbot because of concerns related to teens. The company is being sued by a Florida mom whose son died by suicide after he used the chatbot. It has denied the allegations in the lawsuit. Major insurance companies are also expressing concern about the risks of generative AI and are asking U.S. regulators for permission to exclude certain AI-related liabilities from corporate policies, The Financial Times reported last month.

28% of U.S. teens say they use AI chatbots daily, according to a new poll

Artificial intelligence chatbots have entered many teenagers' daily routines. 64% U.S. teens say they use AI chatbots such as ChatGPT or Google Gemini, with about 28% saying they use chatbots daily, according to survey results released Tuesday by the Pew Research Center, a nonpartisan polling firm. The survey results provide a snapshot of how far AI chatbots have entered mainstream culture, three years after the release of ChatGPT set off a wave of AI investment and marketing by the tech industry. It illustrates the extent to which chatbots have begun to influence teens' daily lives even as the impact of such technology on child development remains unclear. According to the Pew survey, about 4% of teen respondents said they used AI chatbots "almost constantly," while 12% said they used them several times a day and another 12% said they did so about once a day. About 18% said they used AI chatbots several times a week. "It's striking that a majority of teens are using these apps," said Michelle Faverio, a research associate at Pew who worked on the survey. On the other end of the spectrum, about 36% of teen respondents said they do not use AI chatbots at all. The survey was conducted online Sept. 25 to Oct. 9 and asked 1,458 teens ages 13 to 17 about their online habits. The teens were recruited to participate via their parents, according to Pew. Though Pew surveys American teens each year, it was its first time asking broadly about AI chatbot use. AI chatbot use by minors has been especially controversial. Chatbot makers such as OpenAI and Character.AI are facing wrongful-death lawsuits from families who say teens died by suicide with the help of their chatbots. The companies have said they should not be held responsible and that they are trying to prevent similar misuse in the future. In October, two senators said they were introducing legislation to ban companies from providing AI chatbots to minors at all. Previous research has shown that teens are using AI chatbots for schoolwork, emotional support and creative projects including music, among other uses. About 14% of surveyed U.S. adults said in June that they used AI chatbots "very often," according to an NBC News Decision Desk Poll powered by SurveyMonkey, although results from different polls are difficult to compare because of the wording of questions and other factors. AI chatbot usage among teens varied somewhat by race, according to the Pew survey: 35% of Black teens and 33% of Hispanic teens said they used AI chatbots daily, while 22% of white teens said the same. And older teens -- those age 15 to 17 -- were more likely to be daily users than younger teens, the survey found. Boys and girls were equally likely to say in the survey that they had ever used an AI chatbot. ChatGPT was the most-used AI app among teens, with 59% saying they had ever used it. Google Gemini was second at 23%, and Meta AI was third at 20%, according to Pew. The survey also asked teens which other apps and websites they use, and the results were consistent with other survey results in recent years. No. 1 was YouTube, with 92% of respondents saying they used it, followed by TikTok at 68%, Instagram at 63% and Snapchat at 55%. At the back of the pack were Facebook at 31%, WhatsApp at 24%, Reddit at 17% and X at 16%. Despite threats early this year that the U.S. government could ban TikTok, the share of teens who say they are on TikTok almost constantly rose slightly to 21% this year from 16% in 2022, according to Pew.

How to partner AI with human compassion in suicide prevention

Too often, there's a new story about the challenges that people face when relying on artificial intelligence chatbots for advice and support. Tragically, several involve deaths by suicide of teens and young adults who used AI chatbots as "therapists" -- in California, Colorado, Florida and Texas, to name a few. The stories compel us to wrestle with a painful truth: While AI tools are accessible, trusting bots with suicide prevention is a gamble that too many people engage in. More than 1 million active users of ChatGPT, for example, have "conversations that include explicit indicators of potential suicidal planning or intent." In response, a new California law mandates that people who engage in extensive conversations with AI chatbots are reminded every three hours that they are not talking to a real person. Chatbot platform Character.ai took one step further in October, announcing that its services would not be available to anyone under the age of 18. The restrictions improve on the parental controls introduced in September by OpenAI, ChatGPT's parent company. But the problem starts before the first message is sent. The stark reality is that almost 60 percent of youth who die by suicide never saw a mental health professional. Suicide remains the second leading cause of death in young people in the U.S. Access to a firearm triples these risks. Suicidal thoughts are often shrouded in secrecy. Research suggests that more than half of people contemplating suicide tell no one and many fear involuntary hospitalization. Even when young people find their way into a clinic, consistent access to evidence-based care is far from guaranteed Because the U.S. health care system does not adequately serve the needs of many young people with mental health concerns, they are turning to other means of support. AI chatbots have been available for decades, but the technology and interactivity have become more sophisticated and personalized. Some AI "therapists" even have names. As mental health professionals, we get the appeal. Chatbots are always accessible. They don't cost much. They don't judge. Their responses come off as caring and concerned. If you're lying awake at 2 a.m., feeling alone, it can feel easier to type into a chatbox than to wait months for an appointment or risk embarrassment by asking for help in person. AI is quick. It provides the illusion of privacy. It's available when no one else is, and there is no need to obtain parental consent. Suicide risk guidelines and protocols have been programmed into chatbots, such as de-escalation approaches and safety planning. AI can easily pick up on words like "hopeless" or "suicidal." But assessing suicide risk is often more nuanced. Think about a doctor trying to understand chest pain. Sometimes it's indigestion. Sometimes it's a heart attack. You don't want a chatbot making that call. The same goes for assessing risk for suicide. A passing thought of despair isn't the same as someone ready to act, and algorithms can't reliably tell the difference. Suicide prevention isn't just about talking. It's about connection, context and accountability. And those are things that machine learning can't always give you. In suicide prevention, the small human gestures matter most. A clinician noticing a pause before an answer. A friend who insists on staying until morning. A teacher who makes a phone call because something didn't feel right in class. These things save lives. A chatbot can't tilt its head, lean forward, or look you in the eye. We're not saying AI has no place in mental health care. It could help with reminders, track symptoms or give people structured exercises between therapy visits. That's useful. But AI should support care, not replace it. When someone's life is at risk, the only safe path is human connection with trained, accountable professionals. What we need is insurers to cover real, evidence-based treatment. We need more clinicians, especially in rural and underserved communities, trained in best practices for suicide prevention. We need to chip away at stigma so people feel safe asking for help. And we need regulations to make sure AI tools have guardrails in life of death situations. The most recent data available shows that almost 50,000 Americans died by suicide in 2023, so our ability to meet young people where they are, whether it be on social media and ChatGPT, has never been more important. But there is an urgent need for partnerships with expert suicide prevention researchers to understand how to improve identification and understand which users could be harmed by AI chats and under what conditions. AI organizations must not be left to design suicide prevention tools in a vacuum. They need to partner with suicide prevention researchers and clinicians to test their products, understand which users could be harmed and build safeguards grounded in evidence. Without that collaboration, the risks will outweigh the benefits for users, especially youth, at risk for suicide. Holly C. Wilcox, Ph.D., and Paul S. Nestadt, MD, lead the Johns Hopkins Center for Suicide Prevention.