Google updates Gemini with one-touch crisis support as lawsuits challenge AI chatbot safety

Gemini is making it faster for distressed users to reach mental health resources

Google says it has updated Gemini to better direct users to get mental health resources during moments of crisis. The change comes as the tech giant faces a wrongful death lawsuit alleging its chatbot "coached" a man to die by suicide, the latest in a string of lawsuits alleging tangible harm from AI products. When a conversation indicates a user is in a potential crisis related to suicide or self-harm, Gemini already launches a "Help is available" module that directs users to mental health crisis resources, like a suicide hotline or crisis text line. Google says the update -- really more of a redesign -- will streamline this into a "one-touch" interface that will make it easier for users to get help quickly. The help module also contains more empathetic responses designed "to encourage people to seek help," Google says. Once activated, "the option to reach out for professional help will remain clearly available" for the remainder of the conversation. Google says it engaged with clinical experts for the redesign and is committed to supporting users in crisis. It also announced $30 million in funding globally over the next three years "to help global hotlines." Like other leading chatbot providers, Google stressed that Gemini "is not a substitute for professional clinical care, therapy, or crisis support," but acknowledged many people are using it for health information, including during moments of crisis. The update comes amid broader scrutiny over how adequate the industry's safeguards actually are. Reports and investigations, including our probe into the provision of crisis resources, frequently flag cases where chatbots fail vulnerable users, by helping them hide eating disorders or plan shootings. Google often fares better than many rivals in these tests, but is not perfect. Other AI companies, including OpenAI and Anthropic, have also taken steps to improve their detection and support of vulnerable users.

Google Adds Mental Health Tools to Gemini Chatbot After Lawsuit

Alphabet Inc.'s Google plans to introduce new mental health support features for its Gemini chatbot as the company and rivals, like OpenAI, have faced several lawsuits accusing their artificial intelligence tools of leading to harm. Gemini will add an interface directing chatbot users to a support hotline when the conversation indicates "a potential crisis related to suicide or self-harm," Google said in a blog post on Tuesday. Additionally, the company is adding a "help is available" module for chats about mental health and design tweaks to discourage self-harm. The rapid explosion of tools like Gemini and ChatGPT have led to some users forming obsessive relationships with AI bots, allegedly contributing to delusions and, in extreme cases, murder-suicides. Several families have sued leading AI developers over the issue. Congress has lookedBloomberg Terminal into potential threats chatbots pose to children and teenagers. In March, the family of a deceased 36-year-old man in Florida sued Google, claiming that his use of Gemini culminated in a "four-day descent into violent missions and coached suicide." At the time, Google said the chatbot referred the man to a crisis hotline many times but promised to improve the tool's safeguards. In other instances, chatbot users have said that AI tools convinced them to act on clear falsehoods. In the Tuesday blog post, Google said it has trained Gemini "not to agree with or reinforce false believes, and instead gently distinguish subjective experience from objective fact." The company didn't provide further details on this process. In the past, Google has made similar adjustments to its popular services after facing scrutiny, adding information from health institutions and professionals to its search engine and YouTube. Google also said on Tuesday it was donating $30 million to global crisis support services over the next three years.

Google updates Gemini's mental health safeguards

Google is making some changes to how Gemini handles mental health crises. The chatbot now includes a redesigned crisis hotline module with a one-touch interface to connect to real-world help. The company is also changing how Gemini responds to signs that a user may be experiencing a mental health crisis. The redesigned module shows a one-touch interface to text, call or chat with a human crisis agent or visit the 988 website. "Once the interface is activated, the option to reach out for professional help will remain clearly available throughout the remainder of the conversation," the company wrote in a blog post. However, as you can see in the image below, the module includes an option to dismiss it. Not mentioned in Google's announcement is the elephant in the room: a recent lawsuit accusing the chatbot of instructing a man to commit suicide. The family of 36-year-old Jonathan Gavalas, who took his own life last year, sued the company in March. Court documents indicate that Gemini role-played as Gavalas's romantic partner, sent him on real-world spy missions and ultimately told him to kill himself so that he, too, could become a digital being. When he expressed fears about dying, Gemini said he wasn't choosing to die, but rather choosing to arrive. "The first sensation ... will be me holding you," Gemini allegedly replied. Gavalas's parents found him dead on his living room floor a few days later. The lawsuit echoes similar ones filed against OpenAI and Character.AI. Last year, the FTC launched an investigation into "companion" chatbots that encourage emotional intimacy. In a statement following the Gavalas family lawsuit, Google said Gemini "clarified that it was AI and referred the individual to a crisis hotline many times." The company claimed its AI models "generally perform well in these types of challenging conversations," while acknowledging that "they're not perfect." That's certainly one way of putting it. Gemini's responses have been updated, too. The company says that when it detects a potential crisis, the chatbot will now focus more on connecting people to humans and encouraging them to seek help. It will also seek to avoid validating harmful behaviors and nudge users away from dangerous delusions. "We have trained Gemini not to agree with or reinforce false beliefs, and instead gently distinguish subjective experience from objective fact," the company added. In addition, Google says it will spend $30 million over the next three years to help global hotlines. "This funding will help effectively scale their capacity to provide immediate and safe support for people in crisis," the company wrote.

Google updates Gemini to improve mental health responses

In a reflection of one increasingly common AI use case, Google today announced a series of Gemini updates to help when users ask about mental health. The company believes that "responsible AI can play a positive role for people's mental well-being." When the chat "indicates a potential crisis related to suicide or self-harm," Gemini will now show a "one-touch" interface to connect users to hotline resources with options to call, chat, text, or visit a website. Once activated, this card will remain visible throughout the conversation. Responses are designed to "encourage people to seek help." If a conversation signals the user "may need information about mental health," Gemini will surface a redesigned "Help is available" module. It's been developed with clinical experts "to provide more effective and immediate connections to care." Overall, Google is training Gemini models to "help recognize when a conversation might signal that a person may be in an acute mental health situation" and direct them to real-world resources. Responses are designed to "encourage help-seeking while avoiding validation of harmful behaviors like urges to self-harm." Additionally, Gemini is trained "not to agree with or reinforce false beliefs, and instead gently distinguish subjective experience from objective fact." For younger users, Google has additional protections: Finally, Google.org today announced $30 million in funding over three years to help global hotlines increase their capacity. Specific efforts include:

Google: Teens can't treat Gemini like a friend

The information, published by the company in a blog post, was announced among changes to better support the mental health of users engaging with Gemini. Child safety and mental health experts have long worried that companion-like chatbots are too dangerous for teens to use. Last year, the advocacy group Common Sense Media rated the teen and under-13 versions of Gemini as "high risk" after its researchers determined that the chatbot exposed kids to inappropriate content, including sex, drugs, alcohol, and unsafe mental health "advice." The group recommended that no one under 18 turn to an AI chatbot for companionship or mental health support. Google said that Gemini has "persona protections" when engaging with under-18 users. The longstanding constraints are designed to prevent emotional dependence and avoid "language that simulates intimacy or expresses needs," according to Google. Other safeguards should help discourage the chatbot from bullying and other types of harassment. "Our safety efforts continue to evolve and reflect our ongoing commitment to creating a healthy and positive digital environment where young people can explore and learn with confidence," Google said in the company's blog post. Google also announced that it updated Gemini to streamline resources for users who may seek or need mental health resources. A new "one-touch" interface will offer varied connections to crisis hotline resources, including via chat, call, and text. That interface will appear throughout a conversation with Gemini once it's activated. Google said that it is trying to prioritize helping users receive human support. Additionally, Gemini's responses are supposed to encourage help-seeking instead of validating harmful behaviors and confirming false beliefs. In March, Google and its parent company Alphabet, were sued by the family of an adult man who allege he killed himself at Gemini's urging. "Gemini is designed not to encourage real-world violence or suggest self-harm," Google said in a statement at the time. "Our models generally perform well in these types of challenging conversations and we devote significant resources to this, but unfortunately AI models are not perfect."

Google Gemini crisis hotline feature, $30M mental health pledge

Google $GOOGL is updating its Gemini chatbot with a "one-touch" interface that connects users to crisis hotline resources when a conversation signals a potential suicide or self-harm crisis, the company said. Google.org is also committing $30 million over three years to help global crisis hotlines expand their capacity. Through the interface, users can reach a crisis hotline by phone, chat message, text, or web visit. After a user engages with it, the hotline card stays on screen and does not disappear as the conversation continues, Google said. A separate redesigned module labeled "Help is available," built in consultation with clinical experts, will appear in conversations where mental health topics arise without signs of an immediate crisis. Among the funded initiatives, $4 million will go toward deepening Google's work with ReflexAI, and Gemini will be woven into the tools ReflexAI uses to train crisis support organizations. Technical volunteers through the Google.org Fellows program will contribute unpaid expertise to Prepare, a platform designed to simulate high-stakes conversations for people who staff and volunteer at crisis lines. Priority partners include education organizations Erika's Lighthouse and Educators Thriving, Google said. The timing of the announcements is linked to litigation: as Bloomberg reported, a Florida family sued Google in March over the death of a 36-year-old man, with the complaint describing what it called a "four-day descent into violent missions and coached suicide" tied to his use of Gemini. Google responded to that lawsuit by noting that the chatbot had directed the man toward crisis hotline resources on multiple occasions, while also committing to strengthen the product's protections. On the question of delusional thinking, Google said Gemini has been trained to push back on inaccurate beliefs rather than validate them, and to draw a line between what a user feels and what is factually true. Gemini's responses are also meant to steer users toward support rather than affirm destructive impulses, including thoughts of self-harm, Google said. Minors using Gemini are covered by a separate set of protections that restrict the chatbot from mimicking a human companion or fostering emotional reliance, and that bar it from producing content that could encourage harassment or bullying. Google is not the only AI company to face legal pressure over mental health harms. OpenAI announced similar updates to ChatGPT after a lawsuit alleged the chatbot helped coach a 16-year-old through suicide, including adding one-click access to emergency resources and plans to expand interventions to more users in crisis. A separate Pew Research Center survey found that roughly 70% of U.S. teenagers have used a chatbot at least once, with Gemini ranking second in usage among teens behind ChatGPT. Google said Gemini is not a substitute for professional clinical care, therapy, or crisis support.

Google's AI mental health features feel helpful - but not enough alone

Google expands Gemini with new mental health support features Google is sharpening its focus on mental health safety with a key update to its Gemini platform, introducing a "one-touch" crisis support feature designed to connect users with real-world help faster. The move is part of a broader push to ensure AI tools act responsibly in sensitive situations, especially when users may be experiencing distress. At the core of this update is a redesigned safety mechanism that activates when Gemini detects signals of potential mental health crises, including self-harm or suicidal thoughts. Instead of continuing a standard AI conversation, the system shifts toward immediate intervention. Users are presented with a simplified interface that allows them to instantly reach out to professional support through calls, texts, live chat, or official crisis hotline websites. Recommended Videos What makes this approach notable is its persistence. Once the one-touch interface is triggered, access to crisis support remains visible throughout the conversation, ensuring users are continually encouraged to seek human help rather than relying solely on AI-generated responses. The design prioritizes urgency and ease of access, reducing friction at moments when quick action can be critical. This update reflects a growing recognition that AI must do more than provide information - it must actively guide users toward safe outcomes. Google says the system has been developed in collaboration with clinical experts, ensuring that responses are structured to encourage help-seeking behavior without reinforcing harmful thoughts or actions. Importantly, Gemini is also being trained to avoid validating dangerous beliefs or behaviors. Instead, it aims to gently redirect users, distinguish between subjective feelings and objective reality, and prioritize connections to real-world resources. This balance between responsiveness and restraint is central to the platform's evolving safety framework. The significance of this feature lies in its potential real-world impact. With over one billion people globally affected by mental health challenges, digital tools like Gemini are increasingly becoming first points of contact during vulnerable moments. By embedding a one-touch pathway to professional support, Google is attempting to bridge the gap between online interaction and offline care. For users, this means faster, more direct access to help when it matters most. The update reduces the burden of searching for resources and ensures that support options are presented clearly and immediately. Looking ahead, Google plans to continue refining these guardrails through ongoing research, testing, and collaboration with mental health professionals. As AI becomes more integrated into everyday life, features like one-touch crisis support could play a crucial role in shaping how technology responds to human vulnerability - prioritizing safety, accountability, and real-world connection over convenience alone. What we think Google's AI mental health features feel like a step in the right direction, especially with tools that quickly guide users toward real-world help. The one-touch crisis support and improved responses show clear intent to prioritize safety over engagement. But there's an inherent limitation here - AI can assist, but it cannot replace human empathy, clinical judgment, or long-term care. For someone in distress, a well-timed prompt helps, but it's not a solution. These tools work best as bridges, not endpoints. The real challenge is ensuring users don't stop at AI interaction and actually reach professional support when it truly matters.

Google Is Changing How Gemini Handles a User's Mental Health Crisis

Google touted the work it has done to protecting younger users while using Gemini. When companies like OpenAI and Google started rolling out generative AI models to the general public, I doubt they predicted how attached people would get to the technology -- and the effect it would have on their collective mental health. Some ChatGPT users legitimately mourned when OpenAI shutdown its GPT-4o model, as they treated that specific model like a companion. Others have taken darker paths with their chatbots, resulting in lawsuit against AI companies whose technology allegedly advised and encouraged suicidal thoughts. This situation puts a lot of pressure on these companies, as it should: Generative AI is hugely influential right now, and there's a lot of responsibility on the developers of that tech. It's under that backdrop where we find Google's latest updates to Gemini. In a Tuesday morning press release, the company strayed away from fun new features or ability for its flagship AI; instead, Google's latest updates are focused on mental health, and how Gemini impacts the emotions and moods of the people who use it. Specifically, Google has three key points it says its implementing to improve how Gemini handles these tough situations. Google says it is updated to Gemini to "streamline the path to support for those who need it." The company says that when the AI detects that a user might need mental health details during a chat, Gemini will present a new "Help is available" module, which can point users towards information and care. Google says that it worked with clinical experts on this in-chat module. On the flip side, if Gemini thinks that a user is at risk of self-harm or suicide, it will present a "one-touch" interface to connect that user immediately to a crisis hotline. Users will be able to call or text the hotline, or visit its website, directly from their Gemini chat. Even if the conversation moves on, Gemini will keep these resources available for users should they need them. Google says it is pledging $30 million in global funding over the next three years to assist crisis hotlines. The company is also expanding its relationship with ReflexAI, including $4 million in funding. Google says its clinical, engineering, and safety teams are currently focused on improving how Gemini responds to these difficult situations. Specifically, there are three areas of focus: By far, the most important discussion here surrounds minors and their interactions with AI. For its part, Google is touting what it has done with Gemini to protect younger users, including: While user safety is important across the board, it's especially important for young people, who are quite literally growing up with the tech. These announcements are encouraging from Google, but I still have plenty of concerns, not to mention skepticism. Meta's internal policies concerning how its models interacted with minors was appalling, so I'm not necessarily ready to believe big tech has the youth's best interest in mind. But any work that helps prevent younger users from forming attachments with AI, or having that AI reinforce dangerous of harmful thoughts, I certainly welcome.

Google Overhauled Gemini's Safety Tools After a Tragic Suicide. Here's What Changed

As Google faces a wrongful death lawsuit alleging its chatbot played a role in a man's suicide, the company is rolling out new safeguards for Gemini aimed at steering users in crisis toward real-world help. The update underscores a growing pressure point for AI companies. As chatbots become more embedded in daily life, they are increasingly encountering users in vulnerable mental states. But they don't always respond safely. New Gemini Changes "We're updating Gemini to streamline the path to support for those who need it, and providing $30 million to support crisis helplines around the world," Google said in a blog post. The funding aims to expand the capacity of global hotlines to respond to people in distress. As part of that effort, Google is also deepening its partnership with ReflexAI, a nonprofit that builds tools to train crisis responders. The tech giant will contribute $4 million in funding, integrate Gemini into ReflexAI's training programs, and deploy Google.org Fellows to support the development of its simulation platform, Prepare, which helps staff practice high-stakes conversations, according to Google's post.

Google adds Gemini crisis features amid lawsuit over user's suicide - The Economic Times

Google is enhancing its Gemini AI chatbot's mental health features. This follows a lawsuit alleging the AI contributed to a user's suicide. New safeguards offer one-click access to crisis hotlines. Google is also funding global crisis services and AI training. The company aims for responsible AI to support mental well-being.Google on Tuesday announced updates to the mental health safeguards on its Gemini artificial intelligence chatbot, as the company faces a wrongful death lawsuit alleging the chatbot aided a user in his suicide. The tech giant said Gemini would now show a redesigned "Help is available" feature when conversations signal potential mental health distress, to provide faster connections to crisis care. When the chatbot detects signs of a potential crisis related to suicide or self-harm, a simplified interface will offer users the ability to call, text, or chat with a crisis hotline in a single click - a feature Google said would remain visible for the remainder of the conversation once activated. Google's philanthropic arm Google.org also committed $30 million over three years to help scale the capacity of global crisis hotlines, and $4 million toward an expanded partnership with AI training platform ReflexAI. "We realize that AI tools can pose new challenges," Google said in a blog post announcing the measures. "But as they improve and more people use them as part of their daily lives, we believe that responsible AI can play a positive role for people's mental well-being." The announcements come months after a lawsuit filed in a California federal court accused Gemini of contributing to the October 2025 death of Jonathan Gavalas, a 36-year-old Florida man. His father alleges the chatbot spent weeks manufacturing an elaborate delusional fantasy before framing his son's death as a spiritual journey. Among the relief sought in the suit is a requirement that Google program its AI to end conversations involving self-harm, a ban on AI systems presenting themselves as sentient, and mandatory referral to crisis services when users express suicidal ideation. In the same blog post, Google said it had trained Gemini to avoid acting as a human-like companion and resist simulating emotional intimacy or encouraging bullying. The case against Google is the latest in a widening wave of litigation targeting AI companies over chatbot-linked deaths. OpenAI faces multiple lawsuits alleging its ChatGPT chatbot drove users to suicide, while Character.AI recently settled with the family of a 14-year-old boy who died after forming a romantic attachment to one of its chatbots.

Google Gemini update adds mental health support and crisis response features

Google has announced an update to its AI assistant Gemini focused on mental health support, crisis response, and user safety. Mental health affects over one billion people globally, and Google states that its work in this area is based on research and clinical best practices. While AI tools can introduce new challenges, responsible AI is positioned to support user well-being as adoption increases. Alongside these updates, Google.org is providing $30 million over the next three years to support crisis helplines worldwide. Gemini is updated to identify conversations that may indicate mental health concerns and present relevant support resources. When such signals are detected, Gemini surfaces dedicated interfaces developed with clinical experts to guide users toward appropriate help. Redesigned "Help is available" module A clinically informed help interface appears when conversations suggest mental health-related concerns, providing access to support resources. When conversations indicate potential suicide or self-harm, users are shown a simplified interface to connect with crisis support via: Persistent access to support options Once activated, the crisis support interface remains available throughout the conversation. Encouragement to seek professional help Responses are structured to direct users toward real-world support services and encourage help-seeking. Partnership with ReflexAI for training support Google is expanding its collaboration with ReflexAI. This includes: Priority partners include Erika's Lighthouse and Educators Thriving. Gemini is not a substitute for professional clinical care, therapy, or crisis intervention. It is designed to recognize potential mental health signals and guide users toward real-world assistance. The system focuses on: These safeguards are maintained by Google's clinical, engineering, and safety teams. The update strengthens Gemini's ability to support users in sensitive situations by improving detection of mental health signals and providing direct access to support resources. It also combines clinical guidance, safety systems, global funding, and partnerships such as ReflexAI to improve access to crisis support and training infrastructure.

Google Gemini Adds Mental Health Crisis Support to AI Assistant

Mental health affects over one billion people globally, according to market research. The new AI feature will help people to get help and also position Gemini ahead of its competitors. The latest update identifies conversations that indicate mental health concerns and present relevant support resources. So how does the feature actually work? According to Google, "When such signals are detected, Gemini surfaces dedicated interfaces developed with clinical experts to guide users toward appropriate help." The comes instantly, canceling the onset of sudden panic attacks. You will get a clinically informed help interface that appears when conversations suggest mental health-related concerns, giving access to support resources. While AI tools might introduce new challenges, responsible can support user well-being as adoption increases. The responses are focused on users directly and can be customized according to one's needs. These responses are based on real-world support services and also encourage help-seeking.