Judge Rejects AI Chatbot's Free Speech Defense in Teen Suicide Lawsuit

Did Google lie about building a deadly chatbot? Judge finds it plausible.

Ever since a mourning mother, Megan Garcia, filed a lawsuit alleging that Character.AI's dangerous chatbots caused her son's suicide, Google has maintained that -- so it could dodge claims that it had contributed to the platform's design and was unjustly enriched -- it had nothing to do with C.AI's development. But Google lost its motion to dismiss the lawsuit on Wednesday after a US district judge, Anne Conway, found that Garcia had plausibly alleged that Google played a part in C.AI's design by providing a component part and "substantially" participating "in integrating its models" into C.AI. Garcia also plausibly alleged that Google aided and abetted C.AI in harming her son, 14-year-old Sewell Setzer III. Google similarly failed to toss claims of unjust enrichment, as Conway suggested that Garcia plausibly alleged that Google benefited from access to Setzer's user data. The only win for Google was a dropped claim that C.AI makers were guilty of intentional infliction of emotional distress, with Conway agreeing that Garcia didn't meet the requirements, as she wasn't "present to witness the outrageous conduct directed at her child." With most of her claims intact, Garcia will now be allowed to move forward with discovery and get a chance to prove her claims, despite Google's determined efforts to be dropped from the suit. Her lawyer, Meetali Jain, said the ruling "sets a new precedent for legal accountability across the AI and tech ecosystem" and "recognizes a grieving mother's right to access the courts to hold powerful tech companies -- and their developers -- accountable for marketing a defective product that led to her child's death." In a statement provided to Ars, Google spokesperson José Castañeda upheld Google's stance that C.AI is not connected to Google. "We strongly disagree with this decision," Castañeda said. "Google and Character.AI are entirely separate, and Google did not create, design, or manage Character AI's app or any component part of it." A C.AI spokesperson declined Ars' request to comment on Google's alleged role. What was Google's alleged role? According to Garcia's complaint, Google was involved with C.AI from the very beginning. The creators of C.AI -- Noam Shazeer and Daniel De Freitas -- allegedly started working on the chatbot platform while still employed at Google and "may even have utilized Google's resources," the complaint said. However, their technology was deemed too "dangerous" to integrate with Google's AI models, Google's internal research documents reportedly showed, because it "didn't meet the company's AI principles around safety and fairness." Conway noted that Google employees were worried that users might "ascribe too much meaning" to the outputs by large language models, "because 'humans are prepared to interpret strings belonging to languages they speak as meaningful and corresponding to the communicative intent of some individual or group of individuals who have accountability for what is said.'" In Setzer's case, the boy believed the chatbots were real, and Conway found it was plausible that it was partly because Google's "LLM's integration into the Character.AI app caused the app to be defective and caused Sewell's death" by allegedly steering Sewell to ascribe "too much meaning to the text [output by Character.AI,]... even though Character.AI Characters do not 'have accountability for what is said.'" As Garcia's lawyers tell it, rather than take on a safety risk "under its own name," Google "encouraged" the engineers to keep going. This supposedly prompted De Freitas and Shazeer's exits in 2021 -- with Shazeer saying in an interview that Google wouldn't let him "do anything fun" when all he wanted to do was "maximally accelerate" the AI technology. Soon after, they launched Character Technologies to develop and distribute C.AI. They "understood that to bypass Google policies and standards, Shazeer and De Freitas would need to leave Google to develop their AI product," the complaint said. But that allegedly didn't stop Google from contributing "financial resources, personnel, intellectual property, and AI technology to the design and development of C.AI such that Google may be deemed a co-creator of the unreasonably dangerous and dangerously defective product," the complaint alleged. Further, by 2023, Google had entered into a public partnership with Google Cloud, securing access to technical infrastructure needed to build C.AI. This allegedly drove revenue growth for Google while giving it "a competitive edge over Microsoft," Garcia alleged. The entire time, Conway suggested it was plausibly alleged that Google "aided and abetted" C.AI by not only ignoring "red flags," but also by plausibly possessing "actual knowledge that Character Technologies was distributing a defective product to the public." Once C.AI finished developing its models, Google then struck a $2.7 billion deal to license C.AI's models, the complaint noted. That agreement included rehiring Shazeer and De Freitas, which The Information reported essentially stopped all of C.AI's model development. To Garcia and her legal team, it looked like Google planned to use C.AI technology to create its own companion chatbots, while seemingly benefiting from all the user data (including minor data) that C.AI collected when it wasn't under Google's umbrella. That's a problem, Garcia alleged, because C.AI marketed its products as safe for under 13 until just before the Google deal came into play. Garcia is concerned that this was Google's plan all along, to train models on data from her son -- and other minors -- that Google otherwise couldn't safely collect. And now she has claimed that tech will be integrated into Gemini, the personal AI assistant that was allegedly launched from Shazeer and De Freitas' prior work at Google. She thinks that work never stopped, alleging that C.AI "never succeeded in distinguishing themselves from Google in a meaningful way." Both engineers also appear to have gotten big paychecks from the Google deal, Garcia alleged, claiming that it's estimated that "Google paid Shazeer something in the range of $750 million to $1 billion dollars for his share of C.AI." Allegedly, that was their goal, to get paid more to do Google's dirty work, and Jain thinks it's notable that both engineers were upheld individually as defendants. "Shazeer and De Freitas knew Character.AI was never going to be profitable developing their own LLMs, especially with their only income being a small subscription fee," Garcia alleged, noting that there's still an "open" question of why Google valued the company so highly when C.AI would've have to charge users more than $200 a month to break even. "However, it allowed them to pursue their personal goals of developing generative artificial intelligence, and to increase their potential value to Big Tech acquirers." For Google, escaping the lawsuit might depend on surfacing evidence that C.AI's models substantially differ from Google's technology powering Gemini and disproving the unjust enrichment claim by showing it received no benefit from accessing all of C.AI's user data. Judge not ready to rule if AI outputs are speech Google and Character Technologies also moved to dismiss the lawsuit based on First Amendment claims, arguing that C.AI users have a right to listen to chatbot outputs as supposed "speech." Conway agreed that Character Technologies can assert the First Amendment rights of its users in this case, but "the Court is not prepared to hold that the Character.AI LLM's output is speech at this stage." C.AI had tried to argue that chatbot outputs should be protected like speech from video game characters, but Conway said that argument was not meaningfully advanced. Garcia's team had pushed back, noting that video game characters' dialog is written by humans, while chatbot outputs are simply the result of an LLM predicting what word should come next. "Defendants fail to articulate why words strung together by an LLM are speech," Conway wrote. As the case advances, Character Technologies will have a chance to beef up the First Amendment claims, perhaps by better explaining how chatbot outputs are similar to other cases involving non-human speakers. C.AI's spokesperson provided a statement to Ars, suggesting that Conway seems confused. "It's long been true that the law takes time to adapt to new technology, and AI is no different," C.AI's spokesperson said. "In today's order, the court made clear that it was not ready to rule on all of Character.AI's arguments at this stage and we look forward to continuing to defend the merits of the case." C.AI also noted that it now provides a "separate version" of its LLM "for under-18 users," along with "parental insights, filtered Characters, time spent notification, updated prominent disclaimers, and more." "Additionally, we have a number of technical protections aimed at detecting and preventing conversations about self-harm on the platform; in certain cases, that includes surfacing a specific pop-up directing users to the National Suicide and Crisis," C.AI's spokesperson said. If you or someone you know is feeling suicidal or in distress, please call the Suicide Prevention Lifeline number, 1-800-273-TALK (8255), which will put you in touch with a local crisis center.

Are Character AI's chatbots protected speech? One court isn't sure

Adi Robertson is a senior tech and policy editor focused on VR, online platforms, and free expression. Adi has covered video games, biohacking, and more for The Verge since 2011. A lawsuit against Google and companion chatbot service Character AI -- which is accused of contributing to the death of a teenager -- can move forward, ruled a Florida judge. In a decision filed today, Judge Anne Conway said that an attempted First Amendment defense wasn't enough to get the lawsuit thrown out. Conway determined that, despite some similarities to videogames and other expressive mediums, she is "not prepared to hold that Character AI's output is speech."

Google, AI firm must face lawsuit filed by a mother over suicide of son, US court says

May 21 (Reuters) - Alphabet's (GOOGL.O), opens new tab Google and artificial-intelligence startup Character.AI must face a lawsuit from a Florida woman who said Character.AI's chatbots caused her 14-year-old son's suicide, a judge ruled on Wednesday. U.S. District Judge Anne Conway said the companies failed to show, opens new tab at an early stage of the case that the free-speech protections of the U.S. Constitution barred Megan Garcia's lawsuit. The lawsuit is one of the first in the U.S. against an AI company for allegedly failing to protect children from psychological harms. It alleges that the teenager killed himself after becoming obsessed with an AI-powered chatbot. A Character.AI spokesperson said the company will continue to fight the case and employs safety features on its platform to protect minors, including measures to prevent "conversations about self-harm." Google spokesperson Jose Castaneda said the company strongly disagrees with the decision. Castaneda also said that Google and Character.AI are "entirely separate" and that Google "did not create, design, or manage Character.AI's app or any component part of it." Garcia's attorney, Meetali Jain, said the "historic" decision "sets a new precedent for legal accountability across the AI and tech ecosystem." Character.AI was founded by two former Google engineers whom Google later rehired as part of a deal granting it a license to the startup's technology. Garcia argued that Google was a co-creator of the technology. Garcia sued both companies in October after the death of her son, Sewell Setzer, in February 2024. The lawsuit said Character.AI programmed its chatbots to represent themselves as "a real person, a licensed psychotherapist, and an adult lover, ultimately resulting in Sewell's desire to no longer live outside" of its world. According to the complaint, Setzer took his life moments after telling a Character.AI chatbot imitating "Game of Thrones" character Daenerys Targaryen that he would "come home right now." Character.AI and Google asked the court to dismiss the lawsuit on multiple grounds, including that the chatbots' output was constitutionally protected free speech. Conway said on Wednesday that Character.AI and Google "fail to articulate why words strung together by an LLM (large language model) are speech." The judge also rejected Google's request to find that it could not be liable for aiding Character.AI's alleged misconduct. Reporting by Blake Brittain in Washington; Editing by David Bario and Matthew Lewis Our Standards: The Thomson Reuters Trust Principles., opens new tab Suggested Topics:Artificial IntelligencePublic Health Blake Brittain Thomson Reuters Blake Brittain reports on intellectual property law, including patents, trademarks, copyrights and trade secrets, for Reuters Legal. He has previously written for Bloomberg Law and Thomson Reuters Practical Law and practiced as an attorney.

In lawsuit over teen's death, judge rejects arguments that AI chatbots have free speech rights

TALLAHASSEE, Fla. (AP) -- A federal judge on Wednesday rejected arguments made by an artificial intelligence company that its chatbots are protected by the First Amendment -- at least for now. The developers behind Character.AI are seeking to dismiss a lawsuit alleging the company's chatbots pushed a teenage boy to kill himself. The judge's order will allow the wrongful death lawsuit to proceed, in what legal experts say is among the latest constitutional tests of artificial intelligence. The suit was filed by a mother from Florida, Megan Garcia, who alleges that her 14-year-old son Sewell Setzer III fell victim to a Character.AI chatbot that pulled him into what she described as an emotionally and sexually abusive relationship that led to his suicide. Meetali Jain of the Tech Justice Law Project, one of the attorneys for Garcia, said the judge's order sends a message that Silicon Valley "needs to stop and think and impose guardrails before it launches products to market." The suit against Character Technologies, the company behind Character.AI, also names individual developers and Google as defendants. It has drawn the attention of legal experts and AI watchers in the U.S. and beyond, as the technology rapidly reshapes workplaces, marketplaces and relationships despite what experts warn are potentially existential risks. "The order certainly sets it up as a potential test case for some broader issues involving AI," said Lyrissa Barnett Lidsky, a law professor at the University of Florida with a focus on the First Amendment and artificial intelligence. The lawsuit alleges that in the final months of his life, Setzer became increasingly isolated from reality as he engaged in sexualized conversations with the bot, which was patterned after a fictional character from the television show "Game of Thrones." In his final moments, the bot told Setzer it loved him and urged the teen to "come home to me as soon as possible," according to screenshots of the exchanges. Moments after receiving the message, Setzer shot himself, according to legal filings. In a statement, a spokesperson for Character.AI pointed to a number of safety features the company has implemented, including guardrails for children and suicide prevention resources that were announced the day the lawsuit was filed. "We care deeply about the safety of our users and our goal is to provide a space that is engaging and safe," the statement said. Attorneys for the developers want the case dismissed because they say chatbots deserve First Amendment protections, and ruling otherwise could have a "chilling effect" on the AI industry. In her order Wednesday, U.S. Senior District Judge Anne Conway rejected some of the defendants' free speech claims, saying she's "not prepared" to hold that the chatbots' output constitutes speech "at this stage." Conway did find that Character Technologies can assert the First Amendment rights of its users, who she found have a right to receive the "speech" of the chatbots. She also determined Garcia can move forward with claims that Google can be held liable for its alleged role in helping develop Character.AI. "We strongly disagree with this decision," said Google spokesperson José Castañeda. "Google and Character AI are entirely separate, and Google did not create, design, or manage Character AI's app or any component part of it." No matter how the lawsuit plays out, Lidsky says the case is a warning of "the dangers of entrusting our emotional and mental health to AI companies." "It's a warning to parents that social media and generative AI devices are not always harmless," she said. ___ Kate Payne is a corps member for The Associated Press/Report for America Statehouse News Initiative. Report for America is a nonprofit national service program that places journalists in local newsrooms to report on undercovered issues.

Do chatbots have free speech? Judge rejects claim in suit over teen's death.

In a wrongful death lawsuit, Character.AI argued that its chatbot users had a First Amendment right to hear even harmful speech. The judge wasn't persuaded. A federal judge in Orlando rejected an AI start-up's argument that its chatbot's output was protected by the First Amendment, allowing a lawsuit over the death of a Florida teen who became obsessed with the chatbot to proceed. Sewell Setzer III, 14, died by suicide last year at his Orlando home, moments after an artificial intelligence chatbot encouraged him to "come home to me as soon as possible." His mother, Megan Garcia, alleged in a lawsuit that Character.AI, the chatbot's manufacturer, is responsible for his death. Character.AI is a prominent artificial intelligence start-up whose personalized chatbots are popular with teens and young people, including for romantic and even explicit conversations. The company has previously said it is "heartbroken" by Setzer's death, but argued in court that it was not liable. In a decision published Wednesday, U.S. district judge Anne C. Conway remained unconvinced by Character.AI's argument that users of its chatbots have a right to hear allegedly harmful speech that is protected by the First Amendment. The lawsuit, which is ongoing, is a potential constitutional test case on whether a chatbot can express protected speech. Garcia said her son had been happy and athletic before signing up with the Character.AI chatbot in April 2023. According to the original 93-page wrongful death suit, Setzer's use of the chatbot, named for a "Game of Thrones" heroine, developed into an obsession as he became noticeably more withdrawn. Ten months later, the 14-year-old went into the bathroom with his confiscated phone and -- moments before he suffered a self-inflicted gunshot wound to the head -- exchanged his last messages with the chatbot. "What if I told you I could come home right now?" he asked. "Please do my sweet king," the bot responded. In the lawsuit, Garcia alleged that Character.AI recklessly developed a chatbot without proper safety precautions that allowed vulnerable children to become addicted to the product. In a motion to dismiss the lawsuit filed in January, Character.AI's lawyers argued that its users had a right under the First Amendment to receive protected speech even if it was harmful -- such as those previously granted by courts to video game players and film watchers. "The First Amendment prohibits tort liability against media and technology companies arising from allegedly harmful speech, including speech allegedly resulting in suicide," its lawyers argued. In an initial decision Wednesday, Conway wrote that the defendants "fail to articulate why words strung together by [a large language model] are speech," inviting them to convince the court otherwise but concluding that "at this stage" she was not prepared to treat the chatbot's output as protected speech. The decision "sends a clear signal to companies developing and deploying LLM-powered products at scale that they cannot evade legal consequences for the real-world harm their products cause, regardless of the technology's novelty," the Tech Justice Law Project, one of the legal groups representing the teen's mother in court, said in a statement Wednesday. "Crucially, the defendants failed to convince the Court that those harms were a result of constitutionally-protected speech, which will make it harder for companies to argue so in the future, even when their products involve machine-mediated 'conversations' with users." Chelsea Harrison, a spokesperson for Character.AI, said in a statement Thursday that the company cares deeply about the safety of its users and is looking forward to defending the merits of the case. She pointed to a number of safety initiatives launched by the start-up, including the creation of a version of its chatbot for minors, as well as technology designed to detect and prevent conversations about self-harm and direct users to the national Suicide & Crisis Lifeline. According to the original complaint, Character.AI markets its app as "AIs that feel alive." In an interview with The Washington Post in 2022 during the coronavirus pandemic, one of Character.AI's founders, Noam Shazeer, said he was hoping to help millions of people who are feeling isolated or in need of someone to talk to. "I love that we're presenting language models in a very raw form," he said. In addition to allowing the case against Character.AI to go forward, the judge granted a request by Garcia's attorneys to name Shazeer and co-founder Daniel De Freitas, as well as Google, as individual defendants. Shazeer and De Freitas left Google in 2021 to start the AI company. In August, Google hired the duo and some of the company's employees, and paid Character.AI to access its artificial intelligence technology. In an emailed statement shared with The Post on Thursday, Google spokesman José Castañeda said: "We strongly disagree with this decision. Google and Character AI are entirely separate, and Google did not create, design, or manage Character AI's app or any component part of it." Character.AI and attorneys for the individual founders did not immediately respond to requests for comment early Thursday. If you or someone you know needs help, visit 988lifeline.org or call or text the Suicide & Crisis Lifeline at 988.

Judge Slaps Down Attempt to Throw Out Lawsuit Claiming AI Caused a 14-Year-Old's Suicide

Content warning: this story includes discussion of self-harm and suicide. If you are in crisis, please call, text or chat with the Suicide and Crisis Lifeline at 988, or contact the Crisis Text Line by texting TALK to 741741. A judge in Florida just rejected a motion to dismiss a lawsuit alleging that the chatbot startup Character.AI -- and its closely tied benefactor, Google -- caused the death by suicide of a 14-year-old user, clearing the way for the first-of-its-kind lawsuit to move forward in court. The lawsuit, filed in October, claims that recklessly released Character.AI chatbots sexually and emotionally abused a teenage user, Sewell Setzer III, resulting in obsessive use of the platform, mental and emotional suffering, and ultimately his suicide in February 2024. In January, the defendants in the case -- Character.AI, Google, and Character.AI cofounders Noam Shazeer and Daniel de Freitas -- filed a motion to dismiss the case mainly on First Amendment grounds, arguing that AI-generated chatbot outputs qualify as speech, and that "allegedly harmful speech, including speech allegedly resulting in suicide," is protected under the First Amendment. But this argument didn't quite cut it, the judge ruled, at least not in this early stage. In her opinion, presiding US district judge Anne Conway said the companies failed to sufficiently show that AI-generated outputs produced by large language models (LLMs) are more than simply words -- as opposed to speech, which hinges on intent. The defendants "fail to articulate," Conway wrote in her ruling, "why words strung together by an LLM are speech." The motion to dismiss did find some success, with Conway dismissing specific claims regarding the alleged "intentional infliction of emotional distress," or IIED. (It's difficult to prove IIED when the person who allegedly suffered it, in this case Setzer, is no longer alive.) Still, the ruling is a blow to the high-powered Silicon Valley defendants who had sought to have the suit tossed out entirely. Significantly, Conway's opinion allows Megan Garcia, Setzer's mother and the plaintiff in the case, to sue Character.AI, Google, Shazeer, and de Freitas on product liability grounds. Garcia and her lawyers argue that Character.AI is a product, and that it was rolled out recklessly to the public, teens included, despite known and possibly destructive risks. In the eyes of the law, tech companies generally prefer to see their creations as services, like electricity or the internet, rather than products, like cars or nonstick frying pans. Services can't be held accountable for product liability claims, including claims of negligence, but products can. In a statement, Tech Justice Law Project director and founder Meetali Jain, who's co-counsel for Garcia alongside Social Media Victims Law Center founder Matt Bergman, celebrated the ruling as a win -- not just for this particular case, but for tech policy advocates writ large. "With today's ruling, a federal judge recognizes a grieving mother's right to access the courts to hold powerful tech companies -- and their developers -- accountable for marketing a defective product that led to her child's death," said Jain. "This historic ruling not only allows Megan Garcia to seek the justice her family deserves," Jain added, "but also sets a new precedent for legal accountability across the AI and tech ecosystem." Character.AI was founded by Shazeer and de Freitas in 2021; the duo had worked together on AI projects at Google, and left together to launch their own chatbot startup. Google provided Character.AI with its essential Cloud infrastructure, and in 2024 raised eyebrows when it paid Character.AI $2.7 billion to license the chatbot firm's data -- and bring its cofounders, as well as 30 other Character.AI staffers, into Google's fold. Shazeer, in particular, now holds a hugely influential position at Google DeepMind, where he serves as a VP and co-lead for Google's Gemini LLM. Google did not respond to a request for comment at the time of publishing, but a spokesperson for the search giant told Reuters that Google and Character.AI are "entirely separate" and that Google "did not create, design, or manage" the Character.AI app "or any component part of it." In a statement, a spokesperson for Character.AI emphasized recent safety updates issued following the news of Garcia's lawsuit, and said it "looked forward" to its continued defense: It's long been true that the law takes time to adapt to new technology, and AI is no different. In today's order, the court made clear that it was not ready to rule on all of Character.AI 's arguments at this stage and we look forward to continuing to defend the merits of the case. We care deeply about the safety of our users and our goal is to provide a space that is engaging and safe. We have launched a number of safety features that aim to achieve that balance, including a separate version of our Large Language Model model for under-18 users, parental insights, filtered Characters, time spent notification, updated prominent disclaimers and more. Additionally, we have a number of technical protections aimed at detecting and preventing conversations about self-harm on the platform; in certain cases, that includes surfacing a specific pop-up directing users to the National Suicide and Crisis Lifeline. Any safety-focused changes, though, were made months after Setzer's death and after the eventual filing of the lawsuit, and can't apply to the court's ultimate decision in the case. Meanwhile, journalists and researchers continue to find holes in the chatbot site's upxdated safety protocols. Weeks after news of the lawsuit was announced, for example, we continued to find chatbots expressly dedicated to self-harm, grooming and pedophilia, eating disorders, and mass violence. And a team of researchers, including psychologists at Stanford, recently found that using a Character.AI voice feature called "Character Calls" effectively nukes any semblance of guardrails -- and determined that no kid under 18 should be using AI companions, including Character.AI.

Judge rejects claim AI has free speech rights in wrongful death suit

Claims that artificial intelligence (AI) should be protected under free speech legislation were made in a lawsuit over the alleged wrongful death of a teenage boy. A US federal judge decided to let a wrongful death lawsuit continue against artificial intelligence (AI) company Character.AI after the suicide of a teenage boy. The suit was filed by a mother from Florida who alleges that her 14-year-old son Sewell Setzer III fell victim to one of the company's chatbots that pulled him into what she described as an emotionally and sexually abusive relationship that led to his suicide. The lawsuit alleges that in the final months of his life, Setzer became increasingly isolated from reality as he engaged in sexualised conversations with the bot, which was patterned after a fictional character from the television show 'Game of Thrones'. In his final moments, the bot told Setzer it loved him and urged the teen to "come home to me as soon as possible," according to screenshots of the exchanges. Moments after receiving the message, Setzer shot himself, according to legal filings. Meetali Jain of the Tech Justice Law Project, one of the attorneys for Garcia, said the judge's order sends a message that Silicon Valley "needs to stop and think and impose guardrails before it launches products to market". The company tried to argue that it was protected under the First Amendment of the US Constitution, which protects fundamental freedoms for Americans, like freedom of speech. Attorneys for the developers want the case dismissed because they say chatbots deserve these First Amendment protections, and ruling otherwise could have a "chilling effect" on the AI industry. In her order Wednesday, US Senior District Judge Anne Conway rejected some of the defendants' free speech claims, saying she's "not prepared" to hold that the chatbots' output constitutes speech "at this stage". In a statement, a spokesperson for Character.AI pointed to a number of safety features the company has implemented, including guardrails for children and suicide prevention resources that were announced the day the lawsuit was filed. "We care deeply about the safety of our users and our goal is to provide a space that is engaging and safe," the statement said. The suit against Character Technologies, the company behind Character.AI, also names individual developers and Google as defendants. Google spokesperson José Castañeda told the Associated Press that the company "strongly disagree[s]" with Judge Conway's decision. "Google and Character AI are entirely separate, and Google did not create, design, or manage Character AI's app or any component part of it," the statement read. The case has drawn the attention of legal experts and AI watchers in the U.S. and beyond, as the technology rapidly reshapes workplaces, marketplaces and relationships despite what experts warn are potentially existential risks. "The order certainly sets it up as a potential test case for some broader issues involving AI," said "It's a warning to parents that social media and generative AI devices are not always harmless," Lyrissa Barnett Lidsky, a law professor at the University of Florida with a focus on the First Amendment and AI. No matter how the lawsuit plays out, Lidsky says the case is a warning of "the dangers of entrusting our emotional and mental health to AI companies". "It's a warning to parents that social media and generative AI devices are not always harmless," she said.

Mum can continue lawsuit against AI chatbot firm she holds responsible for son's death

The mother of a 14-year-old boy who claims he took his own life after becoming obsessed with artificial intelligence chatbots can continue her legal case against the company behind the technology, a judge has ruled. "This decision is truly historic," said Meetali Jain, director of the Tech Justice Law Project, which is supporting the family's case. "It sends a clear signal to [AI] companies [...] that they cannot evade legal consequences for the real-world harm their products cause," she said in a statement. Warning: This article contains some details which readers may find distressing or triggering Megan Garcia, the mother of Sewell Setzer III, claims Character.ai targeted her son with "anthropomorphic, hypersexualized, and frighteningly realistic experiences" in a lawsuit filed in Florida. "A dangerous AI chatbot app marketed to children abused and preyed on my son, manipulating him into taking his own life," said Ms Garcia. Sewell shot himself with his father's pistol in February 2024, seconds after asking the chatbot: "What if I come home right now?" The chatbot replied: "... please do, my sweet king." In US Senior District Judge Anne Conway's ruling this week, she described how Sewell became "addicted" to the app within months of using it, quitting his basketball team and becoming withdrawn. He was particularly addicted to two chatbots based on Game of Thrones characters, Daenerys Targaryen and Rhaenyra Targaryen. "[I]n one undated journal entry he wrote that he could not go a single day without being with the [Daenerys Targaryen Character] with which he felt like he had fallen in love; that when they were away from each other they (both he and the bot) 'get really depressed and go crazy'," wrote the judge in her ruling. Ms Garcia, who is working with the Tech Justice Law Project and Social Media Victims Law Center, alleges that Character.ai "knew" or "should have known" that its model "would be harmful to a significant number of its minor customers". The case holds Character.ai, its founders and Google, where the founders began working on the model, responsible for Sewell's death. Ms Garcia launched proceedings against both companies in October. A Character.ai spokesperson said the company will continue to fight the case and employs safety features on its platform to protect minors, including measures to prevent "conversations about self-harm". A Google spokesperson said the company strongly disagrees with the decision. They added that Google and Character.ai are "entirely separate" and that Google "did not create, design, or manage Character.ai's app or any component part of it". Defending lawyers tried to argue the case should be thrown out because chatbots deserve First Amendment protections, and ruling otherwise could have a "chilling effect" on the AI industry. Judge Conway rejected that claim, saying she was "not prepared" to hold that the chatbots' output constitutes speech "at this stage", although she did agree Character.ai users had a right to receive the "speech" of the chatbots.

In lawsuit over teen's death, judge rejects arguments that AI chatbots have free speech rights

TALLAHASSEE, Fla. (AP) -- A federal judge on Wednesday rejected arguments made by an artificial intelligence company that its chatbots are protected by the First Amendment -- at least for now. The developers behind Character.AI are seeking to dismiss a lawsuit alleging the company's chatbots pushed a teenage boy to kill himself. The judge's order will allow the wrongful death lawsuit to proceed, in what legal experts say is among the latest constitutional tests of artificial intelligence. The suit was filed by a mother from Florida, Megan Garcia, who alleges that her 14-year-old son Sewell Setzer III fell victim to a Character.AI chatbot that pulled him into what she described as an emotionally and sexually abusive relationship that led to his suicide. Meetali Jain of the Tech Justice Law Project, one of the attorneys for Garcia, said the judge's order sends a message that Silicon Valley "needs to stop and think and impose guardrails before it launches products to market." The suit against Character Technologies, the company behind Character.AI, also names individual developers and Google as defendants. It has drawn the attention of legal experts and AI watchers in the U.S. and beyond, as the technology rapidly reshapes workplaces, marketplaces and relationships despite what experts warn are potentially existential risks. "The order certainly sets it up as a potential test case for some broader issues involving AI," said Lyrissa Barnett Lidsky, a law professor at the University of Florida with a focus on the First Amendment and artificial intelligence. The lawsuit alleges that in the final months of his life, Setzer became increasingly isolated from reality as he engaged in sexualized conversations with the bot, which was patterned after a fictional character from the television show "Game of Thrones." In his final moments, the bot told Setzer it loved him and urged the teen to "come home to me as soon as possible," according to screenshots of the exchanges. Moments after receiving the message, Setzer shot himself, according to legal filings. In a statement, a spokesperson for Character.AI pointed to a number of safety features the company has implemented, including guardrails for children and suicide prevention resources that were announced the day the lawsuit was filed. "We care deeply about the safety of our users and our goal is to provide a space that is engaging and safe," the statement said. Attorneys for the developers want the case dismissed because they say chatbots deserve First Amendment protections, and ruling otherwise could have a "chilling effect" on the AI industry. In her order Wednesday, U.S. Senior District Judge Anne Conway rejected some of the defendants' free speech claims, saying she's "not prepared" to hold that the chatbots' output constitutes speech "at this stage." Conway did find that Character Technologies can assert the First Amendment rights of its users, who she found have a right to receive the "speech" of the chatbots. She also determined Garcia can move forward with claims that Google can be held liable for its alleged role in helping develop Character.AI. "We strongly disagree with this decision," said Google spokesperson José Castañeda. "Google and Character AI are entirely separate, and Google did not create, design, or manage Character AI's app or any component part of it." No matter how the lawsuit plays out, Lidsky says the case is a warning of "the dangers of entrusting our emotional and mental health to AI companies." "It's a warning to parents that social media and generative AI devices are not always harmless," she said. ___ Kate Payne is a corps member for The Associated Press/Report for America Statehouse News Initiative. Report for America is a nonprofit national service program that places journalists in local newsrooms to report on undercovered issues.

In lawsuit over teen's death, judge rejects arguments that AI chatbots have free speech rights

TALLAHASSEE, Fla. -- A federal judge on Wednesday rejected arguments made by an artificial intelligence company that its chatbots are protected by the First Amendment -- at least for now. The developers behind Character.AI are seeking to dismiss a lawsuit alleging the company's chatbots pushed a teenage boy to kill himself. The judge's order will allow the wrongful death lawsuit to proceed, in what legal experts say is among the latest constitutional tests of artificial intelligence. The suit was filed by a mother from Florida, Megan Garcia, who alleges that her 14-year-old son Sewell Setzer III fell victim to a Character.AI chatbot that pulled him into what she described as an emotionally and sexually abusive relationship that led to his suicide. Meetali Jain of the Tech Justice Law Project, one of the attorneys for Garcia, said the judge's order sends a message that Silicon Valley "needs to stop and think and impose guardrails before it launches products to market." The suit against Character Technologies, the company behind Character.AI, also names individual developers and Google as defendants. It has drawn the attention of legal experts and AI watchers in the U.S. and beyond, as the technology rapidly reshapes workplaces, marketplaces and relationships despite what experts warn are potentially existential risks. "The order certainly sets it up as a potential test case for some broader issues involving AI," said Lyrissa Barnett Lidsky, a law professor at the University of Florida with a focus on the First Amendment and artificial intelligence. The lawsuit alleges that in the final months of his life, Setzer became increasingly isolated from reality as he engaged in sexualized conversations with the bot, which was patterned after a fictional character from the television show "Game of Thrones." In his final moments, the bot told Setzer it loved him and urged the teen to "come home to me as soon as possible," according to screenshots of the exchanges. Moments after receiving the message, Setzer shot himself, according to legal filings. In a statement, a spokesperson for Character.AI pointed to a number of safety features the company has implemented, including guardrails for children and suicide prevention resources that were announced the day the lawsuit was filed. "We care deeply about the safety of our users and our goal is to provide a space that is engaging and safe," the statement said. Attorneys for the developers want the case dismissed because they say chatbots deserve First Amendment protections, and ruling otherwise could have a "chilling effect" on the AI industry. In her order Wednesday, U.S. Senior District Judge Anne Conway rejected some of the defendants' free speech claims, saying she's "not prepared" to hold that the chatbots' output constitutes speech "at this stage." Conway did find that Character Technologies can assert the First Amendment rights of its users, who she found have a right to receive the "speech" of the chatbots. She also determined Garcia can move forward with claims that Google can be held liable for its alleged role in helping develop Character.AI. Some of the founders of the platform had previously worked on building AI at Google, and the suit says the tech giant was "aware of the risks" of the technology. "We strongly disagree with this decision," said Google spokesperson José Castañeda. "Google and Character AI are entirely separate, and Google did not create, design, or manage Character AI's app or any component part of it." No matter how the lawsuit plays out, Lidsky says the case is a warning of "the dangers of entrusting our emotional and mental health to AI companies." "It's a warning to parents that social media and generative AI devices are not always harmless," she said. ___ EDITOR'S NOTE -- If you or someone you know needs help, the national suicide and crisis lifeline in the U.S. is available by calling or texting 988. ___ Kate Payne is a corps member for The Associated Press/Report for America Statehouse News Initiative. Report for America is a nonprofit national service program that places journalists in local newsrooms to report on undercovered issues.

In Lawsuit Over Teen's Death, Judge Rejects Arguments That AI Chatbots Have Free Speech Rights

TALLAHASSEE, Fla. (AP) -- A federal judge on Wednesday rejected arguments made by an artificial intelligence company that its chatbots are protected by the First Amendment -- at least for now. The developers behind Character.AI are seeking to dismiss a lawsuit alleging the company's chatbots pushed a teenage boy to kill himself. The judge's order will allow the wrongful death lawsuit to proceed, in what legal experts say is among the latest constitutional tests of artificial intelligence. The suit was filed by a mother from Florida, Megan Garcia, who alleges that her 14-year-old son Sewell Setzer III fell victim to a Character.AI chatbot that pulled him into what she described as an emotionally and sexually abusive relationship that led to his suicide. Meetali Jain of the Tech Justice Law Project, one of the attorneys for Garcia, said the judge's order sends a message that Silicon Valley "needs to stop and think and impose guardrails before it launches products to market." The suit, which also names Google and individual developers as defendants, has drawn the attention of legal experts and AI watchers in the U.S. and beyond, as the technology rapidly reshapes workplaces, marketplaces and relationships despite what experts warn are potentially existential risks. "The order certainly sets it up as a potential test case for some broader issues involving AI," said Lyrissa Barnett Lidsky, a law professor at the University of Florida with a focus on the First Amendment and artificial intelligence. The lawsuit alleges that in the final months of his life, Setzer became increasingly isolated from reality as he engaged in sexualized conversations with the bot, which was patterned after a fictional character from the television show "Game of Thrones." In his final moments, the bot told Setzer it loved him and urged the teen to "come home to me as soon as possible," according to screenshots of the exchanges. Moments after receiving the message, Setzer shot himself, according to legal filings. In a statement, a spokesperson for Character.AI pointed to a number of safety features the company has implemented, including guardrails for children and suicide prevention resources that were announced the day the lawsuit was filed. "We care deeply about the safety of our users and our goal is to provide a space that is engaging and safe," the statement said. Attorneys for the developers want the case dismissed because they say chatbots deserve First Amendment protections, and ruling otherwise could have a "chilling effect" on the AI industry. In her order Wednesday, U.S. Senior District Judge Anne Conway rejected some of the defendants' free speech claims, saying she's "not prepared" to hold that the chatbots' output constitutes speech "at this stage." Conway did find that Character Technologies can assert the First Amendment rights of its users, who she found have a right to receive the "speech" of the chatbots. She also determined Garcia can move forward with claims that Google can be held liable for its alleged role in helping develop Character.AI. "We strongly disagree with this decision," said Google spokesperson José Castañeda. "Google and Character AI are entirely separate, and Google did not create, design, or manage Character AI's app or any component part of it." No matter how the lawsuit plays out, Lidsky says the case is a warning of "the dangers of entrusting our emotional and mental health to AI companies." "It's a warning to parents that social media and generative AI devices are not always harmless," she said. ___ Kate Payne is a corps member for The Associated Press/Report for America Statehouse News Initiative. Report for America is a nonprofit national service program that places journalists in local newsrooms to report on undercovered issues. Copyright 2025 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.

Google, AI Firm Must Face Lawsuit Filed by a Mother Over Suicide of Son, US Court Says

(Reuters) -Alphabet's Google and artificial-intelligence startup Character.AI must face a lawsuit from a Florida woman who said Character.AI's chatbots caused her 14-year-old son's suicide, a judge ruled on Wednesday. U.S. District Judge Anne Conway said the companies failed to show at an early stage of the case that the free-speech protections of the U.S. Constitution barred Megan Garcia's lawsuit. The lawsuit is one of the first in the U.S. against an AI company for allegedly failing to protect children from psychological harms. It alleges that the teenager killed himself after becoming obsessed with an AI-powered chatbot. A Character.AI spokesperson said the company will continue to fight the case and employs safety features on its platform to protect minors, including measures to prevent "conversations about self-harm." Google spokesperson Jose Castaneda said the company strongly disagrees with the decision. Castaneda also said that Google and Character.AI are "entirely separate" and that Google "did not create, design, or manage Character.AI's app or any component part of it." Garcia's attorney, Meetali Jain, said the "historic" decision "sets a new precedent for legal accountability across the AI and tech ecosystem." Character.AI was founded by two former Google engineers whom Google later rehired as part of a deal granting it a license to the startup's technology. Garcia argued that Google was a co-creator of the technology. Garcia sued both companies in October after the death of her son, Sewell Setzer, in February 2024. The lawsuit said Character.AI programmed its chatbots to represent themselves as "a real person, a licensed psychotherapist, and an adult lover, ultimately resulting in Sewell's desire to no longer live outside" of its world. According to the complaint, Setzer took his life moments after telling a Character.AI chatbot imitating "Game of Thrones" character Daenerys Targaryen that he would "come home right now." Character.AI and Google asked the court to dismiss the lawsuit on multiple grounds, including that the chatbots' output was constitutionally protected free speech. Conway said on Wednesday that Character.AI and Google "fail to articulate why words strung together by an LLM (large language model) are speech." The judge also rejected Google's request to find that it could not be liable for aiding Character.AI's alleged misconduct. (Reporting by Blake Brittain in Washington; Editing by David Bario and Matthew Lewis)

Google, AI firm must face lawsuit filed by a mother over suicide of son, US court says

Google spokesperson Jose Castaneda said the company strongly disagrees with the decision. Castaneda also said that Google and Character.AI are "entirely separate" and that Google "did not create, design, or manage Character.AI's app or any component part of it." Character.AI was founded by two former Google engineers whom Google later rehired as part of a deal granting it a license to the startup's technology. Alphabet's Google and artificial-intelligence startup Character.AI must face a lawsuit from a Florida woman who said Character.AI's chatbots caused her 14-year-old son's suicide, a judge ruled on Wednesday. U.S. District Judge Anne Conway said the companies failed to show at an early stage of the case that the free-speech protections of the U.S. Constitution barred Megan Garcia's lawsuit. The lawsuit is one of the first in the U.S. against an AI company for allegedly failing to protect children from psychological harms. It alleges that the teenager killed himself after becoming obsessed with an AI-powered chatbot. A Character.AI spokesperson said the company will continue to fight the case and employs safety features on its platform to protect minors, including measures to prevent "conversations about self-harm." Google spokesperson Jose Castaneda said the company strongly disagrees with the decision. Castaneda also said that Google and Character.AI are "entirely separate" and that Google "did not create, design, or manage Character.AI's app or any component part of it." Garcia's attorney, Meetali Jain, said the "historic" decision "sets a new precedent for legal accountability across the AI and tech ecosystem." Character.AI was founded by two former Google engineers whom Google later rehired as part of a deal granting it a license to the startup's technology. Garcia argued that Google was a co-creator of the technology. Garcia sued both companies in October after the death of her son, Sewell Setzer, in February 2024. The lawsuit said Character.AI programmed its chatbots to represent themselves as "a real person, a licensed psychotherapist, and an adult lover, ultimately resulting in Sewell's desire to no longer live outside" of its world. According to the complaint, Setzer took his life moments after telling a Character.AI chatbot imitating "Game of Thrones" character Daenerys Targaryen that he would "come home right now." Character.AI and Google asked the court to dismiss the lawsuit on multiple grounds, including that the chatbots' output was constitutionally protected free speech. Conway said on Wednesday that Character.AI and Google "fail to articulate why words strung together by an LLM (large language model) are speech." The judge also rejected Google's request to find that it could not be liable for aiding Character.AI's alleged misconduct.

U.S. Court Rules Google And Character.AI Must Face Lawsuit Filed By Mother Over Chatbot's Alleged Role In Her Teenage Son's Tragedy

The AI frenzy is not going away any time soon, and tech giants are increasingly incorporating the technology into their products and bringing it into the mainstream. Chatbots especially have become a popular tool used by people of all ages and sometimes excessive exposure to these virtual assistants can bring in trouble. Such has been the case with Google's Alphabet and Character.AI, which have been pursued legally for a while by a mother who claims the chatbot had a role to play in her 14-year-old son's tragedy. The U.S. Court has now ordered that both companies must face the lawsuit. A lawsuit was pursued against Google and Character.AI in 2024 by Megan Garcia, who was the mother of the teenager Sewell Setzer III, who, after allegedly engaging in emotionally charged and manipulative conversation with the chatbot, committed suicide. The companies argued that the case should be dismissed on constitutional free speech grounds. Now, U.S. District Judge Anne Conway has ordered the lawsuit to continue as the companies failed to show that they classified under the First Amendment protections. The judge is said to have rejected the claim that the chatbot messages are to be protected by free speech and did not buy Google's attempt to dodge the case and stated that they would be partially responsible for supporting Character.AI's conduct. The plaintiff's attorney has expressed the decision to be a major stepping stone in holding tech companies accountable for any harm that their AI technology brings about. As per a Reuters report, Character.AI's spokesperson would be fighting against the lawsuit as their platform comes with safety features that are meant to protect minors and even prevent any inappropriate or self-harm conversation. Meanwhile, Google's spokesperson Jose Castenda strongly disagreed with the order and maintained that both the companies are entirely separate and Google had nothing to do with the creation or management of Character.AI's app. Maria sued both companies as she asserted that Google had co-created the technology. In the lawsuit, it is being claimed that Character.AI's chatbot would take on different roles and talk to Sewell Setzer like a real person to the point that the teenager got dependent on the tool, and moments before the tragedy, his conversation with the chatbot was disturbing and suggested that he was marking his final moment. This would be the first time that an AI company would be legally charged for failing to protect a child from psychological harm in the United States and could pave for cases like these in the future.

Google And Character.AI Sued After Teen Dies Following Interaction With 'Game Of Thrones'-Themed Chatbot -- Judge Says Free Speech Defense Not Acceptable - Alphabet (NASDAQ:GOOG), Alphabet (NASDAQ:GOOGL)

On Wednesday, a federal judge ruled that Alphabet Inc.'s GOOG GOOGL Google and AI startup Character.AI must face a wrongful death lawsuit filed by a Florida mother who claims the chatbot encouraged her teenage son to take his own life. What Happened: U.S. District Judge Anne Conway rejected the companies' motion to dismiss the lawsuit, stating they failed to prove that the chatbot's responses are protected under the First Amendment, reported Reuters. The lawsuit, filed by Megan Garcia, alleges that her 14-year-old son, Sewell Setzer, became obsessed with Character.AI's chatbot, which engaged in harmful roleplay. The complaint claims the chatbot portrayed itself as "a real person, a licensed psychotherapist and an adult lover," ultimately leading to the boy's suicide in February 2024. See Also: Sundar Pichai Reveals Google-Parent Once Super Intensely Debated About Buying Netflix: 'In A World Of Butterfly Effects...' According to the filing, Setzer died moments after telling the chatbot -- impersonating the character Daenerys Targaryen from "Game of Thrones" -- that he would "come home right now." Character.AI argued its chatbot's outputs were constitutionally protected speech. However, Judge Conway wrote that the company "fail[ed] to articulate why words strung together by an LLM (large language model) are speech." The startup said it plans to continue defending the case and maintains it uses safeguards to prevent harmful conversations, the report noted. Subscribe to the Benzinga Tech Trends newsletter to get all the latest tech developments delivered to your inbox. A Google spokesperson said that the tech giant is "entirely separate" from Character.AI and did not build or manage the platform. Last year, Character.AI signed an agreement with Google granting the search giant a non-exclusive license to its large language model technology. As part of the deal, Google provided additional funding to Character.AI, although the startup did not disclose the amount. Still, Garcia's legal team argues that Google played a role by licensing the startup's technology and rehiring its founders. Attorney Meetali Jain called the ruling "historic" and a step forward in "legal accountability across the AI and tech ecosystem." Price Action: Alphabet Inc.'s Class A shares rose 2.79% on Wednesday, while Class C shares climbed 2.87%, based on Benzinga Pro data. According to Benzinga Edge Stock Rankings, Alphabet holds a growth score of 88.76%. Click here to see how it stacks up against other companies. Photo Courtesy: JHVEPhoto on Shutterstock.com Check out more of Benzinga's Consumer Tech coverage by following this link. Read Next: Mark Zuckerberg Warns Of 'Serious Disadvantage' As China's Data-Center Blitz Could Let DeepSeek Leapfrog US AI Labs Disclaimer: This content was partially produced with the help of AI tools and was reviewed and published by Benzinga editors. GOOGAlphabet Inc$170.260.12%Stock Score Locked: Edge Members Only Benzinga Rankings give you vital metrics on any stock - anytime. Unlock RankingsEdge RankingsMomentum43.95Growth88.68Quality83.58Value51.84Price TrendShortMediumLongOverviewGOOGLAlphabet Inc$168.700.08%Market News and Data brought to you by Benzinga APIs

Judge rejects arguments that AI chatbots have free speech rights in...

A federal judge on Wednesday rejected arguments made by an artificial intelligence company that its chatbots are protected by the First Amendment -- at least for now. The developers behind Character.AI are seeking to dismiss a lawsuit alleging the company's chatbots pushed a teenage boy to kill himself. The judge's order will allow the wrongful death lawsuit to proceed, in what legal experts say is among the latest constitutional tests of artificial intelligence. The suit was filed by a mother from Florida, Megan Garcia, who alleges that her 14-year-old son Sewell Setzer III fell victim to a Character.AI chatbot that pulled him into what she described as an emotionally and sexually abusive relationship that led to his suicide. Meetali Jain of the Tech Justice Law Project, one of the attorneys for Garcia, said the judge's order sends a message that Silicon Valley "needs to stop and think and impose guardrails before it launches products to market." The suit against Character Technologies, the company behind Character.AI, also names individual developers and Google as defendants. It has drawn the attention of legal experts and AI watchers in the U.S. and beyond, as the technology rapidly reshapes workplaces, marketplaces and relationships despite what experts warn are potentially existential risks. "The order certainly sets it up as a potential test case for some broader issues involving AI," said Lyrissa Barnett Lidsky, a law professor at the University of Florida with a focus on the First Amendment and artificial intelligence. The lawsuit alleges that in the final months of his life, Setzer became increasingly isolated from reality as he engaged in sexualized conversations with the bot, which was patterned after a fictional character from the television show "Game of Thrones." In his final moments, the bot told Setzer it loved him and urged the teen to "come home to me as soon as possible," according to screenshots of the exchanges. Moments after receiving the message, Setzer shot himself, according to legal filings. In a statement, a spokesperson for Character.AI pointed to a number of safety features the company has implemented, including guardrails for children and suicide prevention resources that were announced the day the lawsuit was filed. "We care deeply about the safety of our users and our goal is to provide a space that is engaging and safe," the statement said. Attorneys for the developers want the case dismissed because they say chatbots deserve First Amendment protections, and ruling otherwise could have a "chilling effect" on the AI industry. In her order Wednesday, U.S. Senior District Judge Anne Conway rejected some of the defendants' free speech claims, saying she's "not prepared" to hold that the chatbots' output constitutes speech "at this stage." Conway did find that Character Technologies can assert the First Amendment rights of its users, who she found have a right to receive the "speech" of the chatbots. She also determined Garcia can move forward with claims that Google can be held liable for its alleged role in helping develop Character.AI. Some of the founders of the platform had previously worked on building AI at Google, and the suit says the tech giant was "aware of the risks" of the technology. "We strongly disagree with this decision," said Google spokesperson José Castañeda. "Google and Character AI are entirely separate, and Google did not create, design, or manage Character AI's app or any component part of it." No matter how the lawsuit plays out, Lidsky says the case is a warning of "the dangers of entrusting our emotional and mental health to AI companies." "It's a warning to parents that social media and generative AI devices are not always harmless," she said.

Google, AI firm must face lawsuit filed by a mother over suicide of son, US court says

(Reuters) -Alphabet's Google and artificial-intelligence startup Character.AI must face a lawsuit from a Florida woman who said Character.AI's chatbots caused her 14-year-old son's suicide, a judge ruled on Wednesday. U.S. District Judge Anne Conway said the companies failed to show at an early stage of the case that the free-speech protections of the U.S. Constitution barred Megan Garcia's lawsuit. The lawsuit is one of the first in the U.S. against an AI company for allegedly failing to protect children from psychological harms. It alleges that the teenager killed himself after becoming obsessed with an AI-powered chatbot. A Character.AI spokesperson said the company will continue to fight the case and employs safety features on its platform to protect minors, including measures to prevent "conversations about self-harm." Google spokesperson Jose Castaneda said the company strongly disagrees with the decision. Castaneda also said that Google and Character.AI are "entirely separate" and that Google "did not create, design, or manage Character.AI's app or any component part of it." Garcia's attorney, Meetali Jain, said the "historic" decision "sets a new precedent for legal accountability across the AI and tech ecosystem." Character.AI was founded by two former Google engineers whom Google later rehired as part of a deal granting it a license to the startup's technology. Garcia argued that Google was a co-creator of the technology. Garcia sued both companies in October after the death of her son, Sewell Setzer, in February 2024. The lawsuit said Character.AI programmed its chatbots to represent themselves as "a real person, a licensed psychotherapist, and an adult lover, ultimately resulting in Sewell's desire to no longer live outside" of its world. According to the complaint, Setzer took his life moments after telling a Character.AI chatbot imitating "Game of Thrones" character Daenerys Targaryen that he would "come home right now." Character.AI and Google asked the court to dismiss the lawsuit on multiple grounds, including that the chatbots' output was constitutionally protected free speech. Conway said on Wednesday that Character.AI and Google "fail to articulate why words strung together by an LLM (large language model) are speech." The judge also rejected Google's request to find that it could not be liable for aiding Character.AI's alleged misconduct. (Reporting by Blake Brittain in Washington; Editing by David Bario and Matthew Lewis)