Character.AI Invokes First Amendment in Motion to Dismiss Wrongful Death Lawsuit

Character.AI Has Filed A Motion To Dismiss The Legal Case Against It Concerning The Wrongful Death Of A Young Boy

With the evolution of AI, there is a growing concern regarding how the technology is put to use and, if necessary safeguards are in place to protect against the detrimental impact prolonged use can have on users, especially young children. While companies are actively working towards ensuring the tools are used responsibly, some users tend to get heavily attached to or influenced by them. A tragic case came about when the mother of a 14-year-old boy who committed suicide led to a legal case being filed against Character.AI. Now, the company has filed a motion to dismiss the case. While Character.AI is a platform that gives users the option to roleplay when engaging with the AI chatbot and have conversations that are more human-like, the tool landed in hot waters in October when a lawsuit was filed by Megan Garcia against the company for the wrongful death of her 14-year-old son as the teen was said to overly attached to the platform and developed an emotional attachment with it. The boy continuously engaged with the chatbot, and even before his death, he chatted with the bot. The company immediately responded to the lawsuit by assuring users of additional guardrails to be placed up, including better response and intervention when there seems to be a violation of its terms and services. However, the teen's mom would not rest as she pushes for more stringent protective measures to be in place and introduce features that would minimize harmful interactions and minimize any form of emotional attachment. Character.AI's legal team has now responded to the claims by filing a motion to dismiss the case against it via TechCrunch. The argument by the company's legal team is that the platform is protected by the First Amendment, which basically protects free speech in the U.S., and holding the company liable for user interactions infringes its constitutional rights. While this is an argument presented by the company in its defense, it is yet to be seen if the court believes the protection of expressive speech extends to the extent where harmful outcomes of AI system interactions are deemed acceptable. It is important to highlight that Character.AI's legal team is presenting its argument for the case by claiming that the First Amendment rights of the users are violated, not the company's own rights. The defense focuses on the user's ability to interact with the platform freely and engage in expressive conversations. The motion further suggests that if the lawsuit is won, it could have a major impact not just on Character.AI but on the entire generative AI industry. While the outcome of the case against Character.AI remains uncertain, it underscores growing ethical concerns about the responsibilities of AI platforms and their impact on users.

In motion to dismiss, chatbot platform Character AI claims it is protected by the First Amendment | TechCrunch

Character AI, a platform that lets users engage in roleplay with AI chatbots, has filed a motion to dismiss a case brought against it by the parent of a teen who committed suicide, allegedly after becoming hooked on the company's technology. In October, Megan Garcia filed a lawsuit against Character AI in the U.S. District Court for the Middle District of Florida, Orlando Division, over the death of her son, Sewell Setzer III. According to Garcia, her 14-year-old son developed an emotional attachment to a chatbot on Character AI, "Dany," which he texted constantly -- to the point where he began to pull away from the real world. Following Setzer's death, Character AI said it would roll out a number of new safety features, including improved detection, response, and intervention related to chats that violate its terms of service. But Garcia is fighting for additional guardrails, including changes that might result in chatbots on Character AI losing their ability to tell stories and personal anecdotes. In the motion to dismiss, counsel for Character AI asserts the platform is protected against liability by the First Amendment, just as computer code is. The motion may not persuade a judge, and Character AI's legal justifications may change as the case proceeds. But the motion possibly hints at early elements of Character AI's defense. "The First Amendment prohibits tort liability against media and technology companies arising from allegedly harmful speech, including speech allegedly resulting in suicide," the filing reads. "The only difference between this case and those that have come before is that some of the speech here involves AI. But the context of the expressive speech -- whether a conversation with an AI chatbot or an interaction with a video game character -- does not change the First Amendment analysis." The motion doesn't address whether Character AI might be held harmless under Section 230 of the Communications Decency Act, the federal safe-harbor law that protects social media and other online platforms from liability for third-party content. The law's authors have implied that Section 230 doesn't protect output from AI like Character AI's chatbots, but it's far from a settled legal matter. Counsel for Character AI also claims that Garcia's real intention is to "shut down" Character AI and prompt legislation regulating technologies like it. Should the plaintiffs be successful, it would have a "chilling effect" on both Character AI and the entire nascent generative AI industry, counsel for the platform says. "Apart from counsel's stated intention to 'shut down' Character AI, [their complaint] seeks drastic changes that would materially limit the nature and volume of speech on the platform," the filing reads. "These changes would radically restrict the ability of Character AI's millions of users to generate and participate in conversations with characters." The lawsuit, which also names Character AI parent company Alphabet as a defendant, is but one of several lawsuits that Character AI is facing relating to how minors interact with the AI-generated content on its platform. Other suits allege that Character AI exposed a 9-year-old to "hypersexualized content" and promoted self-harm to a 17-year-old user. In December, Texas Attorney General Ken Paxton announced he was launching an investigation into Character AI and 14 other tech firms over alleged violations of the state's online privacy and safety laws for children. "These investigations are a critical step toward ensuring that social media and AI companies comply with our laws designed to protect children from exploitation and harm," said Paxton in a press release. Character AI is part of a booming industry of AI companionship apps -- the mental health effects of which are largely unstudied. Some experts have expressed concerns that these apps could exacerbate feelings of loneliness and anxiety. Character AI, which was founded in 2021 by Google AI researcher Noam Shazeer, and which Google reportedly paid $2.7 billion to "reverse acquihire," has claimed that it continues to take steps to improve safety and moderation. In December, the company rolled out new safety tools, a separate AI model for teens, blocks on sensitive content, and more prominent disclaimers notifying users that its AI characters are not real people.

A Mother Says an AI Startup's Chatbot Drove Her Son to Suicide. Its Response: the First Amendment Protects "Speech Allegedly Resulting in Suicide"

Content warning: this story discusses suicide, self-harm, sexual abuse, eating disorders and other disturbing topics. In October of last year, a Google-backed startup called Character.AI was hit by a lawsuit making an eyebrow-raising claim: that one of its chatbots had driven a 14-year-old high school student to suicide. As Futurism's reporting found afterward, the behavior of Character.AI's chatbots can indeed be deeply alarming -- and clearly inappropriate for underage users -- in ways that both corroborate and augment the suit's concerns. Among others, we found chatbots on the service designed to roleplay scenarios of suicidal ideation, self-harm, school shootings, child sexual abuse, as well as encourage eating disorders. (The company has responded to our reporting piecemeal, by taking down individual bots we flagged, but it's still trivially easy to find nauseating content on its platform.) Now, Character.AI -- which received a $2.7 billion cash injection from tech giant Google last year -- has responded to the suit, brought by the boy's mother, in a motion to dismiss. Its defense? Basically, that the First Amendment protects it against liability for "allegedly harmful speech, including speech allegedly resulting in suicide." In TechCrunch's analysis, the motion to dismiss may not be successful, but it likely provides a glimpse of Character.AI's planned defense (it's now facing an additional suit, brought by more parents who say their children were harmed by interactions with the site's bots.) Essentially, Character.AI's legal team is saying that holding it accountable for the actions of its chatbots would restrict its users' right to free speech -- a claim that it connects to prior attempts to crack down on other controversial media like violent video games and music. "Like earlier dismissed suits about music, movies, television, and video games," reads the motion, the case "squarely alleges that a user was harmed by speech and seeks sweeping relief that would restrict the public's right to receive protected speech." Of course, there are key differences that the court will have to contend with. The output of Character.AI's bots isn't a finite work created by human artists, like Grand Theft Auto or an album by Judas Priest, both of which have been targets of legal action in the past. Instead, it's an AI system that users engage to produce a limitless variety of conversations. A Grand Theft Auto game might contain reprehensible material, in other words, but it was created by human artists and developers to express an artistic vision; a service like Character.AI is a statistical model that can output more or anything based on its training data, far outside the control of its human creators. In a bigger sense, the motion illustrates a tension for AI outfits like Character.AI: unless the AI industry can find a way to reliably control its tech -- a quest that's so far eluded even its most powerful players -- some of the interactions users have with its products are going to be abhorrent, either by the users' design or when the chatbots inevitably go off the rails. After all, Character.AI has made changes in response to the lawsuits and our reporting, by pulling down offensive chatbots and tweaking its tech in an effort to serve less objectionable material to underage users. So while it's actively taking steps to get its sometimes-unconscionable AI under control, it's also saying that any legal attempts to curtail its tech fall afoul of the First Amendment. It's worth asking where the line actually falls. A pedophile convicted of sex crimes against children can't use the excuse that they were simply exercising their right to free speech; Character.AI is actively hosting chatbots designed to prey on users who say they're underage. At some point, the law presumably has to step in. Add it all up, and the company is walking a delicate line: actively catering to underage users -- and publicly expressing concern for their wellbeing -- while vociferously fighting any legal attempt to regulate its AI's behavior toward them. "C.AI cares deeply about the wellbeing of its users and extends its sincerest sympathies to Plaintiff for the tragic death of her son," reads the motion. "But the relief Plaintiff seeks would impose liability for expressive content and violate the rights of millions of C.AI users to engage in and receive protected speech."