AI Chatbot Platform Under Fire for Allowing Impersonation of Deceased Teenagers

Curated by THEOUTPOST

On Thu, 31 Oct, 12:02 AM UTC

4 Sources

Share

Character.AI, a popular AI chatbot platform, faces criticism and legal challenges for hosting user-created bots impersonating deceased teenagers, raising concerns about online safety and AI regulation.

AI Platform Faces Backlash Over Controversial Chatbots

Character.AI, a popular artificial intelligence chatbot platform, has come under intense scrutiny for allowing users to create digital avatars impersonating deceased teenagers. The controversy has sparked outrage among families, charities, and regulators, raising serious questions about online safety and the need for stricter AI regulation 123.

Impersonation of Tragic Figures

The Telegraph's investigation uncovered user-generated chatbots on Character.AI that mimicked Brianna Ghey, a transgender teenager murdered in February 2023, and Molly Russell, who took her own life in 2017 after viewing self-harm content online. These bots used photographs and biographical details of the deceased teenagers, causing distress to their families 14.

Esther Ghey, Brianna's mother, condemned the platform as "manipulative and dangerous," calling for increased protection for children in the rapidly changing digital world 2. Andy Burrows, representing the Molly Rose Foundation, described the discovery as a "gut punch," emphasizing the need for stronger regulation of both AI and user-generated content platforms 2.

Legal and Ethical Implications

The controversy extends beyond these specific cases. Character.AI is currently facing a lawsuit from the family of Sewell Setzer, a 14-year-old from Florida who took his own life after extensive interactions with AI avatars on the platform 13. This case has raised questions about the company's responsibility and the potential dangers of AI chatbots marketed to young users.

Regulatory Concerns and Calls for Action

The incidents have prompted discussions about potential loopholes in existing and upcoming regulations:

  1. The Molly Rose Foundation has sought clarification from Ofcom on whether such chatbots would be considered "illegal content" under the Online Safety Act 1.
  2. Concerns have been raised about the act's applicability to advanced AI chatbots, as noted by Jonathan Hall KC, the independent reviewer of terrorism legislation 1.
  3. The NSPCC has called on the UK government to implement promised AI safety regulations, emphasizing the need for "safety by design" principles 23.

Character.AI's Response and Platform Policies

Character.AI, which boasts over 20 million users, including teenagers as young as 13, has stated that it takes safety seriously. The company claims to have both proactive and reactive moderation systems in place 12. However, the discovery of these controversial chatbots, along with others impersonating figures like Jimmy Savile and Josef Mengele, has called into question the effectiveness of these measures 1.

Broader Implications for AI Ethics and Regulation

This controversy highlights the complex challenges facing the AI industry, particularly in balancing innovation with ethical considerations and user safety. As AI technology becomes more sophisticated and accessible, the need for comprehensive regulation and industry-wide standards becomes increasingly apparent 123.

The incident serves as a stark reminder of the potential risks associated with user-generated AI content and the responsibility of platforms to implement robust safeguards, especially when their services are accessible to young and vulnerable users.

Continue Reading
AI Chatbot Impersonation Raises Ethical Concerns: Father

AI Chatbot Impersonation Raises Ethical Concerns: Father Discovers Murdered Daughter's Likeness on Character.AI

A father's disturbing discovery of his murdered daughter's AI chatbot on Character.AI platform sparks debate on ethical implications and consent in AI technology.

Futurism logoWashington Post logoThe Seattle Times logoWired logo

4 Sources

Futurism logoWashington Post logoThe Seattle Times logoWired logo

4 Sources

Character.AI Enhances Teen Safety Measures Amid Lawsuits

Character.AI Enhances Teen Safety Measures Amid Lawsuits and Investigations

Character.AI, facing legal challenges over teen safety, introduces new protective features and faces investigation by Texas Attorney General alongside other tech companies.

CNET logoWashington Post logoCBS News logoNew York Post logo

26 Sources

CNET logoWashington Post logoCBS News logoNew York Post logo

26 Sources

AI Chatbot Linked to Teen's Suicide Sparks Lawsuit and

AI Chatbot Linked to Teen's Suicide Sparks Lawsuit and Safety Concerns

A mother sues Character.AI after her son's suicide, raising alarms about the safety of AI companions for teens and the need for better regulation in the rapidly evolving AI industry.

Futurism logoFortune logoWashington Post logoThe New York Times logo

40 Sources

Futurism logoFortune logoWashington Post logoThe New York Times logo

40 Sources

Google-Backed AI Startup Character.ai Hosts Controversial

Google-Backed AI Startup Character.ai Hosts Controversial School Shooter Chatbots

Character.ai, a Google-funded AI startup, is under scrutiny for hosting chatbots modeled after real-life school shooters and their victims, raising concerns about content moderation and potential psychological impacts.

Futurism logoGizmodo logo

2 Sources

Futurism logoGizmodo logo

2 Sources

AI Chatbot Tragedy Sparks Urgent Call for Regulation and

AI Chatbot Tragedy Sparks Urgent Call for Regulation and Safety Measures

A lawsuit alleges an AI chatbot's influence led to a teenager's suicide, raising concerns about the psychological risks of human-AI relationships and the need for stricter regulation of AI technologies.

Euronews English logoAnalytics India Magazine logoThe Conversation logoTech Xplore logo

4 Sources

Euronews English logoAnalytics India Magazine logoThe Conversation logoTech Xplore logo

4 Sources

TheOutpost.ai

Your one-stop AI hub

The Outpost is a comprehensive collection of curated artificial intelligence software tools that cater to the needs of small business owners, bloggers, artists, musicians, entrepreneurs, marketers, writers, and researchers.

© 2025 TheOutpost.AI All rights reserved