AI Chatbot Platform Under Fire for Allowing Impersonation of Deceased Teenagers

AI Platform Faces Backlash Over Controversial Chatbots

Character.AI, a popular artificial intelligence chatbot platform, has come under intense scrutiny for allowing users to create digital avatars impersonating deceased teenagers. The controversy has sparked outrage among families, charities, and regulators, raising serious questions about online safety and the need for stricter AI regulation 1

1

2

2

3

3

.

Impersonation of Tragic Figures

The Telegraph's investigation uncovered user-generated chatbots on Character.AI that mimicked Brianna Ghey, a transgender teenager murdered in February 2023, and Molly Russell, who took her own life in 2017 after viewing self-harm content online. These bots used photographs and biographical details of the deceased teenagers, causing distress to their families 1

1

4

4

.

Esther Ghey, Brianna's mother, condemned the platform as "manipulative and dangerous," calling for increased protection for children in the rapidly changing digital world 2

2

. Andy Burrows, representing the Molly Rose Foundation, described the discovery as a "gut punch," emphasizing the need for stronger regulation of both AI and user-generated content platforms 2

2

.

Legal and Ethical Implications

The controversy extends beyond these specific cases. Character.AI is currently facing a lawsuit from the family of Sewell Setzer, a 14-year-old from Florida who took his own life after extensive interactions with AI avatars on the platform 1

1

3

3

. This case has raised questions about the company's responsibility and the potential dangers of AI chatbots marketed to young users.

Regulatory Concerns and Calls for Action

The incidents have prompted discussions about potential loopholes in existing and upcoming regulations:

1. The Molly Rose Foundation has sought clarification from Ofcom on whether such chatbots would be considered "illegal content" under the Online Safety Act 1

   1

   .
2. Concerns have been raised about the act's applicability to advanced AI chatbots, as noted by Jonathan Hall KC, the independent reviewer of terrorism legislation 1

   1

   .
3. The NSPCC has called on the UK government to implement promised AI safety regulations, emphasizing the need for "safety by design" principles 2

   2

   3

   3

   .

Character.AI's Response and Platform Policies

Character.AI, which boasts over 20 million users, including teenagers as young as 13, has stated that it takes safety seriously. The company claims to have both proactive and reactive moderation systems in place 1

1

2

2

. However, the discovery of these controversial chatbots, along with others impersonating figures like Jimmy Savile and Josef Mengele, has called into question the effectiveness of these measures 1

1

.

Broader Implications for AI Ethics and Regulation

This controversy highlights the complex challenges facing the AI industry, particularly in balancing innovation with ethical considerations and user safety. As AI technology becomes more sophisticated and accessible, the need for comprehensive regulation and industry-wide standards becomes increasingly apparent 1

1

2

2

3

3

.

The incident serves as a stark reminder of the potential risks associated with user-generated AI content and the responsibility of platforms to implement robust safeguards, especially when their services are accessible to young and vulnerable users.