AI-Generated Pope Sermons Flood Social Media, Raising Concerns About Misinformation

AI-Generated Pope Content Proliferates on Social Media

In a startling development, artificial intelligence (AI) generated videos and audios purporting to be sermons by Pope Leo XIV have flooded popular social media platforms YouTube and TikTok. An investigation by AFP has uncovered dozens of pages on these platforms that have been producing AI-generated messages in the pope's voice or attributed to him since he assumed leadership of the Catholic Church last month 1

1

2

Source: France 24

Scale and Impact of AI-Generated Content

The investigation revealed hundreds of fabricated sermons and speeches in both English and Spanish, highlighting the ease with which AI-created hoaxes can evade detection and mislead viewers. Some of these AI-generated videos have garnered significant attention, with one Spanish-language video on TikTok receiving 9.6 million views and another, despite carrying an AI label, amassing 32.9 million views 1

1

2

Platform Response and Moderation Challenges

In response to AFP's findings, YouTube terminated 16 channels for violating its policies against spam, deceptive practices, and scams. The platform also removed an additional channel for violating its terms of service and excluded six pages from its partner program that allows content monetization 1

1

TikTok took similar action, removing 11 accounts with over 1.3 million combined followers that were flagged by AFP. The platform cited violations of its policies on impersonation, harmful misinformation, and misleading AI-generated content of public figures 1

1

2

Labeling and Disclosure Issues

Both YouTube and TikTok require creators to label posts containing realistic AI-generated content. However, the investigation found that these disclaimers were often difficult to locate or entirely absent. On YouTube, labels indicating "altered or synthetic content" are only visible in the video description, which users must click to open 1

1

2

Related Stories

Pope Francis Warns Davos Summit of AI's Potential to Exacerbate 'Crisis of Truth'

Pope's AI Warnings Flagged as AI-Generated by Detection Tool With 99.98% Accuracy Rate

AI Deepfakes Impersonate Pastors to Scam Congregations Across the United States

Pope Leo XIV Rejects AI Avatar, Warns of Technology's Impact on Humanity

Expert Opinions and Concerns

Source: ET

Experts have raised alarms about the potential consequences of this trend. Oren Etzioni, founder of TrueMedia.org, noted that the public's natural interest in a new pope's stance and style creates "a perfect opportunity to sow mischief with AI-generated misinformation" 1

1

Brian Patrick Green, director of technology ethics at Santa Clara University, attributed the moderation difficulties to rapid AI developments inspiring "chaotic uses of the technology" 1

1

2

Broader Implications and Future Challenges

The proliferation of AI-generated content raises concerns beyond immediate misinformation. Experts warn that even seemingly harmless fakes can be problematic, potentially being used to build trust before pivoting to more harmful content. Green emphasized that these AI-generated sermons not only "corrode the pope's moral authority" but also "make whatever he actually says less believable" 1

1

2

The pope himself has warned about the risks of AI, and Vatican News has called out a deepfake showing Leo praising a controversial political figure. AFP has also debunked clips depicting the pope criticizing political leaders 1

1

2

As the line between real and fake content becomes increasingly blurred, Green concludes, "There's a real crisis here. We're going to have to figure out some way to know whether things are real or fake" 1

1

2