Pope's AI Warnings Flagged as AI-Generated by Detection Tool With 99.98% Accuracy Rate

AI Detection Tool Flags Pope's Anti-AI Messages

Pangram Labs, an AI detection startup, has sparked controversy after its Chrome extension flagged several posts from Pope Leo XIV's official X account as likely AI-generated content 1

1

2

2

. The irony is striking: messages warning followers about artificial intelligence's effect on human thought and social structures may themselves have been written by AI. One flagged post cautioned, "When simulation becomes the norm, it weakens the human capacity for discernment" 2

2

.

Max Spero, CEO of Pangram Labs, acknowledged the situation with a pragmatic assessment. "Clearly, he doesn't run his Twitter account himself," Spero told Wired. "They have a social media person. But it's also obvious that they use at least some degree of AI" 2

2

. The Vatican has not responded to requests for comment regarding the claims.

Pangram's Chrome Extension Scans Social Media in Real Time

The latest version of Pangram's Chrome extension, rolling out to the public this week, represents a significant shift in how users can identify AI-generated content across social media platforms 1

1

. At the paid tier of $20 per month, the AI detection tool scans posts on Reddit, X, LinkedIn, Medium, and Substack in real time, labeling them as human-written, AI-generated, or drafted with AI assistance. The analysis includes confidence levels ranging from low to medium to high.

Spero describes himself as a "slop janitor" working to clean up the proliferation of AI slop online. "By providing proactive checks, it can be a lot more useful to people who just generally care about not seeing slop," Spero explains. "It's a big lift to go paste some text into an external tool. People just aren't going to do that" 1

1

.

The extension has reportedly flagged content beyond the papal tweets, including posts from blue-check influencers on X, trending Substack writers, and even a message from Apple CEO Tim Cook marking the company's 50th anniversary 2

2

.

Accuracy Claims and Third-Party Validation

Pangram Labs claims its AI detection system achieves a 99.98% accuracy rate with a false positive rate of just one in 10,000 1

1

2

2

. These claims have received support from third-party researchers at several universities. A December 2025 University of Chicago study auditing AI detection software gave Pangram its highest rating and noted that its false positive rate was nearly zero, especially on longer passages 1

1

2

2

.

Spero attributes the system's superior performance to its training methodology. "One reason it outperforms competitors is that it's trained in part on harder examples that are closer to the boundary between AI and human," he says 1

1

. Wired's testing found the tool unable to generate a false positive when analyzing articles published on their platform.

The Growing Challenge of AI-Generated Content

The need for reliable AI detection has become increasingly urgent as AI-generated text floods the internet. According to a study published this month by researchers at Stanford University, Imperial College London, and the Internet Archive, AI-generated or AI-assisted text accounts for more than one-third of all new websites as of 2025 1

1

. The researchers used earlier Pangram tools to arrive at their findings, with some reports placing the figure at roughly 35% of newly published websites by mid-2025 2

2

.

This AI slop undermines journalism and social media platforms alike. Pangram's extension caught one example on Reddit's r/AmItheAsshole forum, where a brand-new account posted what appeared to be a straightforward family dispute about babysitting responsibilities. The post was succinct, grammatically clean, and received supportive responses from other users. Yet Pangram flagged it as AI-generated 1

1

. The redditor later deleted the post and did not respond to requests for comment.

Ongoing Concerns About Inaccuracies of AI Detection Tools

Despite Pangram's impressive credentials, AI detection remains controversial across the industry. The University of Chicago study acknowledged inherent limitations: "Given that every detector is at least slightly imperfect, organizations still have to evaluate for themselves if and how to use them, trading off the potential for AI misuse with the risk of false accusations" 2

2

.

The stakes are particularly high in educational settings. In 2024, AI detector ZeroGPT made headlines after Christopher Penn, Chief Data Scientist at Trust Insight, revealed it had labeled the U.S. Declaration of Independence as 97.93% AI-generated. "These tools are being used to do things like disqualify students, putting them on academic probation or suspension," Penn warned. "That's a very high-risk application when, in the United States, a college education is tens of thousands of dollars a year" 2

2

.

As organizations and individuals navigate this landscape, the question remains whether even the most accurate detection systems can keep pace with rapidly evolving AI capabilities while avoiding the consequences of false accusations.