arXiv Bans AI-Generated Computer Science Reviews Amid Quality Crisis

arXiv Implements Strict New Policy Against AI-Generated Papers

The prestigious preprint repository arXiv has announced a significant policy change that will ban computer science review and position papers unless they have already undergone peer review at an academic journal or conference. The decision, announced on October 31, represents a direct response to what administrators describe as a "flood" of low-quality, AI-generated submissions that have overwhelmed the platform's volunteer moderation system 1

1

.

The policy shift affects one of academia's most important platforms for early research dissemination, particularly in fast-moving fields like artificial intelligence and machine learning. arXiv has become critical infrastructure for scientific communication, allowing researchers to share findings before the lengthy peer review process is complete 2

2

.

Surge in AI-Generated Submissions Overwhelms Moderators

The numbers tell a stark story of the platform's quality crisis. According to Steinn Sigurðsson, arXiv's scientific director and an astrophysicist at Penn State University, rejection rates have skyrocketed from just 2-3% of submissions a year ago to 10% currently 1

1

. The platform now receives hundreds of review articles every month, a dramatic increase from the small number of high-quality reviews historically submitted by senior researchers 3

3

.

Thomas Dietterich, a computer scientist at Oregon State University who chairs arXiv's computer science section, described the problematic submissions as "many surveys that are just annotated bibliographies without analysis, synthesis or road mapping." He noted that citations often include papers on completely unrelated subjects, suggesting potential citation manipulation schemes 1

1

.

The platform's 240 volunteer moderators have found these AI-generated papers particularly time-consuming to evaluate, despite not representing the largest volume of submissions. Stephanie Orphan, arXiv's programme director, explained that modern chatbots have become sophisticated enough to produce content that escapes simple plagiarism detection tools 1

1

.

Broader Impact on Academic Publishing

The arXiv policy change reflects a wider crisis in academic publishing as AI tools make it trivially easy to generate superficial research papers. Recent studies have documented the scope of this problem: research published in Nature Human Behaviour found that nearly a quarter of all computer science abstracts showed evidence of large language model modification by September 2024 3

3

.

The issue extends beyond preprint servers to peer-reviewed journals, where AI-generated content has led to embarrassing retractions and raised questions about the integrity of the review process. Even peer reviewers themselves have been caught using ChatGPT to assist with their evaluations, partly due to the overwhelming demands placed on their time 2

2

.

Mixed Reactions from Research Community

The research community's response to arXiv's new policy has been divided. Stephen Casper, an AI safety researcher, raised concerns that the restrictions might disproportionately harm early-career researchers and those working on ethics and governance topics. "Review/position papers are disproportionately written by young people, people without access to lots of compute, and people who are not at institutions that have lots of publishing experience," he argued 3

3

.

However, others in the academic community have supported the move as necessary. Richard Sever, chief science and strategy officer of openRxiv, which operates bioRxiv and medRxiv, called it "a wise move," noting that both biomedical preprint servers have maintained "no narrative reviews" policies from their inception 1

1

.

Future Implications and Enforcement

Under the new policy, authors submitting computer science review or position papers must provide documentation of successful peer review, including journal references and DOIs. Workshop reviews will not meet this standard. arXiv emphasized that the change currently affects only the computer science category, though other sections may adopt similar policies if they experience comparable surges in AI-generated submissions 3

3

.

Executive director Ramin Zabih indicated that this policy change is likely just the beginning, stating that arXiv is "actively contemplating more measures along these lines" as the platform continues to grapple with the challenges posed by AI-generated content 1

1

.