UMass Study Reveals AI in Classroom Boosts Student Engagement Without Improving Test Scores

Groundbreaking Classroom Experiment

A semester-long controlled study at the University of Massachusetts Amherst has provided new insights into the role of artificial intelligence in higher education, revealing that while AI tools don't improve academic performance, they significantly enhance student engagement and learning efficiency. The research, led by Christian Rojas, professor and chair in the Department of Resource Economics, compared two sections of an upper-division antitrust economics course with identical curricula but different AI policies 1

1

Source: CBS

The experiment involved 57 students across two back-to-back sections of the same course. One section of 29 students was encouraged to use AI tools such as ChatGPT with structured guidance and disclosure requirements, while the other section of 28 students was prohibited from using AI and received parallel non-AI study support 2

2

Source: Gizmodo

Key Research Findings

The study's most significant finding was that AI access had no measurable effect on exam scores or final grades. Both sections performed equally well on paper-and-pencil exams administered without notes, AI, or other technology. However, the AI-permitted class consistently reported higher satisfaction levels and demonstrated markedly different learning behaviors 3

3

Students with AI access participated more actively in real-time classroom activities and concentrated their AI use into longer, more substantive study sessions lasting 15-30 minutes. They also developed more reflective learning habits, including editing AI outputs, identifying mistakes, and preferring their own answers over machine-generated responses 2

2

Enhanced Learning Experience

Rojas emphasized that AI didn't help students learn more content but enabled them to learn more efficiently and confidently. "They spent less time outside the classroom on homework and exam preparation," he noted. The AI-using students also gave significantly higher standardized course evaluations, particularly regarding instructor preparation and use of class time 1

1

The study revealed that students with AI access were far more likely to express intentions to pursue careers involving intensive AI use, suggesting the experience influenced their career aspirations. By semester's end, both sections reported similar frequency of AI use in other courses, but the AI section students demonstrated more sophisticated usage patterns 2

2

Related Stories

AI in Education: Reshaping Learning and Challenging Academic Integrity

Universities scramble to rethink exams as 92% of UK students now rely on AI for coursework

AI in Education Surges as Experts Warn of Critical Thinking Decline and Learning Risks

AI Grade Inflation Surges 30% at Universities as Students Learning Less, Study Reveals

Implications for Education Policy

The research supports a "permit with scaffolding" approach to AI integration in education, where students receive explicit instructions about effective AI use and clear disclosure requirements. Rojas suggests this framework allows educators to maintain academic rigor while embracing technological advancement. "There's an opportunity for instructors to be more open about AI usage. Letting students engage with it just creates a different environment," he explained 3

3

The experiment intentionally assigned AI permission to an afternoon section that historically performed slightly worse, creating a conservative test against detecting outsized AI effects. This methodological choice strengthens the study's findings by demonstrating that even potentially disadvantaged students didn't gain unfair advantages from AI access 1

1

Study Limitations and Future Research

Rojas acknowledges several limitations in the research, including the small sample size and reliance on significant amounts of self-reported data from students. The study, detailed in a working paper titled "Allowing Generative AI in Class: Evidence from a Semester-Long Controlled Teaching Study," has been submitted for publication in a peer-reviewed journal and calls for further research on a larger scale 2

2