AI Coding Assistants Reduce Critical Thinking Among Developers, Study Finds

Study Methodology and Scope

Researchers at Saarland University conducted an empirical study comparing traditional human-human pair programming with human-AI pair programming using GitHub Copilot 1

1

2

2

. The study involved 19 students with programming experience, divided into pairs: six worked with human partners while seven collaborated with AI assistants. Participants tackled programming tasks involving implementing features within an existing codebase of approximately 400 lines, distributed across 5 files containing both Python code and comments 1

1

.

The research team tracked conversational "episodes" between human pairs using speech recognition tools and monitored human-AI interactions through screen recordings. They analyzed these conversations for "contribution to knowledge transfer," focusing on information exchange patterns between participants 1

1

.

Key Findings on Knowledge Transfer Patterns

The study revealed significant differences in knowledge transfer between the two approaches. Human-human pairings generated 210 episodes compared to 126 episodes in human-AI pair programming sessions 1

1

. However, the nature of these interactions varied considerably.

"Code" conversations were more frequent in human-machine pairings, while "lost sight" outcomes—where conversations became sidetracked—were more common in human pairings 1

1

. The research identified "a high level of TRUST episodes in human-AI pair programming sessions," with developers showing a tendency to accept AI-generated suggestions without critical evaluation 1

1

2

2

.

Professor Sven Apel, who led the research, noted that "the programmers who were working with an AI assistant were more likely to accept AI-generated suggestions without critical evaluation. They assumed the code would work as intended" 2

2

. This contrasted sharply with human pairs, who "were much more likely to ask critical questions and were more inclined to carefully examine each other's contributions" 2

2

.

Implications for Software Development Quality

The research highlights concerning implications for software development practices. While AI assistants demonstrated efficiency in generating code quickly, they also reduced the broader knowledge exchange that characterizes effective human collaboration 1

1

. The study found that "when it comes to building deeper knowledge it must be treated with care, especially for students" 1

1

.

Apel warns that uncritical reliance on AI could lead to the accumulation of "technical debt," representing hidden costs of future work needed to correct mistakes and complications in software development 2

2

. This concern is amplified by separate research from Cloudsmith, which found that despite developers being "acutely aware of the perils of LLM generated code," including recommendations for non-existent or malicious packages, "a third of developers were deploying AI generated code without review" 1

1

.

Industry Adoption and Market Impact

The findings come as AI coding assistants gain widespread adoption across the software development industry. GitHub's latest Octoverse report revealed that 80 percent of new users are embracing Copilot technology 1

1

. The influence extends beyond adoption rates, with AI assistants "shaping the languages developers use, with a shift to more strongly typed languages which lend themselves to code generation platforms" 1

1

.

Despite the efficiency gains, the research suggests that AI assistants cannot fully replicate the richness of human collaboration in software development. As Apel explains, "They are certainly useful for simple, repetitive tasks, but for more complex problems, knowledge exchange is essential—and that currently works best between humans, possibly with AI assistants as supporting tools" 2

2

.