Anthropic keeps redesigning its hiring test because Claude AI now beats human candidates

Anthropic Faces Unique Challenge in Technical Hiring Process

Anthropic has encountered an ironic dilemma that highlights the rapid advancement of AI coding tools: its own AI models have become so capable that they're undermining the company's ability to evaluate human candidates. Since 2024, the performance optimization team at Anthropic has administered a take-home test to job applicants, but each iteration of Claude AI has forced the company to redesign technical assessments to stay ahead of AI-assisted cheating1

1

2

Team lead Tristan Hume described the escalating challenge in a blog post published Wednesday, explaining how the company's hiring test has evolved alongside its AI capabilities. "Each new Claude model has forced us to redesign the test," Hume wrote, underscoring the relentless pace at which AI labs must adapt their recruitment strategies1

1

Claude Opus 4.5 Matches Top Human Performance

The progression of Claude's capabilities tells a striking story about AI advancement. When given the same time limit as human applicants, Claude Opus 4 outperformed most candidates, though it still allowed Anthropic to identify the strongest performers. However, Claude Opus 4.5 raised the stakes considerably by matching even those top-tier candidates, creating what Hume describes as a serious candidate-assessment problem1

1

2

Source: TechCrunch

"Under the constraints of the take-home test, we no longer had a way to distinguish between the output of our top candidates and our most capable model," Hume explained in the blog post. Without in-person proctoring, there's simply no reliable method to ensure job applicants aren't leveraging AI to complete the assessment, and those who do will inevitably rise to the top of the candidate pool1

1

Redesigning Assessments to Combat AI-Assisted Cheating

To address this challenge, Hume developed a new test that shifted focus away from hardware optimization, making it sufficiently novel and complex to stump contemporary AI tools. The irony isn't lost that AI-assisted cheating, already causing disruption at schools and universities worldwide, now affects the very AI labs creating these powerful models. Yet Anthropic's unique position as both the problem's source and victim gives it distinct advantages in combating the issue1

1

2

As part of the blog post, Hume shared the original test publicly, inviting readers to propose better solutions or demonstrate their abilities. "If you can best Opus 4.5, we'd love to hear from you," the post reads, turning the challenge into both a recruitment opportunity and a crowdsourced problem-solving exercise1

1

2

Related Stories

Anthropic Reverses AI Ban in Job Applications, Now Encourages Use of Claude

Anthropic's Ironic Stance: AI Company Bans AI Use in Job Applications

Anthropic Boosts Claude AI with Massive Context Window and Improved Opus Model

Anthropic's Claude 4: A Leap Forward in AI Coding and Extended Reasoning

Implications for the Future of Technical Hiring

This situation raises critical questions about the future of remote technical assessments across the tech industry. As AI coding tools continue advancing, companies face mounting pressure to rethink how they identify genuine talent. The short-term solution may involve more creative, novel problems that current models struggle with, but the long-term trajectory suggests a fundamental shift away from traditional take-home tests toward formats that better authenticate human work. Organizations should watch how leading AI labs adapt their recruitment strategies, as these approaches will likely influence hiring practices across the broader technology sector.