Claude Fable 5 dominates Google's revamped Android AI dev benchmark with 84.5% accuracy

Claude Fable 5 Leads Android AI Development Rankings

Google has significantly updated Android Bench, its AI model benchmark designed to evaluate how large language models perform in Android AI development 1

1

. The revamped leaderboard introduces eight new models and adopts the Harbor framework, making it easier for developers to test and compare AI agents for Android coding tasks. Claude Fable 5 emerges as the clear winner, achieving 84.5% accuracy across 100 Android development tasks—a comfortable 4-point lead over GPT 5.5, which scored 80.2% 2

2

. Claude Sonnet 5 trails nearly 10 points behind Fable 5, while Gemini 3.1 Pro sits in fifth place, highlighting a concerning performance gap for Google's own AI models for Android coding.

Harbor Framework Enables Community Collaboration

The shift to the Harbor framework marks a strategic pivot in how Google evaluates AI agent evaluation. Previously, Android Bench relied on a mini-swe-agent v1 benchmark tool, but the new standardized approach allows Android developers to analyze models for individual use cases and submit their own Android development tasks for inclusion in official testing . Google re-ran all previous tests with Harbor to establish a new baseline, with historical data archived separately. The company emphasizes transparency, making methodology and test harness publicly available on GitHub to encourage community collaboration . This open approach invites developers to shape the future of Android Bench by sharing benchmark evaluations and contributing real-world coding scenarios.

Operational Costs Reveal Trade-offs in AI Model Selection

While Claude Fable 5 and GPT 5.5 dominate in accuracy, their operational costs present significant considerations for developers. Both models consume more than $130 in tokens to complete the 100-problem, 10-run benchmark 1

1

. By comparison, Gemini 3.1 Pro costs $87 per run despite lower accuracy scores. Surprisingly, Gemini 3.5 Flash—designed as a cost-efficient option—registers the highest cost on the leaderboard at $165 per run due to its 28-hour runtime 1

1

. These metrics help developers balance performance against budget constraints when selecting agentic development tools.

Google Faces Pressure as Gemini Lags in Android Coding

The Android coding performance gap poses challenges as Google shifts toward agentic development across its projects. The company would naturally prefer Android developers adopt Google's tools in their workflows, yet Anthropic and OpenAI models consistently outperform Gemini variants 1

1

. This competitive pressure may explain reports that Google has been offering to purchase application source code from developers for AI training purposes. The updated Android Bench now includes open-weight models alongside closed-weight variants, with additions like Claude Opus 4.8, GLM 5.2, Kimi K2.7 Code, MiniMax M3, Qwen 3.7 Plus, and Qwen 3.7 Max joining the evaluation suite . As code generation emerges as a primary application for large language models, choosing the right tool becomes critical for separating useful outputs from inadequate results. Developers can now access the updated Android Bench dataset and instructions through GitHub to run their own tests and contribute to this evolving standard for measuring AI capabilities in Android app development.