OpenAI Claims Breakthrough in COBOL Code Crisis with 98% Migration Accuracy Using GPT-6.1 Sol

5 Sources

Share

OpenAI announced its GPT-6.1 Sol model achieved 97.9% accuracy in migrating COBOL programs to modern languages like Python and Java while preserving behavior. The breakthrough addresses the decades-old challenge of modernizing legacy systems that power 95% of ATM transactions and 40% of online banking systems, though experts caution production deployment still requires extensive testing.

OpenAI Targets Decades-Old COBOL Code Crisis with AI Models

OpenAI announced substantial improvements in its AI models' ability to work with legacy programming languages like COBOL, potentially addressing a crisis that costs the US economy an estimated $105 billion annually

1

. The company's GPT-6.1 Sol model achieved 86.6% behavior preservation accuracy at low reasoning effort when migrating COBOL programs to modern languages, climbing to 97.9% at higher reasoning settings

2

. This marks a dramatic leap from GPT-5.6 Sol's mere 5.4% accuracy at the same task

4

.

Source: Engadget

Source: Engadget

The breakthrough matters because COBOL, created in 1959, remains embedded in critical infrastructure worldwide

5

. According to IBM figures cited by OpenAI, COBOL supports more than 40% of online banking systems, handles 80% of in-person credit card transactions, and processes 95% of ATM transactions

1

. Yet the programmers who understand these systems have largely retired or died, creating an urgent need for AI-assisted software modernization.

Testing AI Models Against Legacy Programming Languages

OpenAI developed two evaluation benchmarks to measure its models' capabilities with legacy systems. MultiLangBench asks models to build SQLite-compatible database engines from scratch in 20 programming languages

4

. GPT-6 Astra achieved an average test pass rate of 87.4% across all languages, compared to 26.8% for GPT-5.6 Sol

2

. COBOL remained the hardest language to crack, with Astra's pass rate reaching 72.1%

4

.

Source: Gizmodo

Source: Gizmodo

The second benchmark tests COBOL migration to modern languages including C++, Java, Kotlin, Python, and Rust while preserving exact program behavior

3

. This requires models to maintain critical details like how COBOL rounds numbers with excess decimal digits and handles missing records—nuances that could produce different financial results even if translated code runs without errors

2

. OpenAI's hidden tests require byte-for-byte parity on observable outputs, and a migration counts as successful only if all tests pass

5

.

Anthropic Enters the Legacy Code Modernization Race

Anthropic also sees opportunity in legacy systems. In February, the company published a code modernization playbook and positioned its Claude Code agent as a solution to migrate COBOL code into modern languages like Java or Python

1

. Anthropic claimed "the economics of COBOL modernization have shifted," suggesting teams could modernize codebases in quarters instead of years

3

. IBM shares fell 13% following Anthropic's announcement—the company's biggest single-day slide since 2000—as investors worried about threats to IBM's mainframe business that helps companies maintain legacy systems

3

.

Source: The Verge

Source: The Verge

However, Anthropic offered no performance benchmarks showing how well Claude Code could perform such migrations, unlike OpenAI's detailed testing results

1

. The competition signals growing recognition that AI-assisted software modernization represents a massive market opportunity as organizations face mounting pressure to modernize legacy software before the remaining pool of COBOL engineers disappears entirely.

Expert Warnings: Progress Doesn't Mean Production-Ready

Despite OpenAI's impressive benchmarks, experts caution that these advances remain far from enabling safe replacement of critical infrastructure. "Rewriting code is not the same as replacing a system," said Vadim Zaytsev, associate professor of software evolution at the University of Twente

1

. He noted the largest system he'd seen contained 250 million lines of COBOL, and code represents only part of the challenge. Systems may use multiple languages, store data in various formats, integrate with other systems, and must meet expectations of people and organizations that depend on them

1

.

OpenAI spokesperson Oscar Haines acknowledged that legacy code "is still far from a solved problem" and said the company is exploring ways to build better tests and improve its models' understanding of coding languages

1

. The company emphasized that applying behavior-preserving capabilities to production software still requires expert review and testing against business rules and dependencies

2

.

What Organizations Should Watch For

The short-term implication is that financial institutions and government agencies now have AI tools capable of assisting with code modernization, though not replacing human expertise. Banks that still rely on COBOL systems face mounting pressure as COBOL programmers age out of the workforce, making migration increasingly urgent

2

. Organizations should begin pilot programs testing these AI capabilities on non-critical systems while maintaining rigorous human oversight.

Longer-term, the question is whether AI models can eventually handle the full complexity of production legacy systems. OpenAI's GPT-6.1 Sol costs 61% less than GPT-5.6 Sol while delivering dramatically better results, suggesting economic viability is improving

4

. However, the gap between 98% accuracy in controlled tests and the 100% reliability required for banking systems and critical infrastructure remains significant. Watch for how quickly organizations move beyond pilot programs to production deployments, and whether any major COBOL migration failures emerge as early adopters test these tools at scale.

© 2026 TheOutpost.AI All rights reserved