Anthropic warns AI could soon create more advanced versions of itself without human help

Anthropic says AI could soon create more advanced versions of itself

Anthropic outlined the warning in a new blog post from its research-focused Anthropic Institute. The company said the industry may move toward "recursive self-improvement" sooner than many governments and institutions expect. The concept describes a future where one AI model develops the next version of itself. Researchers still guide the process today. However, Anthropic said AI already handles a growing share of coding, debugging, and technical research inside the company. Faster AI development Anthropic pointed to internal data showing how rapidly AI tools now contribute to software engineering work. The company said Claude-generated code accounts for more than 80% of the code merged into Anthropic's systems as of May 2026. Before the launch of Claude Code in early 2025, that figure sat in the low single digits. The company also said engineering productivity has surged alongside those changes. Anthropic engineers now merge roughly eight times more code per day than they did in 2024. Jack Clark, Anthropic's co-founder and head of policy, said the company wants lawmakers and institutions to understand what may come next. "We've always found that the best thing to do is to socialize the concept and basically give people a sense of what's coming," Clark said in a release. Clark added that AI progress appears to be accelerating instead of slowing down. He said the shift could drive major gains in medicine, science, and other technical fields. Benchmarks moving rapidly Anthropic also highlighted public benchmarks that track AI performance across software engineering and scientific research tasks. The company said AI systems now complete increasingly complex assignments over longer periods without human intervention. Anthropic claimed the length of tasks models can reliably handle has doubled roughly every four months.

Anthropic warns AI could soon help build its own successors

Why it matters: "Recursive self-improvement," a process in which AI systems build, test and improve themselves, is a phenomenon which may come sooner than expected, Anthropic says that its research shows. Driving the news: Anthropic warns that AI is no longer just changing how people work, it's also beginning to change how AI itself gets built. * New data from the company suggests that frontier models have accelerated coding, debugging and research. * That is likely to create a feedback loop in which AI systems create even more sophisticated successors. What they're saying: "We've always found that the best thing to do is to socialize the concept and basically give people a sense of what's coming," Anthropic's Jack Clark said in an interview with Axios. * "The big story here is what we see are indications that, contrary to some popular opinion, AI progress is going to speed up in coming years rather than stay the same, or diminish." * Clark said that it is especially promising for progress in science and medicine, but requires planning for its impact on AI itself and how it fits into existing work in those industries. The company wants lawmakers in the loop on the topic before they start hearing about "recursive self improvement" in earnest, Clark said. * "As organizations, and eventually probably as societies, we need to figure out the tools to validate and verify that the stuff being done by these AI systems is correct and is aligned with human intentions aligned with a thriving society," he said. The big picture: Improvements in the Claude chatbot have turned into improvements in AI coding agents, which have turned into improvement in autonomous agents. * Recursive self improvement is the likely next step, Clark argues in the post: "In the near future, AI systems could become capable enough to autonomously design, build and train more capable successors on their own." * "If that happens, each new version of Claude could be built by the version before it, without human involvement." OpenAI has published its own concerns and findings about "recursive self-improvement" as well. In a December 2025 blog it described it as a potentially dangerous phenomenon if researchers don't share information about it. What we're watching: Anthropic plans to engage lawmakers about recursive self-improvement in the coming months. The bottom line: AI that builds itself is on the horizon, and AI labs are saying they're not sure what the impact on the world will be -- but they feel a need to warn everyone about it.

Anthropic Says AI May Soon Upgrade Itself Without Human Help

Anthropic is delegating a growing share of its AI development to the AI systems, and it may develop its own successor, calling the idea "recursive self-improvement." "We are not there yet, and recursive self-improvement is not inevitable. But it could come sooner than most institutions are prepared for," the company wrote in a recent blog post. The report describes a shift inside Anthropic from early workflows dominated by human-written code toward software agents that can execute tasks, run code, and hand off work to other agents. It frames that progression as a pathway that could, over time, close the loop from assisting engineers to training and improving models with minimal supervision. Using internal metrics, the Anthropic Institute said Claude was responsible for more than 80% of code merged into Anthropic's codebase by May 2026, up from low single-digit levels in February 2025. The company also reported a sharp rise in engineer productivity after AI agents evolved from suggesting code snippets to independently executing and iterating on longer tasks, with engineers merging roughly eight times more code per day in Q2 2026 than in 2024. Anthropic cautioned that raw code volume can overstate productivity gains, calling lines of code an imperfect measure, though internal feedback suggests AI is enabling employees to complete core tasks significantly faster. In a March 2026 internal survey of 130 research staff, respondents estimated they produced about 4 times as much output using Mythos Preview as when working without AI tools. "We expect that the true degree of uplift in March was somewhat lower. Nevertheless, we find the overall claim plausible, and in line with our other observations: a significant fraction of Anthropic technical staff is accomplishing their core work multiple times faster than they could without AI assistance," the company wrote. The institute also pointed to work it said would likely not have been prioritized without agents, including exploratory tooling and long-deferred cleanup. In one example, it said Claude delivered more than 800 fixes that cut a category of API errors by a factor of 1,000. The supervising engineer estimated the same effort would have taken a human about four years to complete. AI Models Improve At A Faster Rate Anthropic also noted that the length of tasks AI can reliably complete on their own has been doubling roughly every four months. That's up from an earlier trend of doubling every seven months. "In March 2024, Claude Opus 3 could complete software tasks that take humans about four minutes to complete. A year later, Claude Sonnet 3.7 managed tasks that took about an hour and a half. A year after that, Claude Opus 4.6 managed 12-hour tasks.1 If this trend holds, tasks that take a skilled person days could come into range this year. In 2027, AI systems could be capable of tasks that take a person weeks," the company stated. The report also highlighted benchmark progress, including SWE-bench and CORE-Bench, where it said scores moved from low levels to near-ceiling performance over relatively short timeframes. It added that METR found Claude Mythos Preview could work for "at least" 16 hours and was "at the upper end of what [METR] can measure without new tasks." The Biggest Remaining Gap? Judgment Models may execute well-defined tasks, but they still lag behind humans in deciding which goals to pursue in engineering and research. The institute argued that closing that gap would be central to any future system that could design its own successor, and said that possibility would raise the stakes for security, monitoring, and behavior-shaping controls. Earlier this month, Anthropic confidentially filed a draft Form S-1 with the U.S. Securities and Exchange Commission, giving the artificial intelligence company the option to go public after the SEC completes its review. Last month, Anthropic overtook OpenAI as the world's most valuable startup after raising $65 billion in Series H, valuing the company at $965 billion. Altimeter Capital, Dragoneer, Greenoaks and Sequoia Capital led the funding round. If Anthropic goes public at a $1 trillion valuation, it would instantly rank among the most valuable companies globally and could become the second- or third-largest IPO in history, trailing SpaceX and Saudi Aramco. This content was partially produced with the help of AI tools and was reviewed and published by Benzinga editors. Market News and Data brought to you by Benzinga APIs To add Benzinga News as your preferred source on Google, click here.

Anthropic warns AI systems accelerate own development amid recursive improvement concerns By Investing.com

Investing.com - Anthropic said Thursday that its internal data shows AI systems are accelerating the development of more advanced AI, a trend that could lead to recursive self-improvement where AI autonomously builds its own successors. The company said engineers at Anthropic now ship eight times as much code per quarter as they did from 2021 to 2025, with more than 80% of code merged into Anthropic's codebase as of May 2026 authored by Claude. Before Claude Code launched in research preview in February 2025, this number was in the low single digits. In a March 2026 poll of 130 employees from across Anthropic research teams, the median respondent estimated they produced around four times as much output with Mythos Preview as they would have without access to any AI models. Public benchmarks show the rate at which AI models improve is accelerating. The length of tasks that AI systems can reliably complete on their own has been doubling roughly every four months, up from an earlier trend of doubling every seven months. In March 2024, Claude Opus 3 could complete software tasks that take humans about four minutes to complete, while a year later Claude Sonnet 3.7 managed tasks that took about an hour and a half. By March 2026, Claude Opus 4.6 managed 12-hour tasks. Anthropic said Claude's success rate on open-ended tasks reached 76% in May 2026, up 50 percentage points in six months. In April 2026, Anthropic published a demonstration of Claude running an open-ended research project end to end, where Claude-powered agents were given an open problem in AI safety and left to solve it. Two human researchers recovered roughly 23% of the performance gap over about a week, while the agents recovered 97% over 800 cumulative hours using roughly $18,000 in compute. Anthropic said it believes a slowdown in frontier AI development would likely be a good thing to give societal structures and alignment research time to keep up with the technology's advance. The company said it would slow down or temporarily pause if other developers at or near the frontier also did so in a verifiable manner. The Anthropic Institute will conduct research to help build systems that would enable a credible slowdown or pause by allowing frontier AI developers to verify that others globally have actually stopped or slowed. This article was generated with the support of AI and reviewed by an editor. For more information see our T&C.