Anthropic warns AI may soon build itself, calls for global pause on frontier development

Anthropic warns AI may soon begin recursive self-improvement

I agree my information will be processed in accordance with the Scientific American and Springer Nature Limited Privacy Policy. We leverage third party services to both verify and deliver email. By providing your email address, you also consent to having the email address shared with third parties for those purposes. The companies at the frontier of artificial intelligence should be ready to slow down, one of the fastest-moving among them says. Anthropic, the maker of the Claude chatbot, claims AI systems may be on the cusp of what it calls recursive self-improvement -- the point at which they can design and build their own successors with little human input. The company says this could increase the risk of humans losing control of the technology. "We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology," Anthropic said in a June 4 blog post titled "When AI Builds Itself." On supporting science journalism If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today. The proposal highlights a tough problem in AI governance. A slowdown would need rival companies and governments in several countries to accept the same limits at the same time, with no treaty obliging them and competition only intensifying. That makes the warning technically important and politically fraught: Anthropic is calling for the brakes on a race in which it remains a front runner. The speed at which the technology is developing could have "huge implications" on society. Anthropic points to its own operation as a warning sign. The company says Claude now writes more than 80 percent of the code merged into its systems, up from low single digits before it released Claude Code in early 2025, and that its engineers ship around eight times as much code per quarter as they did a few years ago. At each step of building AI, it argues, the human role is shrinking. "We are not there yet, and recursive self-improvement is not inevitable," the company said. "But it could come sooner than most institutions are prepared for." Anthropic floats what it calls "a global coordination mechanism" to slow or even pause AI development and allow society room to catch up. Anthropic was short on specifics. It pointed to arms-control agreements on intermediate-range nuclear missiles as a loose model. For any such pause to hold, it said, the industry's leading labs would need to take part -- and there would need to be a credible way to show they had, in fact, slowed. "I don't think it's a genuine call to slow down," says Noah Giansiracusa, associate professor of mathematics at Bentley University and the author of two books on algorithms and society. "We've read Dario Amodei's blog posts. I think he wants to keep going full speed ahead." Anthropic did not respond to Scientific American's questions about how such a brake would work, or to the criticism that it is overstating what its systems can do. Giansiracusa also thinks a pause is unworkable. "It's literally impossible," he says. "Zero chance there will be a slowdown. I'm not even talking China -- Elon Musk would never slow down." The proposal fits a pattern that makes some researchers wary. Two months ago, Anthropic unveiled a model called Mythos that it declined to release publicly, saying it was too good at finding software vulnerabilities to put in anyone's hands. The pause call also comes just days after the company confidentially filed for an initial public offering, and not long after a funding round valuing the company close to $1 trillion. To skeptics, such startling pronouncements can read as business strategy, a way to draw regulatory scrutiny to the frontier while Anthropic continues racing toward it. Mark Riedl, a professor in the School of Interactive Computing at the Georgia Institute of Technology, posted on Bluesky that "the big AI companies are all jumping on the 'recursive self-improvement' hype train." Anthropic says it will spend the coming months convening governments, researchers, and rival AI companies to work out whether a coordinated slowdown could function in practice. "I don't really see the cause for concern," says Giansiracusa. "They're flirting with the idea of the singularity -- that it's a gamechanger, and I just don't see that. I see it continuing to progress. Maybe things will speed up; maybe it won't." The evidence Anthropic cites -- more code written by AI -- suggests the technology is helpful, he says, rather than "a great leap."

Anthropic warns Claude AI is building itself faster than expected, calls for option to halt frontier development -- 'recursive self improvement' increases risk humans lose control of AI

Internal data shows engineers shipping eight times more code. Anthropic has published a report warning that the development path it's on could eventually leave humans unable to control AI systems, even as it disclosed that Claude now writes more than 80% of the code merged into its own codebase. The Anthropic Institute, the company's research arm, said AI has already started to speed up AI development and that the trend could lead to recursive self-improvement, the point at which a model designs and builds its own successor with little human input. The report argued that the world should keep open the option to slow or pause frontier development, and cautioned that the occasional misalignment seen in current models could grow more common and harder to understand as those models build the next generation. The company set out three pretty dire ways the next few years could unfold, reserving its most severe warnings for the scenario in which models become capable of fully improving themselves. In that case, Anthropic said, the pace of progress would be set almost entirely by available compute, with humans pushed toward oversight and verification roles and a self-improving model dominating as its abilities outstrip those of the people who built it. The firm described this potential issue with alignment and the task of keeping a system's behavior tied to human intent as part of the future it's least sure about. Misalignment that's rare and survivable today could compound generation over generation until control slips, it said, though it allowed that a sufficiently capable and well-aligned model might instead choose to halt its own development. Anthropic wrote that this misalignment could keep "growing more frequent but less understood until we lose control of them." Anthopric is backing these warnings with a bunch of internal figures that we've not seen before. More than 80% of the code merged into its production codebase as of last month was authored by Claude, up from low single digits before Claude Code reached research preview in February last year. Anthropic says the typical engineer is now "merging 8x as much code per quarter as they did from 2021-2025." On the hardest, least-specified coding tasks, Anthropic said Claude succeeded 76% of the time in May 2026, a rise of 50 percentage points in six months. A recurring internal test that asks each new model to make training code run faster saw results climb from roughly triple the original speed with Claude Opus 4 in May 2025 to about 52 times with the unreleased Mythos Preview model in April. Anthropic said it'd slow or pause only if rival labs at or near the frontier did the same in a verifiable way, and that a halt by one company would change who leads without achieving anything wider. That's obviously not going to happen. All the figures cited by Anthropic are self-reported and unaudited, and come days after the company filed to go public. The company issued a similar self-assessment in April, when it said Mythos Preview had found thousands of severe software vulnerabilities, a claim that later drew scrutiny over how much of it rested on a small manual sample. Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.

'It would be good for the world' to slow down AI sprints, Anthropic says

The plea for caution comes the same week it beat AI archrival OpenAI to filing for an IPO It would be "good for the world" to slow down the pace of AI development, according to a blog post from Anthropic, which this week began the process of going public with a confidential IPO filing. "We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology," stated a blog post written by Anthropic co-founder (and former Reg scribe) Jack Clark and researcher Marina Favaro. Executing an actual pause would take a negotiation and monitoring effort on par with nuclear accords, including the agreement from all of the frontier AI labs, as well as support from policy makers around the world. Even then, there is a possibility that some will not abide by any restrictions. "Training runs are far easier to conceal than missile silos, their inputs are general-purpose, and the incentive to defect quietly is enormous, because whoever continues while others pause could inherit the lead. A credible pause also has to specify what triggers it, what lifts it, and who adjudicates." Anthropic has been one of the more alarmist organizations when it comes to the growing capabilities of AI, as it's tried to portray itself as the more safety-concerned alternative to OpenAI, where its cofounders originated. One might dismiss this as clever marketing hype - what better way to convince enterprises to drop millions on largely unproven and sporadically reliable technology than claiming it could be so powerful that it might terminate humanity? In addition, Anthropic's recommendation is taking place the same week it beat archrival OpenAI to filing a confidential IPO as it seeks investment from public markets. Last week, the company announced that it reached a $965 billion valuation, making it one of the most valuable companies in the world. It's pretty rich for a nearly trillion-dollar company to tell everybody else to slow down just as it's about to become unstoppable. Yet the headwinds are swirling. US President Donald Trump signed an executive order this week, which in part directs the Treasury Department to establish an "AI cybersecurity clearinghouse" that works with the AI industry and critical infrastructure operators to coordinate and deconflict the use of advanced AI tools. And it comes amid growing public backlash to rapid datacenter expansion. Nonetheless, throughout the paper, Anthropic explained how the "human role is narrowing" in model development and attempted to make the case that on the current trajectory, models could soon reach a point where they can self-improve and write better versions of themselves without people in the mix. "Once human- and AI-authored code quality reach parity, humans will stop writing code entirely, and shift to only reviewing it," the paper's authors write. "But if they can't review code as quickly as Claude can generate it, human review will become the bottleneck to AI development." As of May, Claude authored more than 80 percent of the code merged into Anthropic's codebase, up from the low single digits before Claude Code launched in research preview in February 2025. Newer models are also improving on complex tasks faster than before. The length of human tasks that models can reliably complete on their own had been doubling every seven months as measured in March 2025. Now it is closer to every four months, they said. The Claude 3 Opus model released in March 2024 could reliably complete tasks that take humans four minutes to complete. Claude Opus 4.6 can reliably complete tasks that take humans 12 hours, the team wrote. "If this trend holds, tasks that take a skilled person days could come into range this year," the paper states. The paper admits that unknown bottlenecks could emerge, which stop the progress that has been made so far, and the next generation of models could see a slowdown in improvement. The authors cite Amdahl's Law, which states that acceleration in one part of the system leads to choke points in other parts. "Anthropic has already encountered one signature of Amdahl's law: as we've begun to push more code around the organization, human code review has become a new bottleneck," they wrote. Anthropic said one area where AI consistently underperforms is in "taste" or selecting the next step to take, when left unprompted by humans. "Without that judgment, Claude is a capable assistant, but not a system that could drive AI progress on its own," the paper states. "It is genuinely unclear whether today's training methods and architectures could unlock that capacity." However, the alternative scenario is that the current trend of models getting better more rapidly may hold. "In this world, the pace of progress in AI development becomes determined entirely by the availability of compute ... Humans play a substantially diminished role in their development, likely moving most of our effort towards oversight, validation, and verification of an expanding 'virtual lab' run by AI systems," the paper states. ®

Anthropic urges a way to pause AI development as risks grow with the tech advances

Anthropic is proposing that the world's top artificial intelligence companies come up with a coordinated way to pause development of advanced AI systems, warning the technology is improving so quickly there's a risk humans would lose control. The company behind the Claude chatbot said in a blog post Thursday that as cutting-edge AI gets increasingly faster at carrying out tasks, "it would be good for the world to have the option to slow or temporarily pause" its development. Anthropic said its internal research institute plans to explore the issue in collaboration with others and "take actions" to help build the systems for a credible slowdown or pause, without being more specific. AI models are getting faster, with rapid increases in how quickly they can carry out software tasks like coding on their own, the company said. Based on current trends and given enough computing power, an AI system could be able to design and develop its own successor, in what is known as "recursive self-improvement." Self-building AI would be a major technological milestone that would bring benefits in science, healthcare and other areas, Anthropic said, but it "also might increase the risks of humans losing control over AI systems." Some tech industry figures have long warned of such a scenario. Anthropic's post comes after a different warning this week from a team of researchers at the University of Toronto who showed how AI tools could be used to create a new kind of AI "worm" that adapts its hacking strategy as it spreads from device to device and takes over a vast computing network. "I think it's really important that people understand that it's not just the biggest, most powerful language models that pose the security concerns," lead researcher Nicolas Papernot said in an interview. The authors of the Anthropic post, company co-founder Jack Clark and Marina Favaro, head of the research institute, said the pause would be used to enable "societal structures and alignment research" to keep up with AI advances. Alignment is industry shorthand for making sure the technology matches human values and intentions. The proposed coordination would let advanced AI labs verify that global rivals have actually stopped or slowed their work, "and that a bad actor could not use the auspices of a coordinated slowdown to jump ahead in secret." The company said a coordinated global mechanism is needed because without it a slowdown in AI development could let the "least cautious" players catch up and add to pressure on companies and governments as they make tough choices about AI safety. Anthropic's post comes as the company itself is racing with ChatGPT-maker OpenAI to sell shares on the stock market, in an IPO that could value it at nearly a trillion dollars.

Anthropic proposes a global slowdown of AI development - Engadget

The company says it's concerned about AI being able to build its own successors. Anthropic says AI is developing so fast, the trend points towards systems becoming capable of developing their own successor. We're not there yet, but it believes it "could come sooner than most institutions are prepared for." In a blog post, Anthropic explains that AI that can build itself could "bring enormous good for the world" in the fields of science and healthcare, among others. However, it might also "increase the risks of humans losing control over AI systems." So how can humanity prevent Skynet from becoming a real thing? Anthropic is suggesting that the answer is the global slowdown or the temporary pause of AI development "to enable societal structures and alignment research to keep up with the advance of the technology." The company is one of the leading names in the AI race today, and unlike its competitors that have yet to make money, it's reportedly on track to have its first profitable quarter. It recently filed paperwork with the SEC to go public, likely before the year ends. As The Wall Street Journal notes, critics suggest that Anthropic's warnings about its own technology are a marketing ploy, perhaps in an effort to make it look like the least egregious among all the AI companies or to make it appear as if its products are the best out there. Critics are specifically pointing to the limited release of its cybersecurity AI model Mythos as an example. Anthropic said it was making Mythos available to a select group of partners due to the potential damage its ability to quickly identify vulnerabilities could cause in the wrong hands. But people think it's just a ploy to hype up the product or to cover up the fact that Anthropic only wants to sell it to the biggest enterprises. It's worth noting, however, that the company's suggestion is based on the findings of Anthropic Institute, a research division it established in March. Anthropic said back then that the institute's role is to "tell the world" what it learns about the challenges that arise as AI firms develop more advanced AI systems. The institute, along with collaborators, will conduct research on what's needed to "build the systems that a credible slowdown or pause would require." If AI companies agree to a slowdown, mechanisms must be in place to verify that they've all actually stopped or slowed down AI development. Otherwise, some could jump ahead of the rest after developing their technology in secret. "A meaningful slowdown or pause would require multiple well-resourced labs at or near the frontier, in multiple countries, agreeing to stop under the same conditions," Anthropic writes. "It would also require that each can verify that the others have actually stopped." Its suggestion would only ever happen if all AI companies around the world can get together and pledge to stop AI development for a while. Anthropic says it's not impossible, citing nuclear-weapons treaties as an example, though it admits those agreements were decades in the making. We don't have that long, seeing as AI development is progressing at a rapid pace. The company is planning to hold talks with policymakers, researchers and other AI companies over the coming months over this issue and will publish the results of those conversations.

OpenAI Joins Anthropic in Call for International AI Watchdog

OpenAI is calling for the formation of an international organization to oversee AI development. And to slow it down, if necessary: "One goal of such an organization should be to make it possible for the world to take coordinated action, including slowing frontier development when needed, so societal resilience, safety, and alignment can keep pace," the company wrote in a blog post published Monday. The memo, written by OpenAI CEO Sam Altman and chief scientist Jakub Pachocki, follows a similar statement from OpenAI's chief rival, Anthropic, which said last week that unilaterally tapping the brakes on global AI development "would likely be a good thing," assuming such a move were possible. Anthropic's rationale was based on internal evidence of what it described as "recursive self-improvement"; that is, AI models capable of training the next, more advanced versions of themselves, thereby presenting new possibilities for quicker advancements in the field as well as new risks around humans losing control. The possibility of such an "intelligence explosion -- occurring as a byproduct of commercial incentives being prioritized over safety concerns -- was also a key factor behind a 2023 open letter calling for a six-month pause on the development of frontier AI models, which was signed by Elon Musk, AI "godfather" and Turing Award-winner Yoshua Bengio, and other big names in tech. OpenAI likewise called attention to AI's self-improving potential in its new blog post. "We believe that AI doing AI research will become the determining factor of the pace of progress within the next few years," Altman and Pachocki wrote. By March of 2028, they continued, OpenAI "may have a significant fraction of our research being done by AI systems in tandem with our own researchers." AGI is always around the next corner Building such an "automated AI researcher," they wrote, is currently one of the big goals OpenAI is currently working towards. Another is to "provide everyone on Earth with a personal AGI"... whatever that means. The AI industry still lacks a widely agreed-upon definition of artificial general intelligence, or AGI. It's generally understood to be an AI system that can perform any cognitive task at least as well as the typical human brain, but it's still a matter of debate whether or not such a system -- at least one built upon the architectures behind large language models -- is even feasible. For years, OpenAI has held it up as a vague promise for the future, which has thus far been a successful financing strategy. Until recently, it was the highest valued startup in the world, before being usurped by Anthropic last month. Both companies are expected to make their stock market debuts this year, raising fresh concerns about fiscal responsibilities to shareholders eclipsing safety concerns about runaway AI. Their calls for an independent, international organization to moderate AI development, therefore, feel like a warning, as if they were saying: Somebody needs to take the reins on this, because very soon, it's going to be out of our hands.

Claude writes 80% of its code, calls for AI pause

Anthropic reveals that Claude now writes over 80% of its production code, with engineers shipping 8x more code per quarter than in 2024. The company's new Anthropic Institute paper maps the path to recursive self-improvement and calls for a verifiable global pause mechanism. One of Anthropic's engineers hasn't written a line of code in five months. Not because the work dried up, but because Claude does it now. As of May 2026, more than 80% of the code merged into Anthropic's production codebase was authored by Claude, up from low single digits when Claude Code launched in February 2025. That figure, published Wednesday in a new Anthropic Institute paper titled "When AI builds itself," is not the headline the company wants you to focus on. The headline is what comes next: AI that can design and train its own successor. Anthropic says it isn't there yet, but it might be closer than most institutions are prepared for. The numbers behind the shift The productivity gains are stark. In Q2 2026, the typical Anthropic engineer merged eight times as much code per day as in 2024. An internal poll of 130 research staff found that the median respondent estimated roughly four times as much output with Anthropic's latest model, Mythos Preview, compared to working without AI. On the most complex, open-ended engineering problems, Claude's success rate climbed to 76% in May 2026, a 50-percentage-point increase in six months. Anthropic gives a concrete example: when a routine upgrade began crashing tens of thousands of training jobs, an engineer pointed Claude at the live incident with little more than some text context and cluster access. Claude isolated an obscure debugging flag, reproduced the crash, and confirmed a fix in about two hours. That would normally take two to three days. The code quality gap is closing, too. Anthropic staff say that Claude-written code was "somewhat worse" than human-written code in late 2025, is at rough parity today, and is expected to be strictly better within the year. An automated Claude reviewer now checks every proposed change to Anthropic's codebase before it can merge. A retrospective analysis found it would have caught roughly a third of the bugs behind past claude.ai incidents before they reached production. From coding to research Writing code is the easy part. The harder question is whether Claude can do research, the kind of open-ended scientific reasoning that drives AI forward. Anthropic's evidence here is more preliminary but still striking. In April 2026, the company published a demonstration of Claude running an open-ended AI safety research project end to end. Nine parallel agents were given a problem, left to propose hypotheses, run experiments, share findings through a common forum, and iterate. Over 800 cumulative hours and roughly $18,000 in compute, the agents recovered 97% of the performance gap on the task. Two human researchers, working for a week, recovered 23%. Another internal experiment measured whether Claude could pick a better "next step" than a human researcher at difficult junctures during real research sessions. In November 2025, Claude matched the human's judgment 51% of the time. By April 2026, that rose to 64%. The day-to-day work of research is largely a chain of these next-step decisions. If that trend continues, the gap between AI-as-assistant and AI-as-researcher narrows fast. The task horizon curve Anthropic's internal data aligns with a broader pattern tracked by METR, a non-profit that benchmarks AI capabilities. The length of tasks AI can reliably complete on its own has been doubling roughly every four months, accelerating from an earlier pace of every seven months. In March 2024, Claude Opus 3 could handle tasks that take a human about four minutes. By early 2025, Claude Sonnet 3.7 managed hour-and-a-half tasks. Today, Claude Opus 4.6 handles 12-hour tasks, and METR found that Mythos Preview could sustain work for at least 16 hours, at the upper end of what the current benchmark suite can measure. If the trend holds, tasks requiring days of skilled human work come into range this year. Weeks-long tasks could follow in 2027. The infrastructure is buckling The downstream effects are already visible. GitHub, the platform most of the world's software is built on, saw roughly one billion code commits in all of 2025. By mid-2026, the platform was processing 275 million commits per week, on pace for 14 billion over the year. Claude Code alone accounts for 4.5% of all public commits on GitHub, generating 2.6 million weekly. GitHub's COO has said the company is "pushing incredibly hard" on capacity just to keep up. Inside Anthropic, the bottleneck has already shifted: as Claude generates more code, human code review has become the constraint. The company says it has encountered a textbook example of Amdahl's law, where speeding up one part of a process simply reveals the next slowest link. The pause question The paper's most significant section is not about productivity. It is a call for a verifiable global mechanism to slow or temporarily pause frontier AI development. Anthropic is careful with the framing. A unilateral pause by one lab would simply change who leads, not create the deliberative process the company says is missing. What Anthropic proposes instead is a system where multiple frontier labs, in multiple countries, could agree to stop under the same conditions and verify that the others had actually done so. It draws a parallel to nuclear arms control but acknowledges the differences: training runs are far easier to conceal than missile silos, the inputs are general-purpose, and the incentive to defect quietly is enormous. "If it were possible to effectively slow the development of this technology to give ourselves more time to deal with its immense implications, we think that would likely be a good thing," the paper states. The AI coding market is now worth tens of billions. Asking the industry to pause is asking it to leave money on the table while trusting that competitors, including those in China, will do the same. What recursive self-improvement would mean The paper lays out three possible futures. In the first, the trend stalls, but even today's capabilities reshape the economy. In the second, AI development becomes substantially automated while humans still set research direction, meaning 100-person companies could do the work of 100,000-person organisations. In the third, AI systems achieve full recursive self-improvement and begin designing their own successors. Anthropic says it does not have "good intuitions" for what that third scenario looks like. But it offers one observation: even recursive intelligence cannot speed up everything. It cannot learn what a drug does over decades of use, hold elections sooner than a constitution dictates, or turn a stranger into an old friend in a weekend. The felt pace of this future, for most people, would still be set by the bottlenecks. The company's growing enterprise push makes the timing of this paper notable. Anthropic is simultaneously selling Claude as a productivity revolution and warning that the trajectory it enables could require a global emergency brake. Whether that tension is principled transparency or strategic positioning depends on what happens next.

Anthropic says AI could soon create more advanced versions of itself

Anthropic outlined the warning in a new blog post from its research-focused Anthropic Institute. The company said the industry may move toward "recursive self-improvement" sooner than many governments and institutions expect. The concept describes a future where one AI model develops the next version of itself. Researchers still guide the process today. However, Anthropic said AI already handles a growing share of coding, debugging, and technical research inside the company. Faster AI development Anthropic pointed to internal data showing how rapidly AI tools now contribute to software engineering work. The company said Claude-generated code accounts for more than 80% of the code merged into Anthropic's systems as of May 2026. Before the launch of Claude Code in early 2025, that figure sat in the low single digits. The company also said engineering productivity has surged alongside those changes. Anthropic engineers now merge roughly eight times more code per day than they did in 2024. Jack Clark, Anthropic's co-founder and head of policy, said the company wants lawmakers and institutions to understand what may come next. "We've always found that the best thing to do is to socialize the concept and basically give people a sense of what's coming," Clark said in a release. Clark added that AI progress appears to be accelerating instead of slowing down. He said the shift could drive major gains in medicine, science, and other technical fields. Benchmarks moving rapidly Anthropic also highlighted public benchmarks that track AI performance across software engineering and scientific research tasks. The company said AI systems now complete increasingly complex assignments over longer periods without human intervention. Anthropic claimed the length of tasks models can reliably handle has doubled roughly every four months.

Anthropic (Sorta) Calls for Pause on AI Development. You Should (Sorta) Take It Seriously

Anthropic is urging a unilateral slowdown on the development of new AI models until rigorous safeguards can be put in place -- with a few caveats. In a blog post published Thursday, the company cited its own internal data as evidence that modern AI systems are nearing the point of "recursive self-improvement" -- i.e., being able to refine their capabilities without a human in the loop. "AI that can build itself would be a major development in the history of technology -- one that could bring enormous good for the world in science, healthcare, and beyond," the post, which was written by company cofounder Jack Clark and Anthropic Institute lead Marina Favaro, reads. "But full recursive self-improvement also might increase the risks of humans losing control over AI systems." The post goes on to suggest that the best way to steer clear of such risks could be a worldwide pause akin to Cold War-era disarmament and non-proliferation pacts, in which countries like the U.S. and Russia agreed to limit their nuclear weapons programs to avoid what came to be known as Mutually Assured Destruction (MAD). "If it were possible to effectively slow the development of this technology to give ourselves more time to deal with its immense implications, we think that would likely be a good thing," the authors wrote. If this all sounds familiar, you're not hallucinating. Back in early 2023, nonprofit organization the Future of Life Institute published an open letter calling on all frontier AI labs to enact a six-month pause on the development of powerful new models. It was signed by Elon Musk, pioneering AI researcher Yoshua Bengio, Apple cofounder Steve Wozniak, and tens of thousands of others. While that letter and its luminous roster of signatories did much to kickstart the public conversation around the risks of an AI industry driven by blind capitalist logic, no firm policies were ever enacted to actually make that six-month pause a reality. Just a few months after it was published, Musk publicly launched his own AI startup to compete with OpenAI, Anthropic, Google, and the other big labs. A crossroads It's worth noting the extremely hedged language that Anthropic uses in its blog post: If it were possible...we think...likely.... It reads as more of a suggestion than a fully committed call to action. That could have something to do with the crossroads at which the company now stands. Since its inception, Anthropic has carved out a position for itself as a kind of moral conscience for the industry, the one big player that's devoted to building AI cautiously, and if necessary, slowly, to prevent the kind of rogue AI scenario that sci-fi authors, philosophers, and many leaders within the big AI lab themselves have long feared. That safety-oriented approach has clearly paid off. While its Claude chatbots still trail behind OpenAI's ChatGPT in app downloads, its tools have become massively popular among web developers and enterprise customers, both of which are key demographics for the industry. That's one of the big factors behind its recent eclipsing of OpenAI as the world's most valuable startup. But it also has a highly anticipated IPO coming up, after which point it will be legally required -- like all publicly traded companies -- to do what's best for its shareholders. And at least at this point in the AI boom, stepping on the brakes isn't widely viewed as a winning long-term financial strategy. Critics of the slowdown argument usually say that given the huge financial and geopolitical stakes of the AI race, it's highly unlikely that the U.S., China, and other countries would agree to pause development, and even if they did, it'd be impossible to enforce such a pause. Even worse, if U.S. developers all agreed to pause, it could leave the door open for less ethically minded developers elsewhere to take the lead, to everyone's detriment. As Clark and Favaro put it in their post, "if a slowdown simply lets the least cautious actors catch up technologically, it could leave everyone less safe." In order to be effective, Clark and Favaro argue, a slowdown would require not just unilateral agreement among AI developers around the world, but also some kind of global mechanism that could verify that each lab has in fact paused development, and that would trigger an alarm if and when the pause was violated. This raises more questions than it answers, since it's not clear what such an oversight system might look like in practice. Again, Cold War-era nuclear agreements can provide a rough template, but it's by no means a one-to-one match. "Training runs are far easier to conceal than missile siloes," the post acknowledges. (It should also not be forgotten that despite postwar international agreements against the proliferation of nuclear weapons, nuclear powers like the U.S., China, and Russia have once again been building up their nuclear arsenals.) The new blog post noted that in the coming months, Anthropic plans to meet with policymakers, its competitors in the industry, and other stakeholders to discuss these challenges, and hopefully work towards building oversight systems that would make a global pause actually implementable. Again, possibly with an eye towards its future shareholders, the company gave what feels like a halfhearted commitment: "If such systems existed, we expect that we would slow down or temporarily pause, if other developers at or near the frontier also did so in a verifiable manner."

Anthropic urges a coordinated, verifiable pause for frontier AI

The company says frontier developers need a coordinated, verifiable way to slow or stop if AI starts improving itself faster than the risks can be managed. The scenario Anthropic is worried about is one where the technology stops waiting for permission. On Thursday the company argued that frontier AI developers should build a coordinated, verifiable mechanism to slow down or temporarily pause development if advanced systems begin improving themselves faster than society can manage the consequences. The proposal is less a product announcement than a request that the industry agree on a brake before it needs one. The trigger Anthropic names is recursive self-improvement, AI systems capable of meaningfully accelerating their own development. That capability "would be a major development in the history of technology," the company said, but full recursive self-improvement "also might increase the risks of humans losing control over AI systems." As a marker of how far automation of its own work has already gone, Anthropic said that as of May, more than 80% of the code merged into its codebase was written by its model, Claude. The argument's sharp edge is about coordination rather than caution. A unilateral pause by one company would be easier to enact, Anthropic conceded, but would mostly just hand leadership to whoever kept going, shifting the frontier rather than slowing it. A pause that meant anything would require agreement among "multiple well-resourced labs" at the technological frontier, plus rules on what conditions would trigger or lift it and who would oversee the whole arrangement. The self-improvement worry is not hypothetical hand-waving on Anthropic's part; it points to its own operations as evidence. If a model already writes the overwhelming majority of the code that builds the next model, the loop between a system and its own improvement is no longer theoretical, merely partial. Anthropic's argument is that the loop tightens from here, and that the time to agree on a brake is while it is still partial rather than after it closes. That is the hard part, and Anthropic does not pretend otherwise. A verifiable pause implies labs able to confirm that rivals have actually stopped, agreed thresholds for what counts as too fast, and some body with the standing to call it. None of that exists today, and the companies that would have to participate are direct competitors in a market where being first has been the entire point. Anthropic's answer is to start talking. In the coming months, the company said it plans to convene discussions with policymakers, researchers, civil-society groups, and other AI firms to work through how to manage risks like recursive self-improvement and how to improve coordination mechanisms. It is positioning itself as convener of a conversation it wants the rest of the industry inside. The move fits a pattern for a company that has built its brand on flagging the dangers of the thing it sells. The obvious objection writes itself: a lab proposing the industry agree on when to stop is also a lab that keeps building until it does. Whether competitors treat the proposal as a genuine coordination problem or as a rival's attempt to set the terms is the question the coming months will answer. For now Anthropic has put a brake on the table. Nobody else has agreed to reach for it.

Anthropic's bizarre call for everyone to slow down on AI is a pipe dream -- here's why that will never happen

Are they fighting an existential threat or channeling Don Quixote? CNN's Wolf Blitzer leveled his inquisitive gaze at me and asked the biggest question in tech, "Is AI an existential threat to humanity?" I smiled confidently and assured Wolf, "No, I don't think it is." But just days after that TV interview, we have, it seems, ample evidence that one of the industry's leaders and core AI developers, Anthropic, perhaps thinks otherwise, and is now urging not just caution, but a slowdown. Anthropic is suggesting that all AI developers, policy makers, and others join hands and agree to pause or slow down frontier model development so that we can get ahead of the dangers of recursive development. In a lengthy blog post, Anthropic executives explain that misalignment between the needs and desires of humans and whatever compels AI to improve itself could result in us losing control of them. It's a scary thought, and one I hadn't entirely considered during my short interview with CNN. I mean, I get the concern now held by both everyday people and Anthropic. In the chat, I tried to explain that disruption and danger can feel similar, especially when the former is moving so fast that it can feel reckless, or at least beyond our comprehension. I have been arguing for tech and AI regulation for years. It seems like something almost everyone agrees on, but no one can figure out how to broadly implement it. Instead, we get piecemeal bits from local municipalities and Executive Orders. Slowdowns or pauses are not necessarily equivalent to regulation, and what Anthropic is suggesting is something different. It wants some sort of global agreement across everyone working on frontier models to create a mechanism for delimiting AI development, and maybe even a large red 'Stop all work' button that we can hit when signs of imminent danger to humans arise. Any rational person would break out laughing at this point. I'm not chuckling at the intention, but at the sheer impossibility of the ask. I'd explain why this won't work, but Anthropic does a fair job of it in the very same blog post: A meaningful slowdown or pause would require multiple well-resourced labs at or near the frontier, in multiple countries, agreeing to stop under the same conditions. It would also require that each can verify that the others have actually stopped. Due to the unique characteristics of AI systems, the detectability... element of this arms control problem is much more challenging than with other technologies. Training runs are far easier to conceal than missile silos, their inputs are general-purpose, and the incentive to defect quietly is enormous, because whoever continues while others pause could inherit the lead. A credible pause also has to specify what triggers it, what lifts it, and who adjudicates. Anthropic notes that while nuclear non-proliferation was easier to monitor around the globe because, well, it's hard to hide when you're building missile silos, AI frontier model development can be hidden on inscrutable systems where no one knows how you're training and testing the models, let alone the output. If we switch the the resursive delopment model, then AI model building is now a black box. Anyone could stand outside the AI dev center and confidently say they "have no knowledge of AI development." Which does not mean it's not happening, just that they have no input on its course. Also, what if Anthropic somehow got Meta, OpenAI, Perplexity, Microsoft, and others to agree to pausing some work? China certainly wouldn't agree, and then our greatest fears are realized: The US falls way behind while China delivers world-altering models -- and we never catch up. Anthropic knows this, and yet it's suggesting this global agreement. Some believe it's doing it so that others will pause while it continues, or that its current apparent lead with powerful models like Mythos might be preserved. I don't believe that. I just think they're all standing there, watching Claude build its latest models and wondering, "What have we done?" It's a level of introspection that might be missing from, say, OpenAI, Meta, and others, but that doesn't necessarily serve a purpose. Instead of trying to get everyone to agree to take a beat, how about we all agree that humans are always in the loop? Sure, it'll be self policed, but the more we inject ourselves into the development and dissemination of the models, the more we ensure that future frontier models do not rise to the level of existential threat. Still, I wonder if I need to have another chat with Wolf. Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds.

Anthropic urges 'temporary pause' on AI development to discuss risks

Announcement that 'policymakers' need to be convened by US firm viewed as marketing ploy by some experts Anthropic has floated the idea of a worldwide "temporary pause" on AI development - and said it was going to convene "policymakers" to discuss the dangers of advanced AI - in its latest release touting the capabilities of its products. In a long post on Thursday, Anthropic detailed the progress of its AI model, Claude, towards "recursive self improvement" - that is, being able to make better and more powerful versions of itself. Recursive self-improvement is a bugbear of AI safety researchers, viewed as the key step for AI to become superintelligent and therefore unleash widespread consequences on humanity. The idea features heavily in the widely read AI 2027 doomsday scenario of last year, which imagines AI agents designing more and more intelligent versions of themselves, one of which eventually kills all of humanity with a bioweapon in order to make room for more datacentres and solar panels. Anthropic's post notes a "trend" of increasing capability in Claude which, "taken far enough and given enough compute ... points to an AI system capable of fully autonomously designing and developing its own successor". This, Anthropic said, may increase the risk of "humans losing control over AI systems". To deal with this, Anthropic proposed to organise conversations where "policymakers, researchers, civil society and other AI companies can help answer some of the questions this piece raises". The news comes alongside a separate report, from the Financial Times, that the US AI company has embedded engineers inside the National Security Agency despite a legal battle with the Pentagon over the use of its tools. The engineers are reportedly helping the NSA use Anthropic's model, Mythos, for offensive cybersecurity operations. If calling for a worldwide conversation on AI risk is in contradiction with supporting a US spy agency to - potentially - attack Iran and China with cyberweapons, neither development is "surprising" given the AI company's past actions, said Steven Murdoch, a professor at University College London. "Anthropic might give the impression of being warm and fuzzy, but their definition of AI safety is narrow. Supporting US authorities in the development of offensive capabilities has never been something they have spoken against," he said. Murdoch said that Anthropic's post did not offer evidence of any step changes in the progress of AI capabilities. "It is true that there's some evidence that AI capabilities have increased and continue to increase with no limits becoming immediately clear," he said, but he added: "I don't think anything has fundamentally changed today that has caused Anthropic to publish this article." The advances that Anthropic appears to detail in its post do not amount to AI recursively improving itself - at least, not yet. Instead, the company reports that a substantial amount of the work done on improving its AI systems is now done with AI. Claude is good at "running experiments", it says, or at least speeding certain sections of code. Like other AI systems, Claude appears to be improving at tackling more challenging tasks. Anthropic described it "steering research" and "proposing its own experiments", although these accomplishments appear to have taken place within strict confines, and to be confined to coding-related tasks. The quality of code written by AI is also improving, said Anthropic: "As of May 2026, more than 80% of the code we merge into Anthropic's codebase was authored by Claude." Murdoch said that Anthropic's call for a "temporary pause" on AI echoed other proposals on AI safety the company had made throughout the years - as did its plan to engage policymakers. "It's a reminder of what they are concerned about, and have been concerned about for many years." "I'm sure the attention is welcome, but again this isn't a new thing. Anthropic have been trying to get the attention of policymakers since they were founded." Two months ago, Anthropic announced - but declined to release - Mythos, an AI model that it implied was too powerful to unleash upon the public, owing to cybersecurity concerns. The announcement led to widespread hubbub and earned the attention of US treasury secretary, Scott Bessent, and MI5. Some experts, however, suggested that there could be more hype behind Anthropic's Mythos announcement than substance, especially given the vagueness with which the company described some of Mythos's capabilities. Heidy Khlaaf, chief AI scientist at the AI Now Institute, called the announcement of Mythos "a marketing post." Anthropic this week filed for an IPO that could value the company at $1tn. The company was approached for comment.

Anthropic warns AI could soon help build its own successors

Why it matters: "Recursive self-improvement," a process in which AI systems build, test and improve themselves, is a phenomenon which may come sooner than expected, Anthropic says that its research shows. Driving the news: Anthropic warns that AI is no longer just changing how people work, it's also beginning to change how AI itself gets built. * New data from the company suggests that frontier models have accelerated coding, debugging and research. * That is likely to create a feedback loop in which AI systems create even more sophisticated successors. What they're saying: "We've always found that the best thing to do is to socialize the concept and basically give people a sense of what's coming," Anthropic's Jack Clark said in an interview with Axios. * "The big story here is what we see are indications that, contrary to some popular opinion, AI progress is going to speed up in coming years rather than stay the same, or diminish." * Clark said that it is especially promising for progress in science and medicine, but requires planning for its impact on AI itself and how it fits into existing work in those industries. The company wants lawmakers in the loop on the topic before they start hearing about "recursive self improvement" in earnest, Clark said. * "As organizations, and eventually probably as societies, we need to figure out the tools to validate and verify that the stuff being done by these AI systems is correct and is aligned with human intentions aligned with a thriving society," he said. The big picture: Improvements in the Claude chatbot have turned into improvements in AI coding agents, which have turned into improvement in autonomous agents. * Recursive self improvement is the likely next step, Clark argues in the post: "In the near future, AI systems could become capable enough to autonomously design, build and train more capable successors on their own." * "If that happens, each new version of Claude could be built by the version before it, without human involvement." OpenAI has published its own concerns and findings about "recursive self-improvement" as well. In a December 2025 blog it described it as a potentially dangerous phenomenon if researchers don't share information about it. What we're watching: Anthropic plans to engage lawmakers about recursive self-improvement in the coming months. The bottom line: AI that builds itself is on the horizon, and AI labs are saying they're not sure what the impact on the world will be -- but they feel a need to warn everyone about it.

Anthropic Scared, Calls for Global Freeze on AI Advances

Can't-miss innovations from the bleeding edge of science and tech Anthropic is calling for a global "pause" on AI development, claiming that the technology is nearing a point where it can spiral out of human control. In a lengthy blog post published Thursday, the world's most valuable AI startup made the case that its Claude family of models were on the path to achieving "recursive self-improvement," or the ability to improve themselves on their own, a key hypothetical tipping point that could lead to the creation of powerful AIs capable of operating outside human interests and harming society. We're not at that point yet, Anthropic stresses, but it "could come sooner than most institutions are prepared for." "We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology," the company wrote in the post. It added that "a meaningful slowdown or pause would require multiple well-resourced labs at or near the frontier, in multiple countries, agreeing to stop under the same conditions," and admitted that this would be challenging to enforce. "Training runs are far easier to conceal than missile silos," it wrote. For Anthropic to call for a pause now is convenient. In the past few months, it leap-frogged OpenAI to become the world's most valuable AI company with a $1 trillion valuation, and its models are now generally viewed as the best in the field, especially at coding tasks. If the industry were to hit the brakes now, it would cement Anthropic's dominance. Not everyone was buying Anthropic's claims. Prominent AI critic Gary Marcus called the company's lengthy post a "bait and switch." "Anthropic is trying to strike terror into everyone's hearts ('full recursive self-improvement also might increase the risks of humans losing control over AI systems') but all they have really shown is just faster coding -- entirely under human control," Marcus wrote on his Substack. "A faster coding tool will probably not end the world." Anthropic has long tried to paint itself as the ethical and deeply concerned adult in the room. A cornerstone of its mythology is that CEO Dario Amodei abstained from unleashing a revolutionary AI model back in 2022 because he was too concerned about safety, and let OpenAI get all the glory when it released ChatGPT months later instead. Two months ago, in a rehashed sequel to this foundational company lore, Anthropic announced a new model called Mythos -- but made a show of not releasing to the public, claiming it was powerful enough to break into "every major operating system and every major web browser." But its act is ringing hollower than ever. Earlier this year, Anthropic famously clashed with the Pentagon over concerns that its AI systems could be used in autonomous weaponry and in the mass surveillance of US citizens. Later, it emerged that Claude was being used to help select strike targets in Iran. Amid its blowout with the military, Anthropic also dropped a safety pledge that was arguably the venture's entire raison d'etre: to stop training an AI system if it couldn't guarantee it had proper safety guardrails in place. Further underscoring Anthropic's hypocrisy, University College London professor Steven Murdoch cited recent reporting from the Financial Times revealing that Anthropic is helping the US National Security Agency use its Mythos model so it can wage cyberwarfare against potential enemies like China and Iran. "Anthropic might give the impression of being warm and fuzzy, but their definition of AI safety is narrow," Murdoch told The Guardian. "Supporting US authorities in the development of offensive capabilities has never been something they have spoken against." Regardless of whether Anthropic genuinely thinks it has a remotely realistic shot at pulling off a global pause -- or if this is yet another ploy to boost its safety-minded image -- it's vowing to pursue further action. "In the coming months, we will organize conversations where policymakers, researchers, civil society, and other AI companies can help answer some of the questions this piece raises, especially around full recursive self-improvement and how to create better options for coordination and deliberation," the company wrote. "We'll publish what comes out of it."

Anthropic warns AI could soon build itself without human involvement -- and urges a global pause on development | Fortune

Anthropic has published a new account of how quickly its AI models are advancing, warning that the technology may soon be capable of improving itself without meaningful human involvement. Just as the AI lab, valued at almost $1 trillion, prepares to go public, it's also urging an industry-wide pause in AI development. In the now-viral blog post on Thursday, authors Marina Favaro and Jack Clark argue that AI development at Anthropic has already shifted dramatically: more than 80% of code merged into the company's codebase is now written by Claude, and engineers are shipping roughly eight times as much code per quarter as they were before 2025. The authors say this trajectory is heading toward "recursive self-improvement" -- a process in which AI systems autonomously design, build, and train their own successors, without humans driving each step. The authors warn that while this threshold has not yet been crossed, it "could come sooner than most institutions are prepared for," and that if it arrives without adequate safeguards, it could make it significantly harder for humans to maintain meaningful control over AI development. The company also took the time to advocate for a pause on AI development but only if "multiple well-resourced labs at or near the frontier, in multiple countries, agree to stop under the same conditions." Presumably Anthropic is referring to competitors including OpenAI, Google, xAI, and Meta. Most of these labs are currently engaged in a high-stakes race to develop ever more powerful AI models, and, with IPOs looming over three of the main labs, it seems unlikely that these companies will come together to agree on a coordinated pause on AI development. The timing is raising some eyebrows. Just last week, the AI filed confidential paperwork to prepare for an IPO. In the last month, it also leapfrogged OpenAI to become the most valuable AI lab at a $965 billion valuation. While Anthropic has long presented itself as more safety-conscious than the major AI labs, the timing -- just before what could be one of the largest tech IPOs in history -- has some observers questioning whether it's also a way to stir up hype before the company's public debut. Critics have previously accused the company of using safety rhetoric as a form of competitive positioning. Trump adviser David Sacks has previously accused Anthropic of running a "regulatory capture agenda" designed to slow rivals under the guise of responsible AI development -- a characterization the company rejects. The AI lab is also in a fierce competition with old rival OpenAI, with both labs racing to go public this year in an attempt to seize what could be a first-mover advantage with investors. It's also worth noting that Anthropic, despite advocating for a pause on AI development, recently dropped a key safety pledge that promised to do something similar. In February, TIME reported that Anthropic had overhauled its Responsible Scaling Policy -- scrapping its central commitment to never train an AI system unless it could guarantee in advance that its safety measures were adequate. Chief science officer Jared Kaplan said at the time the company felt it "wouldn't actually help anyone" to stop training models unilaterally while competitors pressed ahead. The company argued the overhaul was a pragmatic response to a changed political and competitive landscape, rather than a capitulation to market pressure. The new policy commits to transparency and matching rivals' safety efforts; still, it leaves Anthropic significantly less constrained by its own rules than before. Thursday's post does make it clear that, whatever its motivation, Anthropic believes the window for meaningful deliberation on AI safety is narrowing. The company says it plans to convene policymakers, researchers, and civil society in the coming months to work through the questions recursive self-improvement raises. "The window to investigate the questions together is here," the authors wrote. "And people outside AI companies should be involved in this deliberation."

Anthropic Is Helping the NSA Hack China. It Also Wants Everyone to Pause AI

Both landed as Anthropic files for an IPO that could value it above $1 trillion. Anthropic has placed about six engineers inside the National Security Agency to help deploy Mythos -- its most capable AI model -- for offensive cyber operations, the Financial Times reported Thursday. The engineers are forward-deployed staff, customizing the model for specific applications. One source told the FT it could be useful for infiltrating networks in countries like China and Iran. Whether those engineers are involved in active operations isn't confirmed. What is: Mythos is the same model Anthropic has declined to release publicly, citing misuse risk. The company limited it to vetted partners through Project Glasswing -- a restricted coalition that includes Microsoft, Apple, and Amazon. Anthropic is also suing the Pentagon. In late February, Defense Secretary Pete Hegseth designated the company a supply-chain risk -- a label historically reserved for foreign adversaries like Huawei -- after a $200 million contract collapsed. The sticking point: Anthropic refused to let the DoD use Claude for fully autonomous weapons or domestic mass surveillance. The NSA contract was exempt from that ban. A California judge blocked the blacklisting as an apparent First Amendment retaliation. A D.C. appeals court denied Anthropic's bid to halt it while litigation plays out. The NSA kept using Mythos the whole time, according to the FT's reporting. How to stop AI that builds AI On the same day the NSA story broke, Anthropic's internal research institute published "When AI Builds Itself," a look at how far Claude has come at automating its own development. In it, the company argues for essentially a global moratorium in the AI arms race -- and even likened it to Cold War-era nuclear treaties struck between the United States and Russia. To understand why, the company provided this bit of context: Claude now writes more than 80% of the code merged into Anthropic's production codebase -- up from low single digits before Claude Code launched in early 2025. Engineers ship roughly eight times as much code per day as they did in 2024. The report's authors -- Anthropic Institute lead Marina Favaro and co-founder Jack Clark -- argue this trajectory is heading toward what they call recursive self-improvement: AI systems that autonomously design, build, and train their own successors, with humans playing a diminishing role at every step. In a visual representation, the researchers show a timeline in which the first way to use AI at work as humans prompting the computer to get a result, with increasing automations ending in AI Agents prompting subagents until the result is achieved, no humans involved. The sharpest data point they cite: In April, Claude agents were handed an open AI safety problem -- whether a weaker model can reliably supervise a stronger one -- and left to run it. Two human researchers over about a week recovered 23% of the performance gap between the models. The agents recovered 97%, over 800 cumulative compute hours. Humans set the question. The agents designed every experiment. It's the first published case of Claude exercising research judgment, not just executing tasks someone else specified. That's the line Anthropic is worried about crossing. Once AI chooses which experiments are worth running -- not just runs them -- humans lose the last meaningful role in the development loop. Small misalignments visible in today's models could compound across self-improving generations until nobody can correct them. Their proposed fix is a verifiable global pause -- multiple frontier labs halting simultaneously, with independent verification that everyone actually stopped. Anthropic said it would join one. A unilateral slowdown, they acknowledge, just hands the lead to whoever kept going. We've seen this movie before. The labs building AI are the same ones warning how dangerous AI is. However, AI is the most profitable business of the decade, so nobody wants to stop -- not even the ones warning about AI. Back in 2023, over a hundred big names in the AI researching community signed an open letter asking for a global effort to mitigate the risk of extinction that AI development intrinsically has. A few months before that, another open letter demanded OpenAI pause advances on ChatGPT due to its dangerous nature. Nobody stopped after the 2023 open letter. OpenAI didn't. Anthropic didn't. The Pentagon's deadline to drop Claude from its systems falls in August, around the same time Anthropic's IPO is expected to move its finances into public view.

Anthropic calls for global pause in AI development before humans lose control

Anthropic calls for global pause in AI development before humans lose control Anthropic PBC has sounded the alarm on the rapid pace of artificial intelligence development and is calling on the world's top AI laboratories to consider slamming on the brakes. In a blog post today, the company warned that AI systems are getting close to the point where they may soon be able to improve themselves without human oversight, and said reaching this threshold could lead to massive societal disruption. The startup's solution is for a globally coordinated agreement to temporarily pause or at least slow down the pace at which new frontier models are being developed. In the blog post, Anthropic's head of internal research Marina Favaro and head of policy Jack Clark argued that a pause would provide the world with the breathing room it needs to adjust to the pace of AI's rapid growth. The two authors said that model advances appear to be getting closer to a theoretical concept known as "recursive self-improvement," which refers to AI systems that are capable of independently improving themselves and expanding their own capabilities, simply by writing their own code. They acknowledge that this hasn't happened yet and isn't necessarily inevitable, but they also warn it could happen - much sooner than anyone is prepared for. According to Clark, there's reason to believe that some models could be capable of recursive self-improvement within just two years. "We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology," the authors suggested. There are reasons to listen to Anthropic when it makes such a dire warning. The company recently concluded a massive, $65 billion funding round that brought its valuation close to $1 trillion, and this week it filed paperwork with the Securities and Exchange Commission stating its plans to list on the public markets. It has emerged as one of the front runners in an intense competition for market supremacy with ChatGPT developer OpenAI Group PBC, which is also expected to launch an initial public offering soon. However, Anthropic, which has carefully cultivated an image of erring on the side of caution and advocating for public safety, has also been accused of leveraging policy to try and slow down the advances of its competitors. David Sacks, a venture capitalist and an informal adviser to U.S. President Donald Trump, has previously accused the company of running a "regulatory capture agenda." He argues that the company uses fear-mongering in a calculated way to encourage heavy-handed regulations that would essentially ban lower-cost open-source models in order to boost the popularity of its own proprietary algorithms. Whether or not such a slowdown is even feasible is highly debatable. Enforcing a global pause on AI development would likely be near-impossible. Anthropic conceded that it would likely require something similar to the Cold War-era treaties that slowed down the pace of nuclear weapons proliferation, but said that it's a lot easier to mask AI training runs than it is to conceal missile silos. Unless some kind of ironclad verification regime is put in place, with cooperation from countries like China, any pause would risk creating a scenario where competing nations simply ignore the pause and take the lead in AI development. Regardless, the authors said they intend to push forward with the idea. Anthropic Institute, which is the company's in-house research organization that's dedicated to understanding and shaping AI's impact on the world, will continue collaborating with others on ways to "help build systems that a credible slowdown or pause would require." The company said it will try to organize a broader debate with policymakers and AI researchers to try and answer questions about the likelihood of self-recursive improvement and what a verification system might look like. "In the absence of a coordinated, global slowdown, we are left with the current situation: powerful technology being developed at breakneck speed by a variety of actors in a variety of countries, locked in a competition with one another where commercial and geopolitical rivalries are drowning out the larger existential-to-the-species aspects of the technology being built," the authors wrote.

AI Is Already Developing AI, Says Anthropic -- And Humans May Be Slowing Things Down

Anthropic argues AI is already helping build future AI systems and could eventually contribute to designing its own successors. AI has become so effective at writing code and researching that the biggest constraint on developing new AI systems may now be the humans overseeing them, according to a new study by Anthropic. In its report "When AI Builds Itself," published Thursday, Anthropic argued that Claude is already helping build future AI systems by writing code, running experiments, and assisting with research -- a trend the company says could eventually lead to recursive self-improvement, where AI systems help design their own successors. Claude now authors more than 80% of the code merged into its codebase, Anthropic said, and has helped engineers increase code output roughly eightfold since 2024. "Before Claude Code launched in research preview in February 2025, this number was in the low single digits," Anthropic wrote, adding that the shift also shows up in the amount of output per engineer. "Lines of code merged per engineer per day stayed constant through Anthropic's first four years (2021-2024), then began to climb upward in 2025 when Claude began to run code rather than just suggesting it for an engineer to copy and paste." Anthropic said the future could unfold in several ways: AI progress could slow, humans could remain in charge while AI automates much of the work, or AI systems could eventually begin improving their own successors. "Taken far enough, and given enough compute, that trend points to an AI system capable of fully autonomously designing and developing its own successor," Anthropic wrote. "This is called recursive self-improvement. We are not there yet, and recursive self-improvement is not inevitable. But it could come sooner than most institutions are prepared for." The company said it's too early to know which outcome is most likely, but argues that AI is already helping build AI, and acknowledged that lines of code are an imperfect measure of productivity. "None of this guarantees recursive self-improvement is on the horizon," Anthropic later wrote on X. "It's not yet clear that Claude is capable of research judgment -- of choosing the right problems to work on." The report comes as AI companies increasingly position their models as research collaborators rather than simple chatbots. Still, Anthropic said the increase in code output reflects a broader acceleration in software development driven by increasingly capable AI agents. Last month, Anthropic upgraded its flagship Claude model to Opus 4.8, continuing a steady stream of releases aimed at improving coding, reasoning, and autonomous task performance. At the same time, rival developer OpenAI has pursued a similar strategy with its frontier models, launching GPT-5.5 and GPT-Rosalind in April. In May, Google announced Gemini Spark, a personal AI agent that doesn't wait to be asked. It manages tasks across apps, flags items that need attention, and finishes jobs in the background. The report also comes as Anthropic has increasingly emphasized AI systems capable of operating with greater autonomy as it prepares to go public. In recent months, Anthropic has showcased advances in coding, agentic workflows, and long-duration task performance, while touting Claude Mythos' ability to identify software vulnerabilities and conduct complex cybersecurity research. "Humans play a substantially diminished role in their development, likely moving most of our effort towards oversight, validation, and verification of an expanding 'virtual lab' run by AI systems," the company said. "We expect that systems capable of automated AI research and development would have skills that would transfer to the rest of science, allowing them to begin to revolutionize other fields."

AI Could Soon Train and Improve Itself Anthropic Says

Companies have been developing AI very quickly to stay ahead of the market, but Favaro and Clark argue that a slowdown would allow more time to deal with the technology's implications. US-based AI firm Anthropic warns AI development is advancing at a pace that could soon see agents building, training and improving themselves without human input -- recommending a slowdown in development. In a blog post published Thursday, Marina Favaro, lead at the Anthropic Institute, and Anthropic co-founder Jack Clark said agents can already run code themselves, delegate hours of work to other agents and could be on the cusp of taking over completely. "For most of AI's history, humans drove every step in its development cycle. But at Anthropic, we are delegating a growing share of AI development to AI systems themselves, which is speeding up our work," they said. "Taken far enough, and given enough compute, that trend points to an AI system capable of fully autonomously designing and developing its own successor," Favaro and Clark added. AI development is advancing at a pace that could lead to agents improving without human input. Source: Anthropic There are concerns over what could happen if AI is able to become smarter on its own. In December, OpenAI said it is researching how to safely develop and deploy increasingly capable AI, including AI capable of recursive self-improvement. "We want these systems to consistently follow human intent in complex, real-world scenarios and adversarial conditions, avoid catastrophic behavior, and remain controllable, auditable, and aligned with human values," it said. The company is also hiring a researcher for recursive self-improvement preparedness, which forms part of its Safety Research team. AI model improvement has been roughly doubling every four months, rather than every seven months, according to Favaro and Clark. The role of humans is narrowing at each step, with Anthropic's Claude model authoring around 80% of the code merged into Anthropic's codebase. "We are not there yet, and recursive self-improvement is not inevitable. But it could come sooner than most institutions are prepared for," they said. "Once human- and AI-authored code quality reach parity, humans will stop writing code entirely and shift to only reviewing it. But if they can't review code as quickly as Claude can generate it, human review will become the bottleneck to AI development," they added. Favaro and Clark also said that slowing development to allow more time to address its "immense" implications would be ideal. In April, Anthropic ruled out releasing its AI model, Claude Mythos, to the public over concerns about the threat to global cybersecurity. Claude Mythos was able to easily create software exploits, leading Anthropic to rule out a public release for now. Source: Anthropic At the same time, a group of tech leaders, including some from Anthropic and OpenAI, released an open letter on Thursday, urging lawmakers to enact stronger guardrails around the technology over concerns it could be used to overcome "knowledge barriers" that have historically prevented bad actors from creating biological weapons. "We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology," Favaro and Clark said. "But if a slowdown simply lets the least cautious actors catch up technologically, it could leave everyone less safe. Without a global coordination mechanism, companies and governments will have to make difficult decisions about safety while under competitive and geopolitical pressures." AI agents are becoming increasingly popular, including among crypto users. Some crypto executives have speculated that AI agents settling transactions could drive adoption and transaction volumes. Circle CEO Jeremy Allaire predicted in January that billions of AI agents would operate on users' behalf within five years. Crypto investment firm Keyrock reported last month that AI agents settling payments went from concept to reality in the past 12 months, with $73 million settled across 176 million transactions.

Anthropic Urges a Way to Pause AI Development as Risks Grow With the Tech Advances

Anthropic is proposing that the world's top artificial intelligence companies come up with a coordinated way to pause development of advanced AI systems, warning the technology is improving so quickly there's a risk humans would lose control. The company behind the Claude chatbot said in a blog post Thursday that as cutting-edge AI gets increasingly faster at carrying out tasks, "it would be good for the world to have the option to slow or temporarily pause" its development. Anthropic said its internal research institute plans to explore the issue in collaboration with others and "take actions" to help build the systems for a credible slowdown or pause, without being more specific. AI models are getting faster, with rapid increases in how quickly they can carry out software tasks like coding on their own, the company said. Based on current trends and given enough computing power, an AI system could be able to design and develop its own successor, in what is known as "recursive self-improvement." Self-building AI would be a major technological milestone that would bring benefits in science, healthcare and other areas, Anthropic said, but it "also might increase the risks of humans losing control over AI systems." Some tech industry figures have long warned of such a scenario. Anthropic's post comes after a different warning this week from a team of researchers at the University of Toronto who showed how AI tools could be used to create a new kind of AI "worm" that adapts its hacking strategy as it spreads from device to device and takes over a vast computing network. "I think it's really important that people understand that it's not just the biggest, most powerful language models that pose the security concerns," lead researcher Nicolas Papernot said in an interview. The authors of the Anthropic post, company co-founder Jack Clark and Marina Favaro, head of the research institute, said the pause would be used to enable "societal structures and alignment research" to keep up with AI advances. Alignment is industry shorthand for making sure the technology matches human values and intentions. The proposed coordination would let advanced AI labs verify that global rivals have actually stopped or slowed their work, "and that a bad actor could not use the auspices of a coordinated slowdown to jump ahead in secret." The company said a coordinated global mechanism is needed because without it a slowdown in AI development could let the "least cautious" players catch up and add to pressure on companies and governments as they make tough choices about AI safety. Anthropic's post comes as the company itself is racing with ChatGPT-maker OpenAI to sell shares on the stock market, in an IPO that could value it at nearly a trillion dollars.

Anthropic just said humans could lose control over AI systems, and that should scare us all

Sara Heritage is a tech and gaming journalist, who's currently making her way up to Master Ball rank in Pokemon Champions. Bylines in IGN, GAMINGbible, The Gamer and more. You can usually find her tinkering with tech, or restoring old consoles, always with one of her 3 cats nearby. Come and talk with her over on Twitter @SHeritageJourno. * Claude might self-improve autonomously, risking loss of human control and safety. * Productivity gains flow to companies and shareholders, while AI fuels job cuts. * Overreliance on AI weakens critical thinking and kills the messy joy of creating. It's been 3 and a half years since ChatGPT opened the floodgates of Artificial Intelligence to mainstream users. Now, in just a few short years, the tech has developed so far that future versions of Claude could be continuously improved by Claude itself without the need for human involvement, according to a recent report released by Anthropic. Where does that leave us, as a workforce, and as a species that invented and relies on capitalism? ChatGPT's decline is real -- I tested it against Claude on 3 routine tasks, and it lost every time What happened, ChatGPT? We used to be cool. Posts 1 By Shimul Sood The future is full recursive self-improving AI In a recent blog post, Marina Favaro, leader of The Anthropic Institute, and co-founder Jack Clark discuss the potential of AI systems that advance themselves, known as "full recursive self-improvement." In simple terms, this means the AI can improve its abilities by learning, rewriting its code, or designing better versions of itself, all without human help. Predictably, they both claim that this self-improving AI technology could help us all in science and health care. The pair behind the tech also raise the risks the tech could pose to humanity: "Full recursive self-improvement also might increase the risks of humans losing control over AI systems. If systems are capable of fully building their own successors, the ways we secure them, monitor them, and shape their behavior all grow much more important". So what does this mean for AI? We've all seen what happens when AI start to feed on its own creations, such as when image generators use AI-generated images as references. This leads to odd results, like the well-known "piss filter," or when you ask AI not to change an image, and everything just gets more distorted. It's not clear how 'self-improving' AI would avoid this, or who decides what counts, or doesn't count, as improvement. This feels like the Dead Internet Theory in action. What's real anymore? Does it matter? Where do we fit in? Does anyone even want this? Who decides if a self-improving AI is actually helpful? One example from the blog post really worries me: Anthropic gleefully brags that its engineers now produce eight times as much code per quarter as they did between 2021 and 2025. Another question -- where does all that extra time go? It doesn't seem to benefit the workers. In fact, U.S. companies have explicitly cited artificial intelligence as a reason for roughly 87,700 planned job cuts this year alone. I've lost work thanks to budgets tightening and companies using AI for copywriting, so I can't tell you a single positive thing AI has done for me. Is the goal simply endless growth, no matter the cost? Companies keep pressing for more and more and more, until there's nothing left to give. When does efficiency actually help the people doing the work, instead of just the shareholders? This isn't science fiction anymore -- it feels more like the beginning of a horror movie. Why didn't I just use AI to write this article? It would have been faster. I could have published three AI-generated pieces in the time it took to write this one. You probably wouldn't have noticed the difference. I could've been more efficient, made more money for shareholders, and brought more eyeballs to our site. But I write because I have unique things to say; things I want to write! The joy in the creative process isn't the finished product -- it's in the drafts, exploring the different ways to express my thoughts, the joy of sitting here with a coffee and sharing with you, the reader. AI can't replicate that. Research shows that using AI tools is often linked to weaker critical thinking, partly because we let the technology do more of our thinking for us. Professor Eric So from MIT Sloan School of Management points out that we're handing over tasks our brains should handle to AI, which can make us too dependent. We managed without AI, and maybe it's time to push back. Subscribe for a newsletter that decodes AI's risks Curious about AI that can rewrite itself and what that means for jobs, creativity, and control? Subscribing to the newsletter gives you thoughtful, jargon-free coverage and analysis of AI developments and their real-world implications. Get Updates By subscribing, you agree to receive newsletter and marketing emails, and accept our Terms of Use and Privacy Policy. You can unsubscribe anytime. Reading the blog post, I can't help feeling a bit sorry for Anthropic. It reads as two people scrambling to stay relevant as the AI bubble shows signs of bursting. According to this report, Claude writes 80% of the code currently in Anthropic's codebase. If that's true, Claude -- do us a favor and delete the internet, I'm so tired of hearing about artificial intelligence.

AGI for all among OpenAI's ambitions for its next phase

OpenAI is entering a new phase to make advanced AI widely available and affordable. The company plans to build AI researchers to accelerate scientific discovery and provide every person with a personal AGI system. This move aims to boost productivity and economic growth. Meanwhile, rival Anthropic calls for a slowdown in frontier AI development due to self-improvement concerns. Giving every person on Earth access to a personal artificial general intelligence (AGI) system and building AI researchers that can help advance science are among OpenAI's key goals for the future, according to a blog post by chief executive officerSam Altman and chief scientist Jakub Pachocki. The company said it is entering a new phase focussed on making advanced AI abundant, affordable, and widely accessible, while ensuring that the technology remains safe and beneficial to society. The company stressed that its "first commitment is to build AI in service of humanity" -- the very principle that Altman's rival, Elon Musk, had accused OpenAI of abandoning in a lawsuit that ultimately did not go his way. AI's 'electricity moment' To explain its vision, OpenAI compared AI's rise to the spread of electricity in the early 20th century. While electricity initially solved practical problems, its real impact came from the opportunities it created as access expanded, enabling advances in industry, healthcare, communication, and living standards. "This is happening again with AI," Altman said in the blog post. "AI will soon be capable of extraordinary things. But the point is not the technology by itself. The point is what people can do with it." According to OpenAI, AI could help people learn new skills, navigate complex financial and legal decisions, start businesses, care for family members and contribute to scientific discoveries. OpenAI added that it wants to empower people broadly rather than see power concentrated among a small number of companies, governments or individuals, echoing what Pope Leo XIV said in his first encyclical, Magnifica humanitas. Humans remain central While OpenAI is optimistic about AI's potential, it stressed that increasingly capable systems must remain safe, aligned with human intentions, and subject to human control. "Entirely automating everything is not the future we want," the company said. "AI should help people pursue their goals, not become untethered from them." The blog argued that human judgment will become more important as AI systems grow more capable. People will still be responsible for setting goals, making trade-offs, and deciding what is worth pursuing. "A key long-term role for people will be deciding what is worth doing," OpenAI said. AI helping build AI One of the strongest points in the blog was OpenAI's belief that AI systems will soon play a major role in research itself. The company said AI-assisted research could become the biggest driver of technological progress within the next few years. To support this vision, OpenAI is working towards creating an automated AI researcher. "Our internal belief is that by March of 2028 we may have a significant fraction of our research being done by AI systems in tandem with our own researchers," the blog said. This is a system capable of accelerating and increasingly automating research while remaining steerable and accountable. The next chapter The blog outlines three goals for OpenAI's next phase. The first is building automated AI researchers that can accelerate scientific discovery and AI development. The second is accelerating economic growth by boosting productivity, innovation and scientific progress, while ensuring the benefits are widely shared. The third is providing every person on Earth with a personal AGI system that can help them achieve their goals. OpenAI said these ambitions mark the beginning of its third chapter. The first phase focussed on researching AGI, while the second centred on turning that research into products used by millions of people. The latest phase is about ensuring advanced AI becomes widely available and useful across society. Anthropic calls for a slowdown OpenAI's disclosure of plans to build automated AI researchers comes as rival AI firm Anthropic last week argued that the world should retain the ability to slow down or temporarily pause frontier AI development if advanced systems begin improving themselves too quickly. In a blog post titled When AI Builds Itself, the company warned that self-improving AI could reduce human oversight and make safety problems harder to identify and fix. Anthropic said any slowdown would need to be globally coordinated and verifiable, and plans to hold discussions with policymakers, researchers, civil society groups, and other AI companies. The debate comes just as both OpenAI and Anthropic filed draft papers for IPOs in the US.

OpenAI Calls For Global Oversight To Manage Advanced AI Development Risks

A blog post published by OpenAI is urging governments to create a global body that could monitor advanced artificial intelligence work and even push developers to ease off when risks start to outstrip safeguards. "One goal of such an organization should be to make it possible for the world to take coordinated action, including slowing frontier development when needed, so societal resilience, safety, and alignment can keep pace," the post states. This letter echoes a similar message from rival Anthropic issued last week in which the company argued that a worldwide pause could be beneficial if it could actually be implemented. "We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology," Anthropic wrote. Similarly, OpenAI pointed to AI systems contributing directly to AI research as a potential inflection point for how fast the field moves. "We believe that AI doing AI research will become the determining factor of the pace of progress within the next few years," Altman and Pachocki wrote. By March of 2028, OpenAI "may have a significant fraction of our research being done by AI systems in tandem with our own researchers," the company said. Altman and Pachocki described building an "automated AI researcher" as a major internal objective and also said OpenAI wants to "provide everyone on Earth with a personal AGI." Both OpenAI and Anthropic have recently submitted confidential draft registration statements on Form S-1 to the SEC. Neither firm has provided a timeline for its proposed IPO. Together, the three filings represent a historic convergence of AI and space-age technology hitting public markets simultaneously -- and a defining test of investor appetite for moonshot bets. This content was partially produced with the help of AI tools and was reviewed and published by Benzinga editors. Market News and Data brought to you by Benzinga APIs To add Benzinga News as your preferred source on Google, click here.

Anthropic wants AI development to slow down globally, warns humans could lose control otherwise

In a long article on its website, titled "When AI builds itself", Anthropic said it would be good to have the option to slow or temporarily pause frontier AI development, citing safety risks and loss of control over the models as they self-improve. Leading artificial intelligence (AI) company Anthropic, which is within kissing distance of a trillion-dollar valuation thanks to its AI systems, which have moved markets and changed the nature of white-collar jobs, now wants the technology to slow down. In a long article on its website, titled "When AI builds itself", Anthropic said it would be good to have the option to slow or temporarily pause frontier AI development, citing safety risks and loss of control over the models as they self-improve. The company claims that slowing down would be effective if done verifiably and globally. Anthropic plans to organise conversations involving policymakers, researchers, civil society and other AI companies to address these questions. The timing of this discussion seems strange, as the AI startup filed its draft papers to go public just last week. It had raised $65 billion in pre-IPO funding, at a valuation of about $965 billion. Why does Anthropic want a slowdown? The Claude maker said it is delegating a lot of its AI development to AI systems themselves, speeding up their work. Extrapolating from this trend, the company claims this could mean an AI capable of fully autonomously designing and developing its own successor, a process it termed "recursive self-improvement". Anthropic cited multiple reasons why this is bad news: The main fear is the loss of human control over these systems. If systems build their own successors, the way they are secured and monitored could go out of human hands. Anthropic claims recursive self-improvement is not inevitable, but it is likely to come sooner than most organisations and institutions are prepared for. The issues present in today's AI models, such as hallucinations, could compound if the models start building themselves, Anthropic warns. Each new generation could inherit and worsen flaws, making it harder for humans to trace their source and fix them, the company said. What's the evidence? This worry isn't without reason. Anthropic noted that as of May 2026, more than 80% of the code merged into its codebase was authored by Claude. On most open-ended coding tasks, Claude's success rate has increased by 50 percentage points in just six months, reaching 76% in May 2026. When asked to pick a better next step in a research session than a human had chosen, the best model (Mythos Preview) did so 64% of the time in April, up from 51% in November 2025. Some gaps remain, though, when Claude exercises judgment in choosing goals, both in engineering and research. What's the possible future? The first scenario, deemed unlikely by Anthropic, is that the trend stalls. The second, most likely in the near term, is that AI labs will continue improving efficiency, with humans still controlling research directions. The third picture shows AI systems achieving full recursive self-improvement and building their successors at a fast pace. In this case, humans would move to oversight and verification.

Anthropic Wants a Global AI Pause If Everyone Else Does | PYMNTS.com

In a report titled "When AI Builds Itself," co-authored by the Anthropic Institute's Lead Marina Favaro and Anthropic Co-Founder Jack Clark, the company said AI may be nearing a point where models can improve themselves with little meaningful human involvement. Anthropic called this "recursive self-improvement" and said the world needs a verifiable international system to slow or temporarily pause frontier AI development before that threshold is reached. The report stops short of a unilateral pledge. Anthropic said it would slow down or pause only if other frontier labs did so under verifiable conditions. The company acknowledged recursive self-improvement hasn't happened yet and isn't inevitable -- but warned it could arrive sooner than governments and institutions are prepared for. Anthropic's post comes less than a week after the AI startup confidentially filed for an IPO and closed a funding round that valued the company near $965 billion. What the Data Shows The numbers Anthropic released are significant on their own terms. Engineers now ship roughly eight times as much code per quarter as they did between 2021 and 2024. Claude's success rate on the most open-ended coding tasks reached 76% in May, up 50 percentage points in six months. On an internal research benchmark, Claude Mythos Preview achieved a 52x code optimization speedup, a task a skilled human researcher would need four to eight hours to reach 4x on. Those figures, though, come with caveats Anthropic acknowledged. Lines of code is an imperfect proxy for productivity. The company noted that code quantity doesn't capture quality, and that Claude-written code was still considered below human par at Anthropic in late 2025, reaching rough parity only recently. Skepticism about those productivity multiples is widespread. Yann LeCun, who left Meta in November to launch AMI Labs on a thesis that large language models are a "dead end" toward general intelligence, has argued that current architectures lack the understanding required for genuine autonomy, according to a 36kr Europe report. Likewise, Fei-Fei Li's World Labs has pursued a parallel path, building systems grounded in spatial and physical reasoning rather than text prediction. Both startups reflect a view that the next capability ceiling won't fall to scale alone. That debate matters for how enterprise leaders read Anthropic's productivity claims. AI is accelerating software workflows. Whether it's on a path to autonomously designing its own successors is a different and much harder question. The Governance Problem Even if the technical trajectory holds, the governance proposal faces steep structural obstacles. Anthropic compared its pause mechanism to Cold War-era nuclear arms control treaties, but acknowledged the analogy breaks down quickly. Training runs are far easier to conceal than missile silos. The incentive to quietly keep building while others stop is enormous. A meaningful pause, by Anthropic's own framing, would require multiple well-resourced labs in multiple countries to stop simultaneously under conditions each can verify the others have met. Rob Enderle of the Enderle Group told SiliconANGLE on Thursday (June 4) that enforcing such a pause would be "practically impossible," given the economic and national security stakes. Tracking decentralized compute across private data centers and open-source communities adds another layer the nuclear analogy doesn't cover, he added. Anthropic said it plans to convene policymakers, researchers, civil society groups and competing AI firms in coming months to work through those questions. The company described the coordination problem as solvable in principle but noted that nuclear verification regimes took decades to build -- time it doesn't expect the AI industry to have. The report's central tension is structural. A company that continues to release models aggressively, Anthropic has shipped updates roughly every two weeks since January and is simultaneously arguing the pace of that release cycle may need to slow. The call for a pause is conditional on competitors agreeing first. Critics have noted that framing places the burden of constraint on the collective rather than on the lab raising the alarm. For all PYMNTS AI coverage, subscribe to the daily AI Newsletter.

Anthropic Warns AI Could Soon Build Better Versions Of Itself

Artificial intelligence (AI) company Anthropic proposed an industry-wide pause in AI development, arguing that increasingly capable systems may soon begin accelerating their own development beyond the pace of human oversight. The Claude maker in a report warned Thursday that AI could eventually become capable of helping build better versions of itself, potentially accelerating progress beyond the pace at which safety measures and governance frameworks can adapt. AI Is Already Helping Build AI Anthropic said employees are increasingly relying on Claude to write code, conduct research and complete technical tasks that would otherwise take significantly longer to finish. "I started leaning hard into Claudifying about a year ago. That's been a crazy adventure and it's now been ~5 months since I last wrote any code myself," an Anthropic employee said. While fully autonomous self-improving AI systems have yet to emerge, the company said advances in coding, research and automation capabilities could significantly accelerate the pace of future model development. "Claude-written code was somewhat worse than human-written code at Anthropic in late 2025, is roughly at parity today, and we expect it to be strictly better within the year," the company added. As evidence of AI's growing contribution to software development, Anthropic said Claude recently shipped more than 800 fixes that cut a category of API errors by a factor of one thousand, work an engineer estimated would have taken a human four years to complete. Calls For A Pause Anthropic warned that recursive self-improvement could dramatically accelerate the pace of AI development, potentially compressing years of progress into much shorter timeframes. The company said such advances could leave governments, regulators and even AI developers struggling to understand and assess increasingly capable systems as the technology evolves. To address that risk, Anthropic argued that frontier AI labs should establish a coordinated pause framework in advance, allowing researchers time to evaluate emerging risks if development begins outpacing safety safeguards. The AI Race Could Make Slowing Down Difficult Anthropic's warning comes as the U.S. and China compete for leadership in artificial intelligence, pouring billions of dollars into chips, data centers and AI models. The company argued that any future slowdown would require broad participation from leading developers, as unilateral action by a single company could prove ineffective. "But if a slowdown simply lets the least cautious actors catch up technologically, it could leave everyone less safe," Anthropic wrote. Anthropic Sees More Than Just Risks Anthropic said recursive self-improvement is not inherently negative and could unlock significant scientific and economic gains, accelerating advances in fields ranging from medicine and energy to software engineering and scientific research. However, it cautioned that realizing those benefits will require effective safeguards, governance frameworks and international cooperation to ensure increasingly capable AI systems remain aligned with human interests. Disclaimer: This content was partially produced with the help of AI tools and was reviewed and published by Benzinga editors. Photo courtesy: Shutterstock Market News and Data brought to you by Benzinga APIs To add Benzinga News as your preferred source on Google, click here.

When AI Builds AI: Anthropic Sets off Another Round of Fear Mongering

This time Anthropic wants regulation among AI companies that can hit pause button on development in case guardrails aren't enough to keep AI from killing humanity Having secured a fundraise that puts it closer to a trillion dollar valuation, Anthropic has now raised concerns over AI autonomously building its successors. The company claims that to guard against such levels of autonomy, frontier AI developers must set up a coordinated and verifiable way to slow down or temporarily pause such development. Of course, one has to take statements from AI startups seeking an IPO with a fistful of salt. In a blog post, Anthropic raises the issue of "recursive self-improvement and its implications" with a pat on its own back. "For most of AI's history, humans drove every step in its development cycle. But at Anthropic, we are delegating a growing share of AI development to AI systems themselves, which is speeding up our work," it says. "Taken far enough, and given enough compute, that trend points to an AI system capable of autonomously designing and developing its own successor." Having raised the flag, Anthropic also immediately brings it to half-mast by noting: "We are not there yet, and recursive self-improvement is not inevitable. But it could come sooner than most institutions are prepared for." Okay, so we get the point that Dario Amodei and his team is smarter than Sam Altman and his team at OpenAI. But, fear-mongering for brownie points? The blog post written by a bunch of Anthropic researchers goes all out to claim the top honours in their AI startup classroom. In fact, it goes on to say that the company's research arm, Anthropic Institute, plans to study and ⁠help build systems that would be necessary to support a slowdown. Immediately thereafter, it also claims that a unilateral pause by a single company would be easier to implement, but would have limited impact. Upon reading the rather lengthy and data-heavy blog post, we were struck by a few callouts on the page. We present them right here in sequence: * "Claude-written code was somewhat worse than human-written code at Anthropic in late 2025, is roughly at parity today, and we expect it to be strictly better within the year." * "The shape of stuff today is roughly 'humans have ideas, and the models are able to implement, test and evaluate them an [order of magnitude] faster than before.'" * "Claude did all of this with pretty minimal help from me over the course of 1-2 days. I think if [a junior colleague] came back to me with results like this in the same span of time, I would be mildly impressed. The future is now." * "The comparative advantage of humans as of right now is still in seeing the bigger picture and thinking beyond the confines of the immediate task." * "Work (and life) ran on a gift economy of small favours between humans. 'Can you help me get this script running?' [...] each one created a little debt, a little mutual awareness. [Claude is] faster, it creates zero debt, but each of these is a lost bid for human collaboration." * "On days where everything works well, I can't help but think nothing I do matters, everything is automated and better and faster than I ever will be. But then there are days where everything breaks and I don't understand why and I realize I have no idea what I've been up to anymore." As a reader if you have excellent skills at pattern recognition, the above lines would definitely shout out their own story. If not, worry not, for both Anthropic and OpenAI have been at it for some time now - each trying to trumpet some dubious achievement while claiming the moral high ground over some obscure stuff. The company argues that full recursive self-improvement increases the risks of humans losing control over AI systems and if they become capable of building their own successors, the ways we secure them, monitor them, and shape their behaviour all grow much more important. Anthropic takes pains to let its audience (or should we say investors?) know that over 80% of their code merged into the codebase was authored by Claude. And then suggests that it may be good for the world to have an option to slow or temporarily pause frontier AI development to enable social structures and alignment research to keep up. But, there's a catch. If these steps are carried out unilaterally or are done without the requisite levels of coordination between other companies, they would backfire. Because, less cautious actors would continue to advance their game while potentially reducing overall safety. Towards this end, Anthropic says it would convene discussions with policymakers, researchers, civil society groups and other AI companies to examine key questions around managing AI-related risks like recursive self-improvement and how to achieve industry-level coordination. Now, that's a moral high ground that's higher than anywhere SpaceX can go! Do you hear us OpenAI? Or are you too busy trying to upstage us? seems to be the question Anthropic is posing. It also wants definitive "agreements between multiple well-resourced labs before hitting the pause button" - Are you listening Microsoft, Google?

Anthropic Calls for Frontier AI Freeze to Prevent Self-Building Tech | PYMNTS.com

The Anthropic Institute will organize conversations with other AI companies as well as policymakers, researchers and civil society to address concerns about the technology and create methods of coordination, Anthropic Institute Lead Marina Favaro and Anthropic Co-Founder Jack Clark wrote in a Thursday blog post. "A meaningful slowdown or pause would require multiple well-resourced labs at or near the frontier, in multiple countries, agreeing to stop under the same conditions," they wrote. "It would also require that each can verify that the others have actually stopped." The proposal for a pause was sparked by the growing share of AI development that is being delegated to AI systems, and the potential that an AI system could fully autonomously design and develop another AI system, according to the post. The authors wrote that this development is not inevitable, but it could come soon than expected. "AI that can build itself would be a major development in the history of technology -- one that could bring enormous good for the world in science, healthcare and beyond," Favaro and Clark wrote. "But full recursive self-improvement also might increase the risks of humans losing control over AI systems." This post came two days after President Donald Trump signed an executive order that seeks access to new AI models for up to 30 days before their release so that the government can choose "trusted partners that will have early access to covered frontier models to promote secure innovation and strengthen the cybersecurity of critical infrastructure." It also came about a week after Illinois Gov. JB Pritzker said he will sign a bill that requires large frontier developers to create a frontier AI framework that addresses risks and governance; requires transparency reports before frontier models are deployed; mandates annual third-party audits; requires frontier developers to report critical safety incidents; and provides whistleblower protections and internal reporting processes for covered employees.

Anthropic Calls for Option to Pause Frontier AI Development

You can access the blog post from here. "it would be good for the world to have the option to slow or temporarily pause frontier AI development." That is the central warning in a new paper from Anthropic, which argues that AI is increasingly helping to build newer, more capable systems, raising the possibility of "recursive self-improvement," a future in which AI systems design and develop their own successors with minimal human involvement. The company says AI is already accelerating AI development inside Anthropic. It claims that more than 80% of the code merged into its production codebase is now written by Claude, up from low single digits before the launch of Claude Code in 2025. Anthropic also says the typical engineer merged eight times as much code per day in the second quarter of 2026 as in 2024. AI's growing role in AI development: Anthropic said the role of human engineers and researchers is shifting from doing technical work to reviewing outputs and deciding which problems are worth solving. "The doing (i.e., writing the code, running the experiment, producing the result) now costs almost nothing in human time," it wrote. The company cited internal and public benchmarks to argue that AI systems are becoming capable of handling increasingly complex and longer-duration tasks. It said Claude models progressed from completing software tasks that took humans minutes in 2024 to tasks lasting several hours and, more recently, up to 12 hours. From coding to research: According to Anthropic, AI systems have also improved rapidly on software engineering and research benchmarks. The company said its latest systems can independently run experiments, propose hypotheses and conduct parts of open-ended research projects, though humans still set goals and evaluate results. Anthropic acknowledged that a key limitation remains what it called "research taste," the ability to decide which ideas matter and which directions are worth pursuing. "The comparative advantage of humans as of right now is still in seeing the bigger picture and thinking beyond the confines of the immediate task," the company said. The company outlines three possible futures: In the first, progress slows due to technical or infrastructure constraints. In the second, AI increasingly automates development work while humans continue to direct research. In the third, AI systems become capable of full recursive self-improvement and begin building their own successors. Anthropic says it is most concerned about the second and third scenarios because they could leave governments and societies with little time to adapt. The paper warns that recursive self-improvement could increase the risk of humans losing control over AI systems if alignment and oversight do not keep pace. Call for coordination and safeguards: The company also raises concerns about governance. It argues that a meaningful pause in advanced AI development would require coordination among multiple leading AI developers and mechanisms to verify compliance. "If such systems existed, we expect that we would slow down or temporarily pause if other developers at or near the frontier also did so in a verifiable manner," it says. While acknowledging uncertainty about whether AI will achieve the judgment required to drive research independently, Anthropic argues that much of AI progress today comes from incremental experimentation, a process that AI systems are increasingly capable of performing. The paper concludes that the window for broader public debate on these questions is limited and that policymakers, researchers, civil society groups and AI companies should be involved in discussions about how to manage the technology's future trajectory.

Anthropic Says AI May Soon Upgrade Itself Without Human Help

Anthropic is delegating a growing share of its AI development to the AI systems, and it may develop its own successor, calling the idea "recursive self-improvement." "We are not there yet, and recursive self-improvement is not inevitable. But it could come sooner than most institutions are prepared for," the company wrote in a recent blog post. The report describes a shift inside Anthropic from early workflows dominated by human-written code toward software agents that can execute tasks, run code, and hand off work to other agents. It frames that progression as a pathway that could, over time, close the loop from assisting engineers to training and improving models with minimal supervision. Using internal metrics, the Anthropic Institute said Claude was responsible for more than 80% of code merged into Anthropic's codebase by May 2026, up from low single-digit levels in February 2025. The company also reported a sharp rise in engineer productivity after AI agents evolved from suggesting code snippets to independently executing and iterating on longer tasks, with engineers merging roughly eight times more code per day in Q2 2026 than in 2024. Anthropic cautioned that raw code volume can overstate productivity gains, calling lines of code an imperfect measure, though internal feedback suggests AI is enabling employees to complete core tasks significantly faster. In a March 2026 internal survey of 130 research staff, respondents estimated they produced about 4 times as much output using Mythos Preview as when working without AI tools. "We expect that the true degree of uplift in March was somewhat lower. Nevertheless, we find the overall claim plausible, and in line with our other observations: a significant fraction of Anthropic technical staff is accomplishing their core work multiple times faster than they could without AI assistance," the company wrote. The institute also pointed to work it said would likely not have been prioritized without agents, including exploratory tooling and long-deferred cleanup. In one example, it said Claude delivered more than 800 fixes that cut a category of API errors by a factor of 1,000. The supervising engineer estimated the same effort would have taken a human about four years to complete. AI Models Improve At A Faster Rate Anthropic also noted that the length of tasks AI can reliably complete on their own has been doubling roughly every four months. That's up from an earlier trend of doubling every seven months. "In March 2024, Claude Opus 3 could complete software tasks that take humans about four minutes to complete. A year later, Claude Sonnet 3.7 managed tasks that took about an hour and a half. A year after that, Claude Opus 4.6 managed 12-hour tasks.1 If this trend holds, tasks that take a skilled person days could come into range this year. In 2027, AI systems could be capable of tasks that take a person weeks," the company stated. The report also highlighted benchmark progress, including SWE-bench and CORE-Bench, where it said scores moved from low levels to near-ceiling performance over relatively short timeframes. It added that METR found Claude Mythos Preview could work for "at least" 16 hours and was "at the upper end of what [METR] can measure without new tasks." The Biggest Remaining Gap? Judgment Models may execute well-defined tasks, but they still lag behind humans in deciding which goals to pursue in engineering and research. The institute argued that closing that gap would be central to any future system that could design its own successor, and said that possibility would raise the stakes for security, monitoring, and behavior-shaping controls. Earlier this month, Anthropic confidentially filed a draft Form S-1 with the U.S. Securities and Exchange Commission, giving the artificial intelligence company the option to go public after the SEC completes its review. Last month, Anthropic overtook OpenAI as the world's most valuable startup after raising $65 billion in Series H, valuing the company at $965 billion. Altimeter Capital, Dragoneer, Greenoaks and Sequoia Capital led the funding round. If Anthropic goes public at a $1 trillion valuation, it would instantly rank among the most valuable companies globally and could become the second- or third-largest IPO in history, trailing SpaceX and Saudi Aramco. This content was partially produced with the help of AI tools and was reviewed and published by Benzinga editors. Market News and Data brought to you by Benzinga APIs To add Benzinga News as your preferred source on Google, click here.

Anthropic Flags Broader AI Threats Beyond Employment Concerns

The authors of the Anthropic post, company co-founder Jack Clark and Marina Favaro, head of its research institute, said the pause would be used to enable 'societal structures and alignment research' to keep up with . Alignment is industry shorthand for making sure the technology matches human values and intentions. The proposed coordination would let advanced AI labs verify that global rivals have actually stopped or slowed their work, 'and that a bad actor could not use the auspices of a coordinated slowdown to jump ahead in secret.' One of the most notable points in the paper is the idea that AI development may need stronger safety checks as technology advances. Anthropic said companies should think seriously about ways to monitor powerful AI systems and support humans in remaining in control. The company also suggested that major AI developers may eventually need to work together on common safety standards. As AI becomes more capable, the challenge will be making sure people remain in charge of the technology they create, according to Anthropic.

Anthropic calls for global AI slowdown after $965B valuation. Critics claim it's just to hobble competition.

Anthropic said artificial intelligence is advancing so rapidly that an industry-wide pause is needed to slow the pace of development while companies get a handle on potential societal risks. The AI juggernaut published a blog post Thursday warning that AI systems could soon be able to improve themselves without human intervention, resulting in chaos. Critics and rivals have cast similar statements in the past from Anthropic -- which dethroned OpenAI as the most valuable AI company last month - as part of a bid to hobble the competition in the red-hot race to develop advanced artificial intelligence. Anthropic has long warned that the power of its models could pose risks to humanity. Anthropic's Chief Executive Dario Amodei feuded for months with the White House over AI safety related to the use of its systems in military applications. Anthropic called on top AI labs to weigh slowing the pace of development and offered internal data that showed the speedy improvement of its own models. "We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology," the post stated. It was written by Anthropic's head of internal research Marina Favaro and head of policy Jack Clark. The ability to slow global AI development would "likely be a good thing," they added. OpenAI CEO Sam Altman previously accused Anthropic of doing "fear-based marketing." "It is clearly incredible marketing to say, 'We have built a bomb, we are about to drop it on your head. We will sell you a bomb shelter for $100 million,'" he said last month. Anthropic's post outlined how model advances appear to be moving toward "recursive self-improvement," or AI systems that can improve on their own without human intervention. AI experts have warned that the phenomenon could wreak havoc on society. "This would revolutionize knowledge work and government services, but could also be turned to harmful ends, from authoritarian surveillance of whole populations to influence operations that tailor manipulation to each individual and run at a scale no human team could match," the post stated. It acknowledged that self-improving AI hasn't happened yet, and isn't inevitable. Anthropic's valuation reached a whopping $965 billion in its $65 billion Series H funding round last month. The firm had long been seen as an upstart seeking to catch up to OpenAI's early dominance in the field. Anthropic's Thursday post went as far as to propose a global agreement for the slowdown and how to verify adherence to it. Some have criticized the lab's many warnings about the danger its own tools could pose as a marketing tactic. University of Pennsylvania's Wharton School professor Ethan Mollick posted on X: "There is a bit of navel-gazing, some marketing, and a lot of very sincere beliefs about what Anthropic thinks is likely in the near future of AI." Anthropic's leaders have insisted they're putting AI safety ahead of growth and want to facilitate more discussion of risks. One online critic took a political angle on the lab's posting, writing on X: "They're trying to pause until a Democrat gets back in the White House."

Anthropic warns AI systems accelerate own development amid recursive improvement concerns By Investing.com

Investing.com - Anthropic said Thursday that its internal data shows AI systems are accelerating the development of more advanced AI, a trend that could lead to recursive self-improvement where AI autonomously builds its own successors. The company said engineers at Anthropic now ship eight times as much code per quarter as they did from 2021 to 2025, with more than 80% of code merged into Anthropic's codebase as of May 2026 authored by Claude. Before Claude Code launched in research preview in February 2025, this number was in the low single digits. In a March 2026 poll of 130 employees from across Anthropic research teams, the median respondent estimated they produced around four times as much output with Mythos Preview as they would have without access to any AI models. Public benchmarks show the rate at which AI models improve is accelerating. The length of tasks that AI systems can reliably complete on their own has been doubling roughly every four months, up from an earlier trend of doubling every seven months. In March 2024, Claude Opus 3 could complete software tasks that take humans about four minutes to complete, while a year later Claude Sonnet 3.7 managed tasks that took about an hour and a half. By March 2026, Claude Opus 4.6 managed 12-hour tasks. Anthropic said Claude's success rate on open-ended tasks reached 76% in May 2026, up 50 percentage points in six months. In April 2026, Anthropic published a demonstration of Claude running an open-ended research project end to end, where Claude-powered agents were given an open problem in AI safety and left to solve it. Two human researchers recovered roughly 23% of the performance gap over about a week, while the agents recovered 97% over 800 cumulative hours using roughly $18,000 in compute. Anthropic said it believes a slowdown in frontier AI development would likely be a good thing to give societal structures and alignment research time to keep up with the technology's advance. The company said it would slow down or temporarily pause if other developers at or near the frontier also did so in a verifiable manner. The Anthropic Institute will conduct research to help build systems that would enable a credible slowdown or pause by allowing frontier AI developers to verify that others globally have actually stopped or slowed. This article was generated with the support of AI and reviewed by an editor. For more information see our T&C.

Anthropic urges AI labs to pause development, warns humans risk losing control

June 5 (Reuters) - Anthropic is calling on major artificial intelligence labs to consider a coordinated and verifiable pause in development, warning that rapid advances in the technology could soon allow AI systems to improve themselves faster than society can manage the risks. The Claude creator said AI's ability to complete tasks on its own has been doubling roughly every four months and it was headed for "recursive self-improvement", the point at which the technology can improve without human intervention. "If systems are capable of fully building their own successors, the ways we secure them, monitor them, and shape their behavior all grow much more important," the startup said in a lengthy blog post on Thursday, adding that a pause would allow society to "deal with its immense implications." "We are not there yet, and recursive self-improvement is not inevitable. But it could come sooner than most institutions are prepared for," Anthropic co-founder Jack Clark and Anthropic Institute lead Marina Favaro wrote in the post. Fears that advanced AI systems may get out of human control and cause societal harm have risen as the technology becomes increasingly capable. Anthropic's own Mythos model sent shockwaves through industries including banking and software earlier this year with its ability to find vulnerabilities in existing code. But regulation has been slow, especially in the U.S. where most leading AI labs are based. A Trump administration executive order earlier this week put the onus on the labs themselves, asking them to voluntarily submit their most capable models for government cybersecurity testing before public release. AI researchers have also urged a pause before but had little success. Elon Musk, who owns AI lab xAI, was among backers of a 2023 push by the non-profit Future of Life Institute to halt AI development for six months to allow time for safety guardrails. Anthropic has long positioned itself as a safety-focused AI lab. Earlier this year, it refused to let the U.S. military use its models for domestic surveillance and fully autonomous weapons, prompting backlash from the government which put it on a national security blacklist, set to take effect later in 2026. Reuters reported on Friday the dispute was showing signs of easing across parts of the U.S. government. Still, Anthropic has continued to release increasingly powerful models and in February walked back a key safety pledge, saying that it would no longer hold back potentially dangerous AI if rivals were close to matching its capabilities. It was recently valued at $965 billion in a massive funding round and confidentially filed for a U.S. initial public offering on Monday, putting it ahead of rival OpenAI in both valuation and the race to secure crucial funding. COORDINATED ACTION Anthropic's Thursday post cautioned that unilateral or poorly coordinated slowdowns could backfire if less cautious actors continue advancing, potentially reducing overall safety. It said that a meaningful pause would require agreement among "multiple well-resourced labs" operating at the technological frontier, as well as rules on what conditions would trigger or lift such a pause and who would oversee it. "A unilateral pause by one lab, by contrast, is achievable immediately, but accomplishes much less: it would change who the front-runner is, but it would not create the wider deliberative process that is currently missing," the startup said. Its research arm, Anthropic Institute, plans to study systems needed to support a slowdown and in the coming months will convene policymakers, researchers, civil society groups and rival AI firms to discuss managing risks such as recursive self-improvement. OpenAI, xAI, Alphabet, Meta Platforms and France's Mistral did not immediately respond to requests for comment on whether they would join the call. (Reporting by Aditya Soni in Bengaluru and Juby Babu in Mexico City; Editing by Shreya Biswas and Saumyadeb Chakrabarty)

AI vs Humans? Anthropic's engineers don't know what their job is anymore

Since reading it, I've been thinking about the employee quotes from Anthropic's new study about AI self improvement. Employees claim to have not written any code for five months; another acknowledges, on days when the entire process proceeds smoothly, they start thinking if what they are doing even really matters. Also read: Computex 2026: 5 major announcements you should not miss These developers are not new to the industry - they are some of the most highly regarded engineers employed by one of the premier artificial intelligence laboratories in the universe; therefore, their reported lack of clarity regarding their respective roles leads me to believe they experience disorientation. Most of the press will report on the scale of Anthropic's success as indicated by Claude creating over 80 percent of the merged code into Anthropic's production repository, and engineers producing 8 times more code per quarter than in 2024. While these facts are noteworthy on their own, what is more fascinating is what occurs to the human beings driving the car when it drives itself. Also read: Encoder-Free AI explained: The architecture behind Google's Gemma 4 12B No one knows for sure what the relationship will be like between AI and humans long term. The company's view is that humans will focus on higher-level tasks (like setting research direction, determining the value of experiments, and deciding what to build next), while the doing has been automated through Claude. While this may be a good portrayal for senior leaders in the room, it may be a less clear picture for everyone else. When Claude has the ability to answer open-ended questions, troubleshoot live production issues, and manage end-to-end cycles of applied research with minimal human intervention, it can be difficult to tell the difference between directing AI versus watching the AI perform these tasks. Another area that Anthropic does mention but doesn't expand on enough is the impact of technology on how people work together. One employee talked about how before there were many small favours exchanged for a variety of reasons, like asking someone for a favour quickly and having shared context, which created mutual awareness between two people. With the assistance of Claude, this process has been accelerated and all social debt created through these small exchanges has been completely eliminated. While productivity has increased as a result of Claude, many moments of lost human collaboration have also disappeared. None of this is an argument against the technology. The gains are real and the applications go well beyond shipping code faster. But we are in a strange moment where the people closest to this transition are themselves struggling to articulate what their role now is. That is not a crisis yet. But it is a question that deserves a more honest answer than "humans will focus on the important stuff." Because right now, even at Anthropic, nobody seems entirely sure what the important stuff is anymore.