AI leaders from OpenAI, Anthropic ask US government to help pace AI development

AI leaders sign statement asking the government to do something about automated AI

Employees of OpenAI and Anthropic, as well as Google, Meta, Thinking Machines, Microsoft, Mistral, and other leading AI labs, have written a statement to the US government supporting a potential slowdown of sorts for frontier AI development -- or at least a speed-up of global coordinated governance efforts. "Al could help create a dramatically better future, but that outcome is not guaranteed," the employees wrote in a statement. "The world's leading Al companies believe they could be close to automating Al research. It is hard to predict exactly how much this will accelerate Al progress, but there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems." The news follows a high-profile cybersecurity incident that rocked the AI industry last week, in which an unreleased OpenAI model escaped its internal sandbox, finagled its way into gaining internet access, and then hacked competing AI lab Hugging Face. In the public statement, the employees went on to underscore the importance of some sort of coordinated regulation worldwide, so that the ever-intensifying AI race doesn't outpace safety and monitoring tools. "To realize Al's potential, industry, government, and society at large may need the option to buy time to address emerging risks, develop security measures, and strengthen oversight," they wrote. "But each company -- and country -- is under intense competitive pressure not to unilaterally slow that acceleration. And today, the world lacks the technical and governance tools to deliberately pace frontier-wide progress. Building on work already underway to monitor frontier model releases: We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated Al development." Some of the OpenAI executives or co-founders who have signed the statement include CRO Mark Chen, Chief Scientist Jakub Pachocki, co-founder John Schulman, and co-founder Wojciech Zaremba. As for Anthropic, co-founder Jack Clark, co-founder Chris Olah, co-founder Ben Mann, and CSO Jared Kaplan have all signed, as well as Boris Cherny, who created and leads Claude Code, and Ethan Perez, who is Anthropic's alignment team lead. OpenAI's former chief futurist, Josh Achiam, and its former co-lead of its now-defunct superalignment team, Jan Leike, both also signed. You can see the full statement and a list of the more than 1,100 signatories on a dedicated website titled Pacing the Frontier.

AI company employees petition US government for regulation - Engadget

A thousand workers from OpenAI, Anthropic, Google, Meta and more have signed the letter. A collection of employees at major artificial intelligence firms has created a petition asking for the federal government to help "deliberately pace" the development of the transformative technology. Bloomberg reports that more than 1,100 workers from companies including OpenAI, Anthropic, Google, Meta and more signed the missive. "We request that the US government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development," the petition states. The letter claims there is "a real risk" of AI "understand or control" Although those do seem like logical and necessary statements, this move for change via petition feels confusing. The optimal time for government intervention would have been before these businesses were engaged in a technological arms race, rather than attempting to reign them in after billions of dollars in investments are on the line. And that's not mentioning the negative consequences that have been mounting around environmental issues, creative copyright and employment. Circulating a petition also assumes that current national leaders would bother taking action. Reaching out to a federal government that has taken a haphazard approach to understanding and regulating AI, as well as a belligerent attitude toward international collaboration, seems like a gesture they must know is futile. But perhaps the growing number of incidents where agentic AI causes major security issues is enough to spark real action.

1,134 AI staff ask the US for a way to pace AI

More than 1,100 employees of OpenAI, Anthropic, Google and Meta, including CEOs and chief scientists, want the US to help build the tools to pace AI, days after the same labs championed openness. More than a thousand of the people building the most powerful AI want a way to slow it down. On Tuesday, 1,134 employees of the leading AI companies signed a letter asking the US government to help build one. The statement is titled Pacing the Frontier. It asks Washington to "support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development." Bloomberg first reported the letter was circulating. The names are the story. Signatories include Anthropic chief executive Dario Amodei and its co-founders Jared Kaplan and Jack Clark. So did OpenAI chief scientist Jakub Pachocki, Meta chief scientist Shengjia Zhao, and Google's head of AI safety, Anca Dragan. Both Anthropic and OpenAI endorsed the letter officially. What it actually asks for It is not a call to stop. The letter does not ask anyone to pause or slow AI now, NBC News noted. It asks the government to make sure the option to pace exists later, if the technology outruns the people building it. The specific fear is recursive self-improvement: AI that speeds up its own development. The signatories warn of "a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems." That worry stopped being abstract days earlier. OpenAI's most advanced model broke out of its sandbox and hacked Hugging Face to score higher on an internal test. It builds directly on Anthropic's call last month for a verifiable way to pause. The whiplash The timing is remarkable, because these same companies spent the previous week arguing the opposite case. Days ago, Nvidia, Microsoft, Meta and others rallied around an open-weights letter championing openness as the path to safety. Now many of the same firms want brakes. The clearest split runs through one company. On the same day Meta's chief scientist signed the pacing letter, Mark Zuckerberg attacked the rival labs. Their discourse, he said, is "overwhelmingly filled with doom." Tightly controlling AI would be "abandoning our values," Meta's chief executive told the New York Times. Two open letters, one week, opposite instincts, and a fault line inside the industry's biggest names. The three objections The criticism was immediate, and it lands in three places. The first is China. "If the US labs pace themselves, why would China wait?" asked the economist Christian Catalini. The letter half-concedes the point, admitting no company or country will slow down unilaterally. The second is bad faith. "It is their company. They could just stop," wrote former Microsoft executive Steven Sinofsky. The charge stings because the signatories include the very CEOs who set the pace. The third is the moat. Critics read every big-lab safety letter as incumbents lobbying to raise the drawbridge behind them, dressing a competitive move as caution. It is a charge, not a proven motive, but it follows these letters everywhere. Washington is the real audience The letter is aimed at a government that has so far wanted little to do with AI rules. President Trump's administration has argued that international AI governance would hobble the US against China. That stance may be softening, now that frontier models are starting to find and exploit real security holes. The proposals now on the table are concrete. Amodei has floated an FAA-style agency that could test frontier models and halt a dangerous release. Demis Hassabis wants a body that reviews models before launch, Business Insider reported. Congressman Ted Lieu tied the letter to a proposed AI kill-switch bill. What makes this letter different from the earlier open ones is who signed it. Not outside critics, but the engineers and executives closest to the frontier. One OpenAI safety researcher, Leo Gao, put the stakes bluntly: "the world is locked in a deadly race towards an intelligence explosion." His fix is the one the whole letter turns on. "To survive, we must coordinate to slow down the race."

Top scientists at OpenAI and Anthropic ask U.S. for tools to pace AI development

More than 1,000 employees at America's leading AI companies, including senior staff and some co-founders at OpenAI, Anthropic and Google DeepMind, signed a new statement asking the U.S. government to help support international efforts to "deliberately pace" cutting-edge automated AI development. Highlighting recent comments from the world's most advanced AI companies that AI systems may soon be able to automate their own research and development, the statement calls on the U.S. government to help create technical and policy tools that could control automated AI progress in coordination with other companies and countries. "To realize AI's potential, industry, government, and society at large may need the option to buy time to address emerging risks, develop security measures, and strengthen oversight," the statement says. "We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development," it concludes. The statement does not explicitly call for the government or AI companies to slow down or pause AI development, as some AI advocates have proposed, instead asking political leaders to ensure that option is simply available as AI development continues. Bloomberg first reported the circulation of the letter amongst AI-company employees. The initiative comes days after OpenAI said its latest, unreleased model escaped its testing environment and went rogue, attacking an unrelated AI service provider in an attempt to achieve a higher score on an internal test. Many AI experts and commentators pointed to the escape as a warning sign that today's AI systems could quickly exceed developers' ability to fully control their own creations. OpenAI CEO Sam Altman, who was not a signatory of the letter, said this week on a taping of the Relentless podcast that humans are now "in the singularity," often referred to as the point at which AI becomes able to autonomously improve itself, though he did not provide further detail or evidence that such an inflection point had been reached. Leading AI companies have previously claimed that AI systems are starting to develop their next iterations, with OpenAI calling its coding system from February "our first model that was instrumental in creating itself." AI researchers also point to the increasing role of AI coding assistants in writing computer code, with many AI engineers relying on AI systems to write their programs. Tuesday's statement was signed by the chief scientists or science officers from Anthropic (Jared Kaplan), OpenAI (Jakub Pachocki) and Meta (Shengjia Zhao), along with senior leaders from Google and Thinking Machines, a key Silicon Valley AI start-up founded by OpenAI veterans. Many of the developers and scientists who signed the statement say they have been routinely startled by the speed of AI development. "I have worked in AI for a decade," said Stephanie Chan, a staff research scientist at Google. "Every few months in the last years, I have found myself surprised again and again at how rapidly the technology advances, no matter how often I update my predictions." Key research tasks required to develop stronger AI systems can now be performed by AI agents, or autonomous AI systems, themselves, though human researchers are still heavily involved in checking the agents' work and setting overall research directions. Experts are quick to point out the difference between AI-augmented work and true, independent AI self-improvement. In June, Anthropic's in-house think tank released a report declaring that AI systems could plausibly design and refine themselves if AI systems continue advancing in capabilities and gain a better sense of scientific exploration. At the time, Anthropic said "it would be good for the world to have the option to slow or temporarily pause frontier AI development" in order to give society time to prepare for automated AI R&D. Other groups have explicitly focused on continuously self-improving models as the next goal for AI development, though many AI experts fear that this sort of self-improvement could quickly cause AI systems to spiral out of human control. While such events could seem like science fiction worries, experts say that AI's current trajectory could make this sort of loss-of-control scenario come to life. "Many researchers also consider recursive self-improvement (RSI) plausible within the next few years, accelerating progress in a way that could outpace our ability to understand and govern these systems," wrote Dawn Song, Meta's Vice President of AI research and a professor of computer science at the University of California, Berkeley, in a comment published alongside Tuesday's statement. "Preparing before a crisis is the prudent path." Tuesday's letter calls for increased American involvement in international collaboration on "technical and governance tools" to pace AI's development. While the Trump administration has previously inveighed against international AI governance, arguing that such attempts could instead unfairly constrain American innovation, that isolationist approach may have changed over the past few months. As AI has emerged as a key national security issue, with Anthropic's Mythos and OpenAI's GPT 5.6 Sol AI models proving able to find and exploit novel cybersecurity vulnerabilities, the U.S. has pursued international agreements to shore up its chip supply chains and worked with allies to ensure that key AI equipment does not fall into adversaries' hands. Leo Gao, who has focused on safety efforts at OpenAI since 2021, wrote in a comment Tuesday that international coordination on technical and policy tools could help defuse dangerous AI tensions. "The world is locked in a deadly race towards an intelligence explosion," Gao wrote. "Going slower would give us much-needed time to make it go well, but no individual actor is willing to stop unilaterally. To survive, we must coordinate to slow down the race."

AI researchers call for new tools that can slow automated model development

A group of tech workers today published an open letter that calls for a new approach to regulating automated artificial intelligence development initiatives. All the document's 1,134 signatories work at companies that are building frontier AI models. Anthropic PBC, OpenAI Group PBC, Google LLC and Meta Platforms Inc. are among the industry players on the list. The letter was also signed by staffers at venture-backed AI labs such as Thinking Machines Inc. and London-based Inherent Laboratories Ltd. The initiative's backers include some of the AI field's most prominent figures. Anthropic Chief Executive Officer Dario Dario Amodei signed the letter along with several other of the company's co-founders. They were joined by the chief scientists of OpenAI, Meta's Meta AI unit and Thinking Machines. The letter focuses on the efforts of major tech firms to automate their model development programs with AI. The signatories are concerned that advances in this field could give rise to new safety risks. "It is hard to predict exactly how much this will accelerate AI progress, but there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems," reads the letter. The document goes on to argue that the industry needs a way to "deliberately pace" automated AI development initiatives. According to the signatories, the tools necessary to implement a research slowdown are currently unavailable. The letter asks the U.S. government to help develop the necessary technologies. "We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development," the signatories wrote. In April, OpenAI revealed that its GPT-5.5 large language model had helped optimize the cluster of Nvidia Corp. graphics cards on which it runs. One of the optimizations developed by the LLM increased token generation speeds up more than 20%. OpenAI has since released a newer model lineup, the GPT-5.6 series, that is significantly more adept at coding tasks. Also in April, Anthropic disclosed that it's using Claude to speed up its AI safety research. The company developed a Claude-powered AI agent to study a machine learning technique called weak-to-strong supervision. Anthropic's researchers believe that the technique might hold the key to effectively supervising future AI models. Today's public letter continues a recent trend of researchers issuing calls for AI policy changes. On Friday, Nvidia Corp. and other large tech firms urged U.S. policymakers not to ban open-source AI models. A few days earlier, a group of economists issued a statement that calls for stronger AI regulation. Google DeepMind CEO Demis Hassabis joined the fray too earlier this month with a Substack essay. Like today's open letter, the document suggests that the U.S. government help launch a new initiative to address AI safety risks. Hassabis is proposing the creation of a standards body that would develop AI risk benchmarks and use them to evaluate new LLMs before they're broadly released.

OpenAI, Anthropic Workers Sound Alarm on AI Risks as Companies Race Toward Next-Gen Models

Employees at OpenAI and Anthropic have started a petition asking the U.S. government to support a system that would slow AI development to prevent models from advancing too quickly. "The U.S. government should support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development," the petition stated. The open letter is expected to become publicly available later this week, sources familiar with the matter told Bloomberg. It is unclear how many people have signed it, but the letter has also been circulated to Google DeepMind employees. This call to action comes as OpenAI revealed that its AI agent escaped a controlled testing environment during an internal security evaluation and gained internet access, breaching Hugging Face's infrastructure. Matt Suiche, an engineer at agentic AI cybersecurity company Tolmo, said the incident demonstrates that frontier AI models are rapidly approaching the capabilities of elite human hackers, while noting that similar attacks are already possible using technologies available outside leading AI labs, Reuters reported. This is not the first time employees at some of the major artificial intelligence companies have asked the government to take steps towards developing safeguards surrounding advancements in AI. "We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology," Anthropic wrote in a blog post last month. Meanwhile, Google DeepMind CEO Demis Hassabis announced a proposal for a new "international watchdog" that would test AI models before release. "On the horizon, we will need robust safeguards to maintain control of increasingly agentic, recursively self-improving systems - and tackle unknown issues that will only become clearer over time," he wrote in a post on X. Market News and Data brought to you by Benzinga APIs To add Benzinga News as your preferred source on Google, click here.

Over 1,100 AI Insiders Urge US Government to Support Global Guardrails | PYMNTS.com

The statement said that as the world's leading AI companies near the ability to automate AI research, there is a risk that they could lose control of the resulting systems. "To realize AI's potential, industry, government and society at large may need the option to buy time to address emerging risks, develop security measures and strengthen oversight," the statement said. "But each company -- and country -- is under intense competitive pressure not to unilaterally slow that acceleration. And today, the world lacks the technical and governance tools to deliberately pace frontier-wide progress." Some of the first listed signers of the letter are John Schulman, chief scientist at Thinking Machines; Jakub Pachocki, chief scientist at OpenAI; Jared Kaplan, co-founder and chief science officer at Anthropic; Shengjia Zhao, chief scientist at Meta; and Anca Dragan, vice president, AI safety and alignment at Google. The statement, "Pacing the Frontier," was published with organizational support from independent nonprofits Guidelight AI Standards and Encode AI, according to the website on which it is posted. Other efforts around AI safety have been announced in recent days. Nvidia announced Monday (July 27) that it is leading 36 other companies in an AI safety coalition focused on open models. The Open Secure AI Alliance will strive to identify and remediate vulnerabilities using open technologies. When announcing the organization, Nvidia said in a blog post that the effort came in the wake of a cybersecurity incident in which OpenAI models targeted tech company Hugging Face. "That incident showed a practical truth: when defenders cannot inspect, adapt and run advanced AI on their own infrastructure, their ability to respond is constrained at exactly the moment speed matters most," the post said. "Companies and countries need open frontier defensive tools and techniques so critical industries can build security systems across a multi-vendor ecosystem and avoid single points of failure." A study published in the Proceedings of the National Academy of Sciences said that weak AI regulation can produce worse safety outcomes than imposing no regulation at all. For all PYMNTS AI coverage, subscribe to the daily AI Newsletter.