Sam Altman Proposes U.S.-China AI Safety Pact as Industry Leaders Call to Slow Development

OpenAI CEO Calls for Historic U.S.-China AI Safety Agreement

Sam Altman, OpenAI CEO, proposed that Presidents Donald Trump and Xi Jinping could jointly win the Nobel Peace Prize by establishing shared AI safety standards between the United States and China. In an exclusive interview with Fortune Editor-in-Chief Alyson Shontell, Altman emphasized that U.S.-China cooperation on AI development guardrails represents "something that should be easy to agree to."1

1

The proposal comes as existential risks from artificial intelligence have reached levels comparable to nuclear threats during the Cold War.

Growing Industry Consensus to Slow Down AI Development

Altman's remarks follow increasing calls from AI industry leaders to decelerate development. Anthropic CEO Dario Amodei published a 3,800-word post announcing his company's commitment to hiring embedded third-party evaluators with full company access to verify safety practices.2

2

Amodei warned that AI has advanced "drastically faster" in recent months and could become capable of taking over the entire internet within 6-12 months, potentially causing hundreds of billions of dollars in damages.2

2

Both OpenAI and Anthropic are now exploring collective pacts to address rapidly increasing national security risks.

Recursive Self-Improvement Emerges as Critical Threshold

The debate centers on recursive self-improvement, where AI systems continuously upgrade themselves and add capabilities. When asked if banning this capability would suffice, Altman acknowledged it "probably would not be sufficient" but emphasized that even basic shared standards between the U.S. and China would represent progress.1

1

Paul Tudor Jones, founder of Tudor Investment Corp., echoed these concerns in a Wall Street Journal op-ed, warning that "a new superpower is about to enter the fray, one that is unknowable and unending and potentially dwarfs the U.S. and China individually or collectively."1

1

Researcher Resignation Highlights Internal Industry Tensions

Jacob Coxon, a researcher at Anthropic, publicly resigned and accused both Anthropic and OpenAI of "racing straight to self-improving superintelligence and gambling with our lives."1

1

Evan Hubinger, Anthropic's alignment science lead, responded by confirming that researchers "really do earnestly believe AI could kill all humans," estimating the probability at more than 10% within the next decade.1

1

These revelations underscore the gravity of existential risks facing the industry.

Framework for International AI Oversight Takes Shape

Altman outlined a framework for U.S.-China cooperation that could be documented in a single page. The agreement would establish rules for development, testing, and monitoring alignment standards, along with oversight mechanisms to prevent either country from gaining disproportionate power or bypassing safety protocols.1

1

Amodei similarly advocated for governments to coordinate on establishing common safety standards, with third-party evaluators verifying each company's practices and reporting incidents.2

2

Balancing Speed Against Catastrophic Risk

The proposals reflect a delicate balance between maintaining competitive advantage and preventing loss-of-control scenarios. Altman explained that "neither of us should be taking loss-of-control risk in the race to beat the other country," while acknowledging concerns that whichever nation achieves superintelligence first gains significant power.1

1

Amodei warned in his CNN interview that AI misuse could enable cyberattacks, bioterrorism, and serious economic disruption, yet cautioned that moving too slowly risks leaving the technology "in charge of the wrong people."2

2

Industry observers now watch whether these proposals translate into concrete action or remain aspirational amid intensifying global AI competition.