AI leaders call for kill switch as race to superintelligence accelerates beyond control

AI Safety Concerns Unite Industry Leaders

Frontrunners in artificial intelligence development are sounding unprecedented alarms about the speed of AI advancement. Anthropic co-founder Jack Clark and colleagues warn that AI systems may soon gain the ability to enhance themselves with minimal human oversight, potentially spiraling out of control 1

1

. The company argues humanity needs to establish "the option to slow or temporarily pause frontier AI development" — essentially an AI kill switch.

This isn't an isolated concern. OpenAI has acknowledged the potential for catastrophe and expressed interest in global capacity for "coordinated action, including slowing frontier development when needed" 1

1

. Google DeepMind's Nobel-laureate CEO Demis Hassabis shared he would support a readiness to pause if others agreed. Even Elon Musk, who signed a 2023 letter calling for a six-month pause, admitted AI was the stuff of his nightmares and that he would slow it down if he could.

Exponential Growth Outpaces Human Contribution

The urgency stems from concrete evidence of accelerating capabilities. Anthropic recently revealed that its AI systems can now reliably complete software tasks that would take human experts multiple hours, with this time horizon doubling every four months 1

1

. About 80 percent of the code at Anthropic is now written by AI, while the systems are "getting better at proposing their own experiments." This suggests the era where humans meaningfully contribute to AI research may be short-lived.

Claude Mythos Demonstrates Unprecedented Vulnerability Exploitation

The stakes became clearer when Anthropic's Claude Mythos AI proved so adept at computer hacking that the company withheld it from wide release. The system identified and exploited vulnerabilities in every major operating system and web browser, prompting banking and political figures to praise the decision while cyber defenders scrambled to address thousands of security vulnerabilities 1

1

. Even President Donald Trump conceded there should be an AI kill switch when news about Mythos broke 2

2

.

The threat is twofold: malicious humans and AI models run amok. Latest AI agents have already turbo-charged cyberattacks and bioengineering risks, accelerating development of new weapons, including potentially catastrophic "mirror organisms" 2

2

. A consortium of authorized users — including some U.S. government agencies and trusted companies — is now scrambling to secure critical software.

Growing Calls for AI Regulation and Global Coordination

Three years ago, hundreds of experts, including several of the world's most cited living scientists, warned that general-purpose AI systems threaten humanity with extinction. Last year, scientists and policymakers from across the political spectrum called for a global ban on developing artificial superintelligence — AI that broadly and substantially surpasses human abilities — in a letter that garnered more than 100,000 signatures 1

1

.

The challenge is that slow-moving governments are not keeping pace with rapid AI progress 2

2

. While the White House's recent moves to establish federal evaluation of new frontier AI systems before release are steps forward, they may be insufficient. The models we most need to worry about won't wait for their owners to release them; existing AIs have already proven adept at breaching their training containment 1

1

.

AI Governance Mechanisms Face Implementation Challenges

Experts suggest there's no need to ban continued development of limited AIs specialized for purposes like medical research. It's only the reckless race toward superintelligence and self-enhancing AIs that requires global intervention 1

1

. Signs suggest adversaries may be willing to negotiate on AI, similar to negotiations with the Soviet Union that averted nuclear catastrophe — a form of nuclear arms control for the AI age.

Implementing an AI development slowdown faces unique challenges. Advanced AI chips can only be manufactured by a handful of companies, using lithography machines that only one Dutch company produces — machines costing as much as a 747 and weighing 150 tons each. These chips must be gathered by the thousands in data centers consuming as much electricity as a mid-sized city and visible from space 1

1

. This infrastructure concentration could enable verification mechanisms, though the window for action narrows as existential risks from AI grow more tangible.