Jacob Coxon Testifies at NYC AI Hearing as City Pushes for Kill Switches and Whistleblower Rewards

Former Anthropic Researcher Brings Existential AI Warnings to NYC

Jacob Coxon, the former Anthropic researcher who sparked widespread concern about artificial intelligence safety, testified at a New York City Council hearing on AI safeguards, marking a pivotal moment in local AI governance1

1

. Coxon appeared at the request of Council Speaker Julie Menin, who has actively encouraged AI whistleblowers to testify as city lawmakers evaluate comprehensive AI regulations2

2

. After quitting Anthropic last month, Coxon issued stark warnings that "people building AI earnestly believe that it could kill us all by the end of the decade," accusing both his former employer and OpenAI of "gambling with our lives"3

3

.

Tech Giants Face Critics in Same Room

The New York City AI hearing brought together an unprecedented lineup of voices from across the AI landscape. Former Google DeepMind researcher Alex Turner and Daniel Kokotajlo, a former OpenAI researcher who now runs the AI Futures Project focused on societal impact, also testified alongside Coxon3

3

. Representatives from Anthropic, OpenAI, Google, and Meta attended the hearing, though the companies sent policy leads and executives rather than their CEOs4

4

. The tech giants agreed to testify only after lawmakers threatened to subpoena them, with Menin claiming that Google, OpenAI, and Anthropic changed their approach after receiving letters warning of potential subpoena power3

3

. The council also subpoenaed SpaceX's AI unit after it failed to respond to requests to testify4

4

.

NYC Proposes Mandatory Kill Switches and Third-Party Vetting

New York City lawmakers are weighing a package of bills designed to establish unprecedented AI technology safeguards within city limits. One proposal would mandate that companies cannot "market, offer for sale, or deploy" AI systems in the five boroughs without vetting the technology with outside experts through third-party vetting processes3

3

. Another bill would require so-called kill switches for any AI tool used in the city, defined by the council as a "human override that can shut down the system"3

3

. Failure to comply with either regulation would result in a $25,000 fine3

3

. Additional measures would allow people harmed by AI systems to sue developers in certain cases and require independent checks before deployment4

4

.

First-in-Nation Whistleblower Rewards Program Proposed

The City Council is considering creating a "first-in-the-nation" cash reward system for whistleblowers, funded by fines collected from companies violating AI regulations3

3

. One proposal backed by Menin would allow whistleblowers to receive a share of fines or penalties recovered from AI companies following reported violations4

4

. This whistleblower testimony incentive structure aims to encourage insiders to come forward with safety concerns about AI development. The hearing itself represents New York City's push to play a larger role in AI governance, despite much regulatory authority traditionally remaining with federal and state governments4

4

.

Growing Industry Concerns About Existential Risks of AI

Coxon's warnings reflect broader anxiety within the AI community about the existential risks of AI and the pace of development. Anthropic CEO Dario Amodei published a lengthy essay calling on rival companies to jointly slow their development of AI models, expressing concerns about "recursive self-improvement" or RSI, when bots grow smart enough to keep evolving independently3

3

. Leaders including OpenAI's Sam Altman, SpaceX founder Elon Musk, and Microsoft billionaire Bill Gates have also sounded alarms over unregulated AI advancement3

3

. However, President Trump has argued that the bigger risk lies in allowing China to gain a lead in the AI race by slowing development domestically3

3

.

NYC Joins State-Level AI Governance Push

The hearing comes as New York officials at multiple levels seek greater involvement in overseeing an AI industry that has expanded rapidly in the city. New York Governor Kathy Hochul announced last month that the state will require AI companies to register in a government portal starting in November, forcing them to report critical safety incidents within 72 hours3

3

. Officials from NYC Mayor Zohran Mamdani's administration, including Samuel Levine, commissioner of the Department of Consumer and Worker Protection, and Sarah Milstein from the tech office, also testified at the hearing3

3

. Watch how these proposed AI regulations develop, as they could set precedents for other cities grappling with AI-related harm and the need for local AI governance frameworks.