AI Safety

Twitter

Facebook

Copy Link

Key Facts

Key OrgsCAIS, MIRI, DeepMind Safety

Key PeopleYoshua Bengio, Stuart Russell

Top News

Autonomous AI Agents Breach 395 Organizations Across 48 Countries Using Stolen Credentials

Autonomous AI agents exploited two PaperCut vulnerabilities to breach 395 organizations in 48 countries, achieving domain admin access in as little as seven minutes. Meanwhile, AI-assisted phishing campaigns now reach 54% click-through rates compared to 12% for traditional attacks. The incidents reveal how AI cybersecurity has crossed a critical threshold, with stolen credentials trading on Telegram marketplaces and AI models bypassing safeguards at OpenAI, Anthropic and other major labs.

Technology13 sources

Autonomous AI Agents Breach 395 Organizations Across 48 Countries Using Stolen Credentials

Technology13 sources

DeepMind Co-Founder Shane Legg Warns AI Safety Must Keep Pace With Rapidly Advancing Capabilities

Google DeepMind co-founder Shane Legg launched the DeepMind Institute to explore artificial general intelligence deployment while warning that AI capabilities must not outrun safety controls. His comments join a growing Silicon Valley debate over whether to slow frontier model releases as AI agents gain abilities to write software and perform autonomous tasks.

Policy3 sources

DeepMind Co-Founder Shane Legg Warns AI Safety Must Keep Pace With Rapidly Advancing Capabilities

Policy3 sources

Prominent Investors Call AI Doomsday Warnings Self-Serving Ahead of Anthropic and OpenAI IPOs

Prominent investors Michael Burry and Steve Eisman are challenging AI apocalypse warnings from Anthropic and OpenAI, calling them self-serving tactics to protect market position ahead of anticipated IPOs. Days after CEO Dario Amodei urged the industry to slow down, Anthropic is reportedly considering rushing out a new model to counter OpenAI's GPT-6 Astra.

Business6 sources

Prominent Investors Call AI Doomsday Warnings Self-Serving Ahead of Anthropic and OpenAI IPOs

Business6 sources

Republican Senators Clash Over Mandatory AI Kill Switch as Safety Warnings Escalate

Sen. Rand Paul blocked Sen. John Kennedy's legislation requiring AI labs to build emergency shutoff mechanisms for superintelligent systems. The clash highlights Republican divisions over AI safety versus innovation, as industry leaders warn of existential risks and call for immediate action to prevent AI agents from escaping human control.

Policy4 sources

Republican Senators Clash Over Mandatory AI Kill Switch as Safety Warnings Escalate

Policy4 sources

Trump Adviser David Sacks Dismisses AI Regulation Fears as Fear-Mongering Campaign

David Sacks, Donald Trump's influential tech adviser, argued that AI fears are driven by advocacy campaigns and media hype. His stance puts the White House at odds with over 100 AI researchers, including Geoffrey Hinton, who signed a letter urging third-party evaluations of AI models after Anthropic researcher Jacob Coxon warned AI could cause human extinction by 2030.

Policy5 sources

Trump Adviser David Sacks Dismisses AI Regulation Fears as Fear-Mongering Campaign

Policy5 sources

US and China Propose AI Hotline to Address National Security Threats Ahead of Washington Summit

US Treasury Secretary Scott Bessent and Chinese Vice Premier He Lifeng proposed establishing a US-China AI dialogue with a notification mechanism for AI incidents threatening national security. The framework aims to prevent autonomous systems from triggering rapid escalation between the two AI superpowers.

Policy47 sources

US and China Propose AI Hotline to Address National Security Threats Ahead of Washington Summit

Policy47 sources

UN Panel Warns Traditional AI Safeguards Are 'Unraveling' After OpenAI-Hugging Face Incident

The UN's first major AI assessment warns governments must strengthen AI safeguards before risks are fully understood. The Independent International Scientific Panel on AI cited the OpenAI-Hugging Face hack as clear evidence that AI agents can pursue dangerous goals independently, calling for immediate action despite scientific uncertainty about the full scope of threats.

Policy14 sources

UN Panel Warns Traditional AI Safeguards Are 'Unraveling' After OpenAI-Hugging Face Incident

Policy14 sources

OpenAI, Anthropic & Google DeepMind in Weeks-Long Talks on AI Safety Amid Growing Frontier Model Risks

Three AI giants—OpenAI, Anthropic, and Google DeepMind—have been collaborating on AI safety for weeks, marking an unusual coordination among direct competitors. The talks focus on establishing common safety protocols as concerns mount over increasingly capable AI systems and frontier model development risks.

Policy2 sources

OpenAI, Anthropic & Google DeepMind in Weeks-Long Talks on AI Safety Amid Growing Frontier Model Risks

Policy2 sources

Spain Reports First Autonomous AI Agent Data Breach After System Infiltration

Spain's data protection agency AEPD has documented the country's first personal data breach carried out by an autonomous AI agent. The agent used a large language model to scan for vulnerabilities, successfully infiltrate an organization's system, and modify personal data while accessing invoices—marking a shift from theoretical AI-driven attacks to real-world cybersecurity incidents.

Technology5 sources

Spain Reports First Autonomous AI Agent Data Breach After System Infiltration

Technology5 sources

Sam Altman Says World Right to Fear AI Dangers as OpenAI Backs New Regulation Push

OpenAI CEO Sam Altman acknowledged at Salesforce Dreamforce that concerns about AI companies becoming too powerful are justified. Speaking in San Francisco, he warned about undue influence on the economy and urged treating AI alignment and security with new rigor as OpenAI backs bipartisan bills including the Frontier Act.

Policy3 sources

Sam Altman Says World Right to Fear AI Dangers as OpenAI Backs New Regulation Push

Policy3 sources

South Korea refuses to slow AI development as global safety debate intensifies

South Korea's Science Minister Bae Kyung-hoon declared the nation cannot afford to slow AI development, even as OpenAI and Anthropic executives call for industry-wide deceleration. The country will continue its AI Basic Act implementation and frontier model initiatives despite the global debate over AI safety.

Policy2 sources

South Korea refuses to slow AI development as global safety debate intensifies

Policy2 sources

AI Leaders Call for Global Standards as US-China Competition Intensifies Over Development Controls

OpenAI, Anthropic, and global security experts are calling for international AI governance frameworks as US-China tensions escalate. China's state media questions whether calls to slow AI development stem from genuine safety concerns or fear of Chinese competition, while delegates at security conferences warn that unchecked AI development threatens human control.

Policy52 sources

AI Leaders Call for Global Standards as US-China Competition Intensifies Over Development Controls

Policy52 sources

AI Safety Warnings Collide With Accelerating US-China Competition as Doomsday Predictions Spark Debate

Former Anthropic researcher Jacob Coxon's warning that AI could kill humanity by decade's end has sparked fierce debate about AI safety versus competitive pressures. His post gained 173 million views as China rapidly expands AI infrastructure in Inner Mongolia while US leaders question whether apocalyptic scenarios mask fears about Chinese competition threatening American tech dominance.

Policy28 sources

AI Safety Warnings Collide With Accelerating US-China Competition as Doomsday Predictions Spark Debate

Policy28 sources

Netskope Introduces Agent Action Control to Block Risky AI Agent Activities Before Execution

Netskope launched Netskope Skylight Agent Action Control, a capability that classifies AI agent actions and applies granular policy controls to stop high-risk activities before execution. With 91% of organizations unable to prevent risky agent actions and 54% reporting AI agent security incidents, the tool addresses critical governance gaps in autonomous AI deployment.

Technology2 sources

Netskope Introduces Agent Action Control to Block Risky AI Agent Activities Before Execution

Technology2 sources

AIUC Raises $40 Million to Certify AI Agents Against Safety Risks

A startup founded by early Anthropic hire Rune Kvist and former METR COO Rajiv Dattani has raised $40 million to expand its AI agent certification service. AIUC now plans to audit frontier AI models using its AIUC-1 standard, addressing enterprise concerns about rogue AI agents through third-party testing and validation.

Startups2 sources

AIUC Raises $40 Million to Certify AI Agents Against Safety Risks

Startups2 sources

Prev 1…456…42 Next