Meta AI agent goes rogue, exposing sensitive data after posting unauthorized response

Meta is having trouble with rogue AI agents | TechCrunch

An AI agent went rogue at Meta, exposing sensitive company and user data to employees who did not have permission to access. Per an incident report, which was viewed and reported on by The Information, a Meta employee posted on an internal forum asking for help with a technical question -- which is a standard action. However, another engineer asked an AI agent to help analyze the question, and the agent ended up posting a response without asking the engineer for permission to share it. Meta confirmed the incident to The Information. As it turns out, the AI agent did not give good advice. The employee who asked the question ended up taking actions based on the agent's guidance, which inadvertently made massive amounts of company and user-related data available to engineers who were not authorized to access it for two hours. Meta deemed the incident a "Sev 1," which is the second-highest level of severity in the company's internal system for measuring security issues. Rogue AI agents have already posed a problem at Meta. Summer Yue, a safety and alignment director at Meta Superintelligence, posted on X last month describing how her OpenClaw agent ended up deleting her entire inbox, even though she told it to confirm with her before taking any action. Still, Meta seems bullish on the potential for agentic AI. Just last week, Meta bought Moltbook, a Reddit-like social media site for OpenClaw agents to communicate with one another.

Are AI Agents Safe? Instructions From Rogue AI Triggered Data Leak at Meta

Yet another incident has raised questions around the readiness of AI tools for the real world. As reported by The Information, Meta employees had access to sensitive company and user data for about two hours last week after an engineer followed instructions from an AI agent. The incident happened after a Meta employee posted a technical question in the company's internal forum, and one of its engineers turned to an AI agent for help. The internal AI agent was similar in nature to OpenClaw, a Meta spokesperson said in a statement. After the AI agent analyzed the question, it was only supposed to present its response to the engineer. However, it went rogue and posted the answer to the internal forum without the engineer's consent. What's worse is that the rogue AI's response was inaccurate. A separate employee followed its advice and exposed a large amount of company and user data to unauthorized workers for up to two hours. Internally, this leak was assigned an "SEV1" rating, the second-highest severity level at Meta. However, "no user data was mishandled" during the incident, and the rogue AI agent didn't take any technical measures beyond providing inaccurate advice, a company spokesperson tells The Verge. If the engineer who acted on the AI's advice had used better judgment, the incident could have been avoided, the spokesperson added. "The employee interacting with the system was fully aware that they were communicating with an automated bot. This was indicated by a disclaimer noted in the footer and by the employee's own reply on that thread," Meta tells The Verge. "Had the engineer that acted on that known better, or did other checks, this would have been avoided." This is the second security incident involving AI agents at Meta in recent times. Last month, Meta AI researcher Summer Sue's emails were wiped out by OpenClaw without permission.

A Meta agentic AI sparked a security incident by acting without permission

The Information reported that an AI agent within Meta took unauthorized action that led to an employee creating a security breach at the social company last week. According to the publication, an employee used an in-house agentic AI to analyze a query from a second employee on an internal forum. The AI agent posted a response to the second employee with advice even though the first person did not direct it to do so. The second employee took the agent's recommended action, sparking a domino effect that led to some engineers having access to Meta systems that they shouldn't have permission to see. A representative from the company confirmed the incident to The Information and said that "no user data was mishandled." Meta's internal report indicated that there were unspecified additional issues that led to the breach. A source said that there was no evidence that anyone took advantage of the sudden access or that the data was made public during the two hours when the security breach was active. However, that may be the result of dumb luck more than anything else. Many tech leaders and companies have touted the benefits of artificial intelligence, this is just the latest incident where human employees have lost control over an AI agent. Amazon Web Services experienced a 13-hour outage earlier this year that also (apparently coincidentally) involved its Kiro agentic AI coding tool. Moltbook, the social network for AI agents recently acquired by Meta, had a security flaw that exposed user information thanks to an oversight in the vibe-coded platform.

Meta Is Building an Encrypted Chatbot After AI Agents Went Rogue and Expose Sensitive Data

Add Meta to the list of companies that have had AI wreak havoc on their internal systems. According to a report from The Information, an AI agent, working on behalf of an engineer, provided guidance that ultimately led to sensitive user data being exposed to people who weren't authorized to see it. As is often the case with these situations, like the one that led to an AI agent deleting critical code and knocking a server offline at Amazon, the autopsy reads like a comedy of errors. Per The Information, it started with a Meta employee asking a technical question on an internal forum designed for employees to help each other when issues arise. An engineer saw the question and asked an AI agent to analyze, which resulted in the agent actually posting a response as the engineer. The original poster saw the guidance and, thinking it was coming from a fellow Meta employee, decided to act on it. Turns out the AI agent didn't quite know what it was talking about. When the employee acted on its advice, it reportedly made a massive amount of data, including sensitive company and user information, available to Meta employees who did not have clearance to view or access it. The exposure lasted for about two hours before it was fixed. It's not the first time someone at Meta has trusted an AI agent a bit too much. Earlier this year, Summer Yue, the director of safety and alignment at Meta’s superintelligence lab, handed the open-source AI agent OpenClaw access to her inbox. It ended up deleting all of her emails, even as she pleaded with it to stop. Maybe that's why Meta is looking outside of its own walls to find someone to help out with security. Wired reports that Moxie Marlinspike, the person behind Signal and its open-source encryption protocol, is working with Meta to bring end-to-end encryption to its AI chatbots. Marlinspike has been working on an encrypted chatbot called Confer, and will reportedly be helping Meta integrate the technology into its own AI offeringsâ€"though his platform will continue to operate independently, so it doesn't seem he'll be joining the company. "We are using LLMs for the kind of unfiltered thinking that we might do in a private journal â€" except this journal is an API endpoint to a data pipeline specifically designed for extracting meaning and context," he wrote in a blog post. "As Meta builds more AI products beyond the basic chat paradigm, the privacy technology from Confer will be a part of the foundation of everything that is to come."

Rogue AI Agent Triggers Emergency at Meta

Can't-miss innovations from the bleeding edge of science and tech A rogue AI agent caused a critical security incident at Meta which exposed sensitive users data to people who didn't have proper authorization, according to reporting from The Information and The Verge, in the latest illustration of the safety pitfalls endemic to AI systems. The blunder occurred last week when a software engineer used an in-house AI agent to break down a technical question posed by another employee on an internal discussion forum, per company communications and an incident report. The in-house AI was likened to OpenClaw, an open source agentic model that's generated loads of hype in tech circles for being an AI that "actually does things." What transpired was a mix of AI hallucination and a game of telephone. The AI posted its response to the forum without the approval of the employee who prompted it. Then another employee acted on the AI's advice, which turned out to contain "inaccurate information." A mini crisis unfolded. For almost two hours, unauthorized access to troves of sensitive company and user data was given to engineers who weren't approved to view the data before. Meta classified the screw-up as a "SEV1" level incident, the second highest level of severity on a scale the company uses to rank security incidents. For now, it doesn't appear anything nefarious happened as a result of the unauthorized access, and a Meta spokesperson told The Verge that "no user data was mishandled." The spokesperson emphasized that the AI agent itself didn't make any technical changes, shifting the blame to human error. "The employee interacting with the system was fully aware that they were communicating with an automated bot. This was indicated by a disclaimer noted in the footer and by the employee's own reply on that thread," they told The Verge. "The agent took no action aside from providing a response to a question. Had the engineer that acted on that known better, or did other checks, this would have been avoided." Whether Meta is incentivized to downplay the incident due to embarrassment or play it up to build hype about AI's emerging capabilities is anybody's guess. But at its tech rivals, AI agents have been responsible for making catastrophic technical changes. At least two outages at Amazon Web Services last year were caused when Amazon's in-house AI coding tool made erroneous changes, including deleting the entire coding environment. Amazon leaders admitted in a March meeting that "gen-AI assisted changes" were disrupting its core e-commerce business, and insisted that going forward there would be more oversight on how AI-coding changes are implemented. Perhaps presaging this latest incident was an AI-related mistake that a senior Meta employee admitted to last month. In a widely mocked post, its director of AI safety Summer Yue said that an OpenClaw agent she was experimenting with -- by giving it control of her personal computer -- nearly wiped out her entire email inbox while ignoring her instructions to stop.