Google deploys User Alignment Critic to secure Chrome's agentic browsing against AI threats

Google details security measures for Chrome's agentic features | TechCrunch

An increasing number of browsers are experimenting with agentic features that will take actions on your behalf, such as booking tickets or shopping for different items. However, these agentic capabilities also come with security risks that could lead to loss of data or money. Google detailed its approach to handling user security on Chrome using observer models and consent for user action. The company previewed agentic capabilities on Chrome in September and said these features will roll out in the coming months. The company said it is using the help of a few models to keep agentic actions in check. Google said it built a User Alignment Critic using Gemini to scrutinize the action items built by the planner model for a particular task. If the critic model thinks that the planned tasks don't serve the user's goal, it asks the planner model to rethink the strategy. Google noted that the critic model only sees the metadata of the proposed action and not the actual web content. What's more, to prevent agents from accessing disallowed or untrustworthy sites, Google is using Agent Origin Sets, which restrict the model to access read-only origins and read-writeable origins. Read-only origin is data that Gemini is permitted to consume content from. For instance, on a shopping site, the listings are relevant to the task, but banner ads aren't. Similarly, Google said the agent is only allowed to click or type on certain iframes of a page. "This delineation enforces that only data from a limited set of origins is available to the agent, and this data can only be passed on to the writable origins. This bounds the threat vector of cross-origin data leaks. This also gives the browser the ability to enforce some of that separation, such as by not even sending to the model data that is outside the readable set," the company said in a blog post. Google is also keeping a check on page navigation by investigating URLs through another observer model. This can prevent navigation to harmful model-generated URLs, the company said. The search giant said that it is also handing over the reins to users for sensitive tasks. For instance, when an agent tries to navigate to a sensitive site with information like banking or your medical data, it first asks the user. For sites that require sign-in, it'll ask the user for permission to let Chrome use the password manager. Google said that the agent's model doesn't have exposure to password data. The company added that it will ask users before taking actions like making a purchase or sending a message. Google said that, in addition to this, it also has a prompt-injection classifier to prevent unwanted actions and is also testing agentic capabilities against attacks created by researchers. AI browser makers are also paying attention to security. Earlier this month, Perplexity released a new open-source content detection model to prevent prompt injection attacks against agents.

Google says Chrome's AI creates risks only more AI can fix

'User Alignment Critic' will review agentic actions so bots don't do things like emptying your bank account Google plans to add a second Gemini-based model to Chrome to address the security problems created by adding the first Gemini model to Chrome. In September, Google added a Gemini-powered chat window to its browser and promised the software would soon gain agentic capabilities that allow it to interact with browser controls and other tools in response to a prompt. Allowing error-prone AI models to browse the web without human intervention is dangerous, because the software can ingest content - perhaps from a maliciously crafted web page - that instructs it to ignore safety guardrails. This is known as "indirect prompt injection." Google knows about the risks posed by indirect prompt injection, and in a Monday blog post Chrome security engineer Nathan Parker rated it as "the primary new threat facing all agentic browsers." "It can appear in malicious sites, third-party content in iframes, or from user-generated content like user reviews, and can cause the agent to take unwanted actions such as initiating financial transactions or exfiltrating sensitive data," Parker wrote. The seriousness of the threat recently led IT consultancy Gartner to recommend that companies block all AI browsers. The Chocolate Factory, having invested billions in AI infrastructure and services, would prefer that people embrace AI rather than shun it. So the ad biz is adding a second model to keep its Gemini-based agent in line. Parker refers to the oversight mechanism "a User Alignment Critic." "The User Alignment Critic runs after the planning is complete to double-check each proposed action," he explains. "Its primary focus is task alignment: determining whether the proposed action serves the user's stated goal. If the action is misaligned, the Alignment Critic will veto it." According to Parker, Google designed the Critic so attackers cannot poison it by exposing the model to malicious content. Enlisting one machine learning model to moderate another has become an accepted pattern among AI firms. Suggested by developer Simon Willison in 2023, it was formalized in a Google DeepMind paper published this year. The technique is called "CaMeL," which stands for "CApabilities for MachinE Learning." Parker adds that Google is also bringing Chrome's origin-isolation abilities to agent-driven site interactions. The web's security model is based on the same-origin policy - sites should not have access to data that comes from different origins (e.g. domains). And Chrome tries to enforce Site Isolation, which puts cross-site data in different processes, away from the web page process, unless allowed by CORS. Google extended this design to agents using tech called Agent Origin Sets that aims to prevent Chrome-based AI from interacting with data from arbitrary origins. The Register understands that Chrome devs have incorporated some of this work, specifically the origin isolation extension, into current builds of the browser, and that other agentic features will appear in future releases. Additionally, Google aims to make Chrome's agentic interactions more transparent, so user directives to tackle some complicated task don't end in tears when things go awry. The model/agent will seek user confirmation before navigating to sites that deal with sensitive data (e.g. banks, medical sites). Also, the robo-browser will also seek confirmation before letting Chrome sign-in to a site using the Google Password Manager. And for sensitive web actions like online purchases, sending messages, or other unspecified consequential actions, the agent will either ask for permission or just tell the user to complete the final step. To ensure that security researchers put Chrome's agentic safeguards to the test, Parker says Google has revised its Vulnerability Rewards Program (aka bug bounties) to offer payouts for folks who find flaws. "We want to hear about any serious vulnerabilities in this system and will pay up to $20,000 for those that demonstrate breaches in the security boundaries," said Parker. ®

Google Chrome adds new security layer for Gemini AI agentic browsing

Google is introducing in the Chrome browser a new defense layer called 'User Alignment Critic' to protect upcoming agentic AI browsing features powered by Gemini. Agentic browsing is an emerging mode in which an AI agent is configured to autonomously perform for the user multi-step tasks on the web, including navigating sites, reading their content, clicking buttons, filling forms, and carrying out a sequence of actions. User Alignment Critic is a separate LLM model isolated from untrusted content that acts as a "high-trust system component." Gemini is Google's AI assistant, that can generate text, media, and code. It is used on Android and various Google services, and integrated into Chrome since September. At the time, Google announced plans to add agentic browsing capabilities in Chrome via Gemini and now the company is introducing a new security architecture to protect it. The new architecture, presented in an announcement from Google's engineer Nathan Parker, mitigates the risk of indirect prompt injection, where malicious page content manipulates AI agents into performing unsafe actions that lead to user data exposure or fraudulent transactions. Parker explains that the new security system involves a layered defense approach combining deterministic rules, model-level protections, isolation boundaries, and user oversight. The main pillars of the new architecture are: Google's layered defense approach towards agentic browsing shows that the company is more careful about giving its LLMs access to the browser than vendors of similar products, who researchers showed to be vulnerable to phishing, prompt injection, and purchasing from fake shops through prompt injection attacks. Google has also developed automated red-teaming systems that generate test sites and LLM-driven attacks to continuously test defenses and develop new ones where required, pushed quickly to users via Chrome's auto-update mechanism. Finally, Google has announced bounty payments of up to $20,000 for security researchers who can break the new system, calling the community to join in the effort to build a robust agentic browsing framework on Chrome.