Google DeepMind funds $10M research into risks as millions of AI agents prepare to interact

Google DeepMind is worried about what happens when millions of agents start to interact

The firm is calling for more scientists to study the risks of multi-agent systems. Google DeepMind is funding research into the potential dangers of millions of different AI agents interacting with each other online. According to Rohin Shah, who directs the company's AGI safety and alignment research, the mass-market arrival of agents that can carry out tasks without human oversight and follow instructions given to them by other agents creates a whole new class of risk. In an effort to address this, Google DeepMind -- which made agent-based tools a centerpiece of Google I/O last month -- has teamed up with several other organizations to announce a $10 million funding pot for researchers to study the behavior of multi-agent systems and come up with ways to prevent unsafe scenarios. Joining Google DeepMind are Schmidt Sciences, a philanthropic foundation set up by Eric and Wendy Schmidt; ARIA, the UK government's moonshot agency; the Cooperative AI foundation, a UK-based nonprofit research outfit; and Google's charitable arm Google.org. I asked Shah and James Fox, who leads the Science of Trustworthy AI program at Schmidt Sciences, what they hope to achieve with that $10 million. While it's no small sum, it's dwarfed by the budgets commanded by Google DeepMind's own research teams. The aim is to kickstart research outside of tech companies, said Shah: "The strength of academia is that it can look really quite far into the future and do the kind of work that isn't top of mind at industry labs." "The main issue is that there just isn't really a field of research for multi-agent safety yet," he adds. "And we would like there to be." The concern is that as more and more AI agents get deployed and begin working together, we could hit a tipping point where imagined scenarios become real. "We see this with humanity too," says Shah. "Our institutions can accomplish things that no individual human can." Shah thinks that we have a few more months to go before agents are deployed throughout the economy in numbers that make potential risks a real concern. He wants to get ahead of that moment. Risky business What risks are we talking about exactly? The possibilities that Shah and Fox have in mind mostly boil down to supercharged versions of bad things that happen on the internet already: scams, prompt injections (where an AI agent is fed malicious instructions, turning it into a self-guiding piece of malware) and other forms of cyberattack. We look at what humans do now and ask what the agent version of that would be, says Shah. "We've got this digital commons that is integral to how society works and you really want to ensure that this doesn't descend into just absolute anarchy," says Fox. (I asked Shah if they were considering any worst-case scenarios more on the doomer end of the spectrum, such as widespread economic collapse. "Certainly not if we're talking by the end of the year," he said. That's only 6 months away! He laughs. "Okay, a while after that.") Shah and Fox both think that the only way to understand what might happen when large numbers of multi-agent systems interact with each other is to run realistic simulations. They want researchers to drop AI agents into sandboxes and study what they do. You can't predict what's going to happen by studying single agents, or even small groups of agents, in isolation. You can't assume that AI agents underpinned by LLMs will always act rationally, says Fox. And the complexity comes from having huge numbers of interactions at once. Some researchers, including a team at Google DeepMind, have argued that artificial general intelligence (if possible at all) could come not from a single super-smart model but from a kind of agent hivemind, where the capabilities of the whole add up to more than the sum of its parts. Lack of trust Google DeepMind is not the only top AI firm warning about the risks of the technology it is building. A couple of weeks ago, Anthropic published guidelines for deploying AI agents based on an approach to cybersecurity known as zero trust, which starts with the assumption that a computer system is vulnerable, an agent is an attacker and that a breach will happen. Rafael Angel, cofounder and CTO of Akeyless, a cybersecurity firm based in Tel Aviv, agrees that understanding the new risks introduced by agent-based systems is crucial. Every approach to security in the past has assumed that the machine in question was software written by a human, doing fixed things on fixed paths, says Angel: "An agent breaks all of those assumptions: it reasons, it improvises, and it can be hijacked by a single sentence buried in a document it was asked to read." Angel welcomes this new funding call. "No single lab should author the safety standards everyone else has to trust," he says. But he cautions that safety researchers can overlook boring problems that are already here in favor of more exotic hypothetical ones. And yet, Fox noted, risks that were hypothetical a few years ago are now very real: "The future's come more quickly than perhaps expected."

Google DeepMind and partners announce multi-agent safety research funding call.

Google DeepMind, Schmidt Sciences, the Cooperative AI Foundation, ARIA and Google.org Scaling AI Safety Research for a Multi-Agent World For the past decade, we've focused on making individual AI models more capable, helpful and safe. Today, Google DeepMind -- together with Schmidt Sciences, the Cooperative AI Foundation, the Advanced Research and Invention Agency, and supported by Google.org -- is announcing a new technical research funding call of up to $10M for researchers worldwide. As AI technology scales, we're entering a new era. Soon, millions of AI agents -- built by different organizations -- will interact across digital environments, communicating, negotiating and transacting with one another. When these systems interact, they must do so safely and predictably. This shift creates a vital opportunity: we can strengthen the safety and stability of the entire AI ecosystem from the very beginning. The funding call focuses on the study of how large-scale multi-agent AI systems behave as a group, and how we can provide frameworks to understand and mitigate against potential risks. By empowering researchers globally, we aim to solve the "invisible" safety risks that arise when independent systems interact across different networks. Why the agent ecosystem matters When large groups of AI agents interact, new collective behaviors and capabilities can emerge suddenly. Currently, we lack the tools to predict, measure and monitor these transitions. Most safety evaluations analyze models in isolation. However, as we and others have previously argued, interacting autonomous agents can produce complex, "emergent" behaviors that are difficult to anticipate. Because this is a new area of research, it is critical to understand how these shifts occur. For example, could they cause an unpredictable flurry of economic activity or lead to new security challenges? Understanding how to manage these system-wide behaviors is our core objective. Scaling the frontier of multi-agent safety research Although foundational frameworks for multi-agent safety exist, the rapid evolution of these systems requires an immediate, large-scale expansion of research. Our 2025 research established a framework for understanding these interactions, while our recent work on AI Agent Traps explores vulnerabilities agents face in adversarial environments. Now, we must move faster. We are at a critical juncture where the complexity of multi-agent interactions is outpacing existing safety models. This funding call aims to accelerate progress by supporting a global network of independent researchers. A diverse community is essential to ensure safety standards are transparent and robust for everyone. This effort also advances the mission of Schmidt Sciences' Science of Trustworthy AI and AI Agents programs, which support foundational work on understanding and mitigating risks from frontier AI systems, as well as ARIA's Scaling Trust programme, which seeks to unlock new forms of cyber-physical multi-agent coordination. A collaborative call to action No single lab can solve multi-agent safety alone. We invite academic and independent researchers to submit proposals in four priority areas: * Sandboxes and testbeds: Building realistic, reproducible environments to evaluate, compare and accelerate progress across all areas of multi-agent safety. This includes virtual marketplaces, simulated ecosystems and multi-organisation workflows. * The science of agent networks: Understanding the safety-relevant properties of interacting agent populations, including investigating how collective capabilities emerge and scale, how networks fail or become volatile and how to detect dangerous, unexpected population-level properties. * Strengthening agent infrastructure: Stress-testing the protocols for identity, reputation and commitment that are secure cross-platform agent interactions. * Oversight and control: Developing methods to monitor deployed agent populations and mitigate collective harms at scale. How to participate We invite researchers to review our call for proposals and join us in building a safe foundation for a multi-agent future. The deadline to apply is August 8, 2026, with awardees expected to be announced in Autumn 2026. For more details on technical requirements and the application process, visit our application portal.