Google's Gemini 2.0 AI Agents: A New Frontier in Gaming Assistance

Playing your favorite games on phones could include AI advice thanks to Gemini 2.0 -- here's how

Google has officially announced Gemini 2.0 to the public, and with it comes word that the tech giant is testing a new feature to help with gaming. Google Gemini has become a huge part of the average Android user's life, and a new blog post has revealed that it could soon impact gaming as well. Google has built AI agents using Gemini 2.0 that "can help you navigate the virtual world of video games." The agents are able to reason about a game based on what is on screen, as well as connect to Google Search to offer information. The blog contains a video that shows off the agents at work helping three people playing Clash of Clans, Hay Day, and Squad Busters. We get to see the AI offering the ideal army composition based on the available units displayed on the screen. The demonstration also shows the connection to Google Search, all of which is done by verbal commands. Finally, the video also shows that you can ask for reminders about quests and more beyond what is happening in the moment. Similar features are possible with the recently announced ChatGPT Advanced Voice with Vision, which can see your phone screen and talk you through what is happening. It doesn't yet have search access, but that is coming soon. The idea is interesting, but on the other hand, the video makes it seem a bit limited at the moment. For instance, do we really need an AI to remind us about daily quests, especially in a game like Clash of Clans? I can see the advantage of being able to check the meta with a simple request, even though Squad Busters isn't a massive meta-focused game. I think the big issue with the video is that the games on display aren't that complex at the best of times, so the agents really aren't needed. Gemini 2.0 is a pretty major enhancement, with CEO Sundar Pichai saying Gemini 2 has advanced reasoning capabilities that will soon come to AI Overviews in Google Search. Gemini 2.0 has been dubbed the starting point for the "agent era" by Google and the model is apparently comparable to OpenAI's o1 model. For reference, o1 has been stated to be one of the smartest programs ever produced, to the degree it's even been found to be capable of plotting. The AI agents are an interesting concept, but until we see it working with more involved games then it might appear as more of a gimmick than anything. However, if it is capable of offering sound advice and ideas then it could help to improve play on some of the best Android phones.

Google's Gemini 2.0 AI agents are being trained to offer gameplay advice and suggestions

This week, Google is adding to its Gemini AI model with the introduction of Gemini 2.0. One of many new Gemini features highlighted in the 2.0 Update blog post points toward a potential future where Google's AI assistant even helps you while you're playing games. Project Gemini 2.0 agents have been tested with games like Clash of Clans and trained with the help of developers to recognize scenarios and be able to give advice and input based on what's currently happening onscreen. Gemini also shows itself to be able to scrape the web for information like character picking and even tier lists, and does at least mention the platform where it's getting that information. The idea of AI being able to competently give gaming advice seems pretty harmless and cool, especially if the insight is being shaped by the original developers in some way. Developers of Clash of Clans Supercell are reportedly collaborating directly with Google on these tests, which means that the developers could provide information on game systems, mechanics, and statistics that Gemini can refer to when giving its insights. The demo shown sees Gemini giving valid team composition advice based on the player's available resources -- information that almost certainly requires developer insight in addition to web search data. But how practical can something like this really be? For truly insightful advice, especially for games in genres with lots of advanced, undocumented mechanics and strategy, a robust community of existing players is often the only way to advance past a certain entry level of skill. And for one-off advice, a quick web search for an existing walkthrough is sure to be much quicker and more accurate than something generated by AI, though these walkthroughs/guides may not exist for brand-new games or niche titles. Ultimately, there's nothing particularly offensive about this potential new addition to Gemini 2.0, but -- as with a lot of AI-related tech -- it's still unclear if this would work as advertised and/or would be useful even if it did. Also, what opportunities for community, connection, and advancement are future gamers losing if we automate the process of finding tips and strategies? (Fortunately for the most hardcore, competitive audiences, that initiative to do the extra work won't stop being a requirement for top-level play any time soon.)

Google is testing Gemini AI agents that help you in video games

Google just announced Gemini 2.0, and as part of its suite of news today, the company is revealing that it's been exploring how AI agents built with Gemini 2.0 can understand rules in video games to help you out. The agents can "reason about the game based solely on the action on the screen, and offer up suggestions for what to do next in real time conversation," Google DeepMind CEO Demis Hassabis and CTO Koray Kavukcuoglu write in a blog post. Hassabis and Kavukcuoglu also say that the agents can also "tap into Google Search to connect you with the wealth of gaming knowledge on the web." Google is testing the agents' "ability to interpret rules and challenges" in games like Clash of Clans and Hay Day from Supercell, according to Hassabis and Kavukcuoglu. I'm not surprised Google is chasing these ideas: in theory, an AI agent coaching you through a strategy or puzzle could be useful. It sounds like this work is very early, though, and I have many questions about whether or not these agents actually give sound advice. Google is also investing in video games and AI in another way: creating playable virtual worlds on the fly from a prompt image using a "foundation world model" called Genie 2 that it showed off last week. That work seems early, too: Genie 2 can only generate consistent worlds for "up to a minute," Google says.

Google's Gemini 2.0 Helps You Dominate Video Games, Look Up Info...

We may earn a commission when you click links to retailers and purchase goods. More info. Alongside Google's Gemini 2.0 announcement and that impressive Project Astra demo, Google showed off an idea they have for video games and how someone could use Gemini as an assistant for help as they play. I'm not talking about using AI to play for you, but instead having AI there to remind you of things, help you with strategy, or to potentially look up information that could help as you play. Google says it is collaborating with game developers to figure out ways that they could utilize Gemini. Games like "Clash of Clans" and "Hay Day" were used in a demo where a virtual assistant is essentially watching as the game is played to take in info and be ready for requests. In one example in this demo, a player asks Gemini as they play to identify quests they need to complete for the day and then remind them later to do so. In another demo example, a gamer asks for help building out the proper troop setup in "Clash" to go on an attack, with Gemini attempting to describe the best way to do that with a breakdown of their reasoning for the composition. One user also asked Gemini to look up the current "meta" and tell them about the best characters that everyone is using. Gemini returned with a response they found on reddit for which character to play. While some of those ideas would probably only be useful when you are first starting out a game and learning how to play it, it's that Reddit example that sticks out to me as being super helpful at any moment. Google says that these AI virtual gaming companions can tap into Google Search, which is where the Reddit info came from. I could have used this yesterday when my kid, who has recently taken up playing Fortnite and wants me to play with him, was wondering where we could find a new item location in the game to complete quests. I had to stop playing and actively look it up to then relay the info. If I could have accessed a virtual game companion at that moment through my headset, this all would have been so much easier and not risk getting eliminated. I'd imagine Google has other plans beyond these few examples and I'm sure you can come up with your own. Here's to hoping that AI remains as an informational tool when it comes to games and not much else.