Gemini screen automation will control Android apps, book rides and order food for you

Gemini gets one step closer to controlling other apps on your Android phone

Google could refer to the functionality as "screen automation." AI agents are the hottest trend in tech, and the increasing momentum leaves no room for doubting that your mobile devices will soon be controlled by them. Back at I/O 2025, Google demonstrated how it plans to put Gemini in charge of controlling your phone for you. This was dubbed Project Astra and showed Gemini's capabilities at not just viewing the text and media on your phone, but also scrolling and tapping when needed, basically displaying a spectrum of agentic force in action to help you with simple tasks. While Project Astra is still in its development phase, we're seeing evidence detailing how it might work.

Google wants Gemini to start using your Android apps

Rajesh started following the latest happenings in the world of Android around the release of the Nexus One and Samsung Galaxy S. After flashing custom ROMs and kernels on his beloved Galaxy S, he started writing about Android for a living. He uses the latest Samsung or Pixel flagship as his daily driver. And yes, he carries an iPhone as a secondary device. Rajesh has been writing for Android Police since 2021, covering news, how-tos, and features. Based in India, he has previously written for Neowin, AndroidBeat, Times of India, iPhoneHacks, MySmartPrice, and MakeUseOf. When not working, you will find him mindlessly scrolling through X, playing with new AI models, or going on long road trips. You can reach out to him on Twitter or drop a mail at [email protected]. Google has been working on Gemini's agentic capabilities, enabling it to carry out actions on our behalf. It recently rolled out an AI-powered Auto Browse feature in Chrome for users on its AI Pro or Ultra tiers. The company is also working on bringing similar capabilities to Gemini on Android, letting it control apps directly on your phone. A new report now sheds more light on this feature. Gemini has already made many complex tasks faster and easier. But this is just scratching the surface of what's possible, as the AI will eventually evolve to take actions on our behalf. A teardown of the latest Google app beta (v17.4) by 9to5Google details how the "Get tasks done with Gemini" feature will work. Codenamed bonobo, it will initially be available as a Labs feature, with the ability to book rides and place orders through screen automation. In practice, this could let you ask Gemini to book an Uber to your office or order dinner from Uber Eats without ever opening the app yourself. Reportedly, Gemini's agentic capabilities will only work in "certain apps" initially. Given that app UIs change frequently, it makes sense for Google to initially limit Gemini's screen automation feature on Android to a small set of apps. Beyond food delivery and ride-hailing, early support could be limited to first-party Google apps. The feature will require at least Android 16 QPR3 to work, since Google has laid the groundwork for screen automation with this release. You will always remain in full control While you can ask Gemini to carry out tasks on your behalf, you'll remain in full control and can manually stop or take over at any time. The strings in the code provide more insight into the feature and the privacy implications. Google notes that, "When Gemini interacts with an app, screenshots are reviewed by trained reviewers and used to improve Google services if Keep Activity is on." AP Recommends: Subscribe and never miss what matters Tech insights about everything mobile directly from the Android Police team. Subscribe By subscribing, you agree to receive newsletter and marketing emails, and accept our Terms of Use and Privacy Policy. You can unsubscribe anytime. Google will also warn users not to enter payment information into Gemini Chats and to use screen automation during emergencies. While not mentioned, Gemini's agentic capabilities on Android could be limited to users on the Pro and Ultra tiers. The company will most likely announce this feature in beta alongside the release of Android 16 QPR3 in March.