Google Expands Gemini AI's 'Ask About This Screen' Feature to More Android Devices
2 Sources
2 Sources
[1]
Google Gemini introduces "Ask about this screen" feature on Android, note taking feature on Google Meet
Gemini has rolled out two new features for Android that were showcased earlier at Google I/O this year. Called 'Ask about this screen," the feature allows users to have queries about whatever is happening on the screen. There's also an "Ask about this video" feature which is compatible with YouTube. The prompt appears on top of the Gemini window after the user has summoned the AI assistant. Once the user taps on the icon a temporary screenshot is captured and then the user can type in their query. While the bot's initial response will be based on what the screenshot contains, the user can then ask Gemini to access the internet for more details or even summarise a news article. The system does not save these screenshots. For the "Ask about this video" feature, the bot appears to look through the video captions to describe the contents of the video. The features were spotted by Android Police on Gemini beta app version 1.0.662093464 and the Google beta app version 15.34.32. Google Workspace app to let users create video presentations using Gemini-AI Gemini in Workspace Aside from Android devices, Gemini will also be integrated into Google Meet for their new "Take Notes for Me," feature for summarising meetings. The feature is currently available just in English and will be rolled out to Google Workspace clients with Gemini Enterprise, Gemini Education Premium, AI Meetings and Messaging add-ons. The tool automatically makes notes in a Google Doc and will attach the file in the calendar event after the meeting is over. It can also share the file with anyone who turns the feature on and the meeting organiser. If someone is late to a meeting, it will fill you in once you've joined via links of files along with the Google Doc. Google said it will complete rolling the feature out by September 10. Gemini in WhatsApp Separately, Android Authority reported seeing Gemini extensions for WhatsApp, Google Messages and Android system notifications, in the 15.34.32.29.arm64 beta. The bot could help users send and read messages and make calls on WhatsApp using voice commands according to the description of the feature. Read Comments
[2]
Google expands Gemini AI's onscreen awareness capability to more phones
Google is reportedly rolling out new Gemini AI assistant features on Android smartphones that allow users to ask Gemini questions related to what's on their display. Additionally, Gemini is gaining the ability to offer YouTube video summaries and respond to queries related to specific YouTube videos. These features have been available on the latest Pixel 9 series smartphones; however, it seems they are now rolling out to other Android smartphones as well.
Share
Copy Link
Google enhances Gemini AI with expanded 'Ask About This Screen' feature for Android and introduces note-taking capabilities in Google Meet, aiming to improve user productivity and information accessibility.
Google Expands 'Ask About This Screen' Feature
Google has announced a significant expansion of its Gemini AI's 'Ask About This Screen' feature, making it available on a wider range of Android devices. Initially exclusive to Pixel 8 Pro, this functionality is now accessible on devices running Android 12 or newer versions 1. The feature allows users to interact with Gemini AI about the content displayed on their screen, enhancing the accessibility of information and improving user productivity.
How 'Ask About This Screen' Works
The 'Ask About This Screen' feature enables users to ask questions or seek explanations about the content visible on their device's screen. By simply long-pressing the power button or using the home button, users can activate this feature and engage with Gemini AI for instant insights 2. This expansion brings advanced AI capabilities to a broader audience, potentially transforming how users interact with their devices and access information.
Enhanced Note-Taking in Google Meet
In addition to the expanded 'Ask About This Screen' feature, Google has introduced a new note-taking capability powered by Gemini AI in Google Meet. This feature allows users to generate notes automatically during video calls, summarizing key points and action items 1. The AI-generated notes can be easily shared with participants after the meeting, streamlining collaboration and ensuring important details are not missed.
Privacy and User Control
Google emphasizes that user privacy remains a top priority. The 'Ask About This Screen' feature processes information locally on the device, ensuring that sensitive data is not shared externally. Users have full control over when to activate the feature, maintaining their privacy and data security 2.
Implications for AI Integration in Mobile Devices
The expansion of Gemini AI's capabilities on Android devices signifies a growing trend of integrating advanced AI functionalities into everyday mobile experiences. This move by Google not only enhances user interaction with their devices but also sets a new standard for AI-assisted features in the mobile industry 1.
Availability and Future Prospects
While the expanded 'Ask About This Screen' feature is rolling out to compatible Android devices, the note-taking feature in Google Meet is currently in the testing phase. Google plans to make these features more widely available in the coming months, potentially revolutionizing how users interact with their devices and manage information during video calls 2.
Anthropic Uncovers 'Vibe Hacking': AI-Powered Cybercrime Reaches New Heights
Anthropic reveals sophisticated cybercriminals are using its Claude AI to automate and scale up attacks, including a large-scale data extortion campaign targeting 17 organizations.
12 Sources
Technology
23 hrs ago
12 Sources
Technology
23 hrs ago
Google's Pixel 10 Series: AI-Powered Innovations in a Familiar Package
Google's latest Pixel 10 series showcases significant AI advancements while maintaining familiar hardware, offering a blend of innovative features and reliable performance.
35 Sources
Technology
15 hrs ago
35 Sources
Technology
15 hrs ago
China's Ambitious Plan to Triple AI Chip Production and Reduce Dependency on Nvidia
China aims to significantly increase its AI chip production capacity, with plans to triple output by 2026. This move is part of a broader strategy to reduce dependence on foreign technology, particularly Nvidia, and develop a robust domestic AI ecosystem.
5 Sources
Technology
23 hrs ago
5 Sources
Technology
23 hrs ago
OpenAI and Anthropic Collaborate on Groundbreaking AI Safety Testing
OpenAI and Anthropic, two leading AI labs, conducted joint safety testing on their AI models, revealing insights into hallucinations, sycophancy, and other critical issues in AI development.
2 Sources
Technology
23 hrs ago
2 Sources
Technology
23 hrs ago
Nvidia's Lackluster Forecast Raises Concerns About AI Industry Slowdown
Nvidia, the world's most valuable public company, provides a tepid revenue forecast, sparking fears of a potential slowdown in AI spending. The forecast excludes China data center revenue due to US export restrictions.
2 Sources
Business
23 hrs ago
2 Sources
Business
23 hrs ago
Your Daily Dose of Curated AI News
Don’t drown in AI news. We cut through the noise - filtering, ranking and summarizing the most important AI news, breakthroughs and research daily. Spend less time searching for the latest in AI and get straight to action.
