Google Gemini Live Set to Revolutionize File Interactions and Video Queries

Two of Google Gemini's best features might be coming together

Gemini's newest extension finally makes it a decent Google Assistant alternative Key Takeaways Google is reportedly working on bringing Gemini Live support for 'Ask about this video' queries on Youtube. With Gemini Live support, users will be able to ask queries about YouTube videos and gain answers in a natural and conversational manner. This feature is still in the development stage, and it's unclear if it will completely replace the traditional text response option. ✕ Remove Ads Earlier this year, in May, at Google's annual I/O developer conference, the tech giant showed off an exciting new addition to its suite of features for Gemini on smartphones. One of those features, 'Ask about this screen,' and/or 'Ask about this video' became available in August -- allowing the AI tool to gain on-screen contextual abilities. This not only allowed the tool to understand what's happening on your device's screen, but also allowed users to probe the AI tool about it. Related Google Gemini finally lets you 'Ask about this screen' and summarize YouTube videos You can try out the assistant feature today Another key Gemini feature first unveiled at I/O is Gemini Live. The tool, which lives inside the Gemini app (which recently arrived on iOS), is a conversational AI assistant with natural-sounding spoken dialogue capabilities. Now, it looks like Gemini's 'Ask about' on-screen contextual abilities and Gemini Live are coming together. ✕ Remove Ads As highlighted by Android Authority, Google might be working on bringing Gemini Live support for 'Ask about this Video' queries. For reference, currently, when you ask Gemini about a YouTube video, you're presented with a block of text as the tool's reply. You can, of course, hear out Gemini's reply, but that's not much of a 'conversation.' No release timeline for now <string name="assistant_robin_conversation_mode_youtube_chip_type">YouTube</string> <string name="assistant_robin_conversation_mode_volume_dialog_message">Before going Live, increase your device's volume so you can hear Gemini</string> Code spotted in version 15.46.31.ve.arm64 beta of the Google app, however, indicates that users will likely soon be able to enter 'conversation mode' when tapping the Ask about this video chip. Conversation mode, as suggested in the code, is tied to Gemini Live, which should allow users to interrupt the assistant, ask follow-up questions, and in general, have the AI tool answer your queries in a more conversational manner. "Before going Live, increase your device's volume so you can hear Gemini," reads a string. ✕ Remove Ads It's worth noting that the functionality has only been spotted in code, and there are currently no screenshots of its implementation. Hence, if it does materialize, it is unclear if Gemini Live will outright replace the traditional Ask about this video experience, or if Google will offer both options.

Gemini Live Might Soon Answer Queries About Your Files

Google made Gemini Live available for all Android users recently Google is reportedly working on adding another new functionality to its Gemini chatbot. The new artificial intelligence (AI) feature is said to be coming to Gemini Live, the two-way verbal conversation feature that offers a hands-free experience of the chatbot. As per the report, the Mountain View-based tech giant is working on adding Gemini Live support to files being uploaded to Gemini. Currently, users can only interact with such content via text, but it might soon be available over voice chats. Android Authority reported about the new Gemini feature. The publication found evidence of the feature during the application package kit (APK) teardown of the Google app beta version 15.45.33.ve.arm64. Several strings of code reportedly point towards the development of this new capability for Gemini Live. As per the publication, the strings highlight phrases such as "Open Live", "Talk about attachment", and "Open Live with attachment". Here, 'Live' likely refers to Gemini Live, and 'attachments' refer to the files that users upload. With this capability, users might be able to use Gemini Live to talk about their uploaded documents and spreadsheets, which is currently not possible. This will make it easier for users to seek insights from text-heavy documents while not being tied down to the Gemini interface. However, the feature is not expected to be available for all users. Currently, only Gemini Advanced subscribers can upload files to Gemini and ask questions about them. So, the Gemini Live support is believed to be for the paid subscribers using Android devices since it is not available on the web. Notably, people can subscribe to Gemini Advanced via the Google One AI Premium plan, which costs Rs. 1,950 a month. Gemini Live was first unveiled by the company at the Google I/O event earlier this year. The tech giant first rolled it out for the paid subscribers in August. Later, it was released for all Android users the next month. The voice-based two-way communication feature also supports Hindi and eight regional Indian languages.

Google thinks attachments could be the perfect conversation starter for Gemini Live

Key Takeaways Gemini Live might soon be able to handle and interact with your files. Code found in the Google app beta suggests Gemini Live will prompt users to discuss uploaded files in a conversational way. Google is laying down the groundwork for the feature, with its release timeline currently uncertain. ✕ Remove Ads Made by Google 2024 was dominated by the tech giant's latest Pixel 9 lineup, but AI advancements weren't far behind. At the event, Google unveiled Gemini Live, a conversational AI assistant with natural-sounding spoken dialogue capabilities. The feature, which started off as a premium one exclusive to Gemini Advanced subscribers on Android, eventually made its way to free users -- and iOS support doesn't seem to be too far behind. Related Google Gemini could soon get a dedicated iPhone app Google is testing the app in one country 1 After learning over 40 new languages in early October, Google now seems to be prepping Gemini Live with support for handling and interacting with files. ✕ Remove Ads Currently, the regular Gemini chat interface allows users to upload files, with Gemini Advanced users being able to task the AI with analyzing the files or make changes to them. However, in Google app's beta version 15.45.33.ve.arm64, the folks over at Android Authority were able to spot code related to Gemini Live being able to handle files -- including an option to 'Talk about [the] attachment.' This might just be NotebookLM within Gemini Live <string name="assistant_zero_state_suggestions_open_live_snippet_highlight">Open Live</string> <string name="assistant_zero_state_suggestions_open_live_snippet_simplified">Talk about attachment</string> <string name="assistant_zero_state_suggestions_open_live_text">Open Live with attachment</string> From the looks of it, when you upload a new file on the regular Gemini interface, the AI tool will automatically prompt/suggest you to open the attachment via Gemini Live for a more interactive approach. ✕ Remove Ads Other code snippets reference 'Open Live with attachment' and 'Talk about attachment,' further reiterating that Google might prompt users to move over to Gemini Live when they upload a new file. While not explicitly hinted at, it is likely that Gemini Live will be limited to discussing the attached file with you, helping you understand the document's key points in a conversational manner -- something akin to the NotebookLM experience. The ability to manipulate and/or make changes to attachments will likely be exclusive to the regular chat interface for Gemini Advanced users. The code is likely just the groundwork for the feature's eventual rollout sometime in the future, and considering that the feature couldn't be activated, it is all the more likely that Google will take its sweet time with it. ✕ Remove Ads

Gemini Live is getting ready to chat with you about your files (APK teardown)

Gemini will introduce this feature by suggesting moving to Live when you make an upload. Google's Gemini AI isn't just around to answer your questions or help you generate pictures, and it's also capable of lending a hand to get some work done. Gemini Advanced users are able to upload files, from regular text documents to complicated spreadsheets, and have the AI modify them or just summarize the information within. Today we're taking a look into what could be the next step in the evolution of this feature, as Google gets ready to start letting you talk to Gemini Live about these files.

Prepare to chat about your files with Gemini Live

Google's Gemini Live is set to revolutionize the way users engage with uploaded files, moving beyond simple question-and-answer interactions. Currently catering to Gemini Advanced users, the platform will soon facilitate direct conversations about uploaded files, enhancing productivity and user experience. Although the feature isn't functional yet, hints in the latest APK teardown by Android Authority suggest that Gemini Live will prompt users to engage with their uploaded content, paving the way for more interactive and contextually relevant assistance. Gemini Advanced users have already enjoyed the ability to upload a variety of files, such as text documents and spreadsheets, for the AI to modify or summarize. Those familiar with this feature will be excited to hear about the anticipated integration of Gemini Live, which aims to create a more conversational and contextual environment. The essential nuance here is that Gemini might recognize when a file is uploaded -- whether from a local drive or directly through Google Drive -- and recommend engaging with the Live feature to maximize its usefulness. Specifically, code strings found in the latest beta of the Google app -- version 15.45.33.ve.arm64 -- reveal prompts like "Talk about attachment" and "Open Live with attachment." This indicates that Gemini Live could soon leverage its conversational nature to assist users more effectively. "Starting Conversation Mode with empty attachments, but expected attachments to be present" even hints that further integration may be on the horizon, leaving users wondering how this will unfold and what level of interaction they can expect. Furthermore, Google's Gemini platform is in the midst of enhancements that align with the evolving needs of its users. Notably, some integrations allow for the automatic recognition of file updates. This development will ensure that users get real-time assistance based on the most current version of their uploaded documents. In conjunction with the arrival of Gemini Live, Google has introduced "Gems" -- custom instances of Gemini aimed at specific tasks. This feature allows users to create tailored responses by uploading up to ten files, thus enhancing the contextual relevance of the output. Files can range from various document formats including Google Docs, PDFs, and even spreadsheets like Google Sheets and Excel files. Google has stated that Gems are "one of the most used Gemini Advanced features" since their introduction, highlighting the growing adherence to custom-tailored AI experiences. These Gems can offer support across diverse workplace scenarios such as refining corporate style guides, enhancing project-specific assistants, and streamlining HR document access. The implications of this are vast. For instance, a marketing team could quickly draft on-brand content or utilize sentiment analysis to gauge customer feedback. The ability to integrate specific files ensures that outputs are not just context-aware, but also aligned with the latest developments or revisions within those documents. Google's rollout of these features is particularly significant for Workspace subscribers, who will see the introduction of several premade Gems aimed at various professional needs. These include marketing insights for calculating customer acquisition costs, crafting compelling sales pitches, and even hiring consultations for consistent job descriptions. By integrating AI into the workflow, Google aims to provide tools that not only save time but also enhance productivity and creativity across teams. Imagine a corporate atmosphere where the tedious task of drafting proposals or sending customer communications could be handled by an AI consultant, which can ingest the required information and suggest tailored responses that reflect real-time updates in company policies or client data. This is not just a dream; it is fast becoming a reality with Gemini's progressive tools. As Google continues its enhancements, users are poised to gain immediate access to tools that not only save time but also promote organizational efficiency. The ability to use Gemini AI for everything from financial forecasting to educational content creation signifies a shift toward a more integrated and user-friendly AI experience. With all these developments, the excitement about Gemini Live and the introduction of custom Gems is palpable. As users await the rollout of these features, it becomes clear that Google is not just transforming how we interact with AI but how we utilize technology to enhance our workflow. Preemptively setting the stage for file-based interactions with Gemini Live is only a part of a larger ecosystem that promises efficiency and ease in various professional realms.

Gemini may get more chatty about YouTube videos so you can dive deeper (APK teardown)

An APK teardown helps predict features that may arrive on a service in the future based on work-in-progress code. However, it is possible that such predicted features may not make it to a public release. When you open the Gemini overlay while watching a YouTube video, the AI assistant will have a contextual chip above it that says "Ask about this video." Tapping on this chip will allow you to ask questions about what's in the video. For example, if you wanted to know which type of laptop is better in our recent Snapdragon X Elite or Lunar Lake laptop video, you could ask that question and the AI would do its best to summarize the answer.