Google's New Pixel Screenshots: AI-Powered Organization with Privacy in Mind

Google Pixel 9 has lots of AI features -- but this one is the best by far

Pixel Screenshots looks like a top addition to Google's new flagships The latest Google Pixel 9 devices boast a bunch of AI-powered capabilities, from inserting different people into group shots to generative image editing. But during my Pixel 9 demo time this week, it became pretty clear right away that the Pixel Screenshots tool is the one I'm going to be using the most once I get my hands on a new Google phone. I take a lot of screenshots the various smartphones that I use. More than a few of those are tied to how-to articles I write that ilustrate different smartphone features, but screenshots serve other purposes, too. Sometimes, they include information I'm too busy to copy into a note. Other times it's funny headlines I like sharing with my wife or gift ideas I want to remember later on. But here's the problem with that approach -- often times I take those screenshots and then forget completely about them. Or even worse, I remember them but can't track them down and spend more time than I care to admit swiping through the images stored in the screenshot album of my default photo app. So much for using screenshots as a time-saver. It turns out I'm not the only person who uses their phone this way, at least according to the Google employee showing off the Pixel Screenshots feature to me on a Pixel 9 Pro XL. Apparently enough people are capturing enough screenshots that they later have difficulty finding for Google to build a specific tool that searches for screenshots by the content they include and retrieves them for our convenience. There's a dedicated Pixel Screenshots app on the new Pixel phones where you can access all the screenshots you've capture, but you can just as easily search for screenshots using Gemini, which is also included on Google's new phones. Type in a search word of phrase -- say "Disneyland" if you're looking for that screenshot of a retro Disneyland poster you could have sworn you captured as a future gift idea -- and Pixel Screenshots will show you a series of thumbnails that you can tap if it's the image you've been trying to track down. This all happens pretty quickly, at least in the Pixel Screenshots demo I saw. And searches take place entirely on your device, so Google's not peaking in on whatever screenshots you've got stored or the search terms your typing into your Pixel. Pixel Screenshots would be a clever enough addition to the Pixel 9 if all it did was help you find things you were looking for. But Google threw in a number of nice touches that make the feature even more useful. When you take a screenshot on your Pixel 9, for example, you can set a reminder about it, reducing the risk of forgetting about what you've just stored on your phone. You can also add tags and notes that are also searchable when you need to track things down again -- handy for retrieving all of your gift ideas at once, just as an example. Of course, if that level of involvement doesn't appeal to you, you can just take a screenshot as you always would. It's automatically routed to the Pixel Screenshots app for future searches even if you do nothing at all. I'm also impressed by the amount of detail that appears along with your screenshot. In my demo, I saw a screenshot of a pair of boots that were on sale. A separate notes field gathered up relevant information about those boots -- the available sizes, the materials used, even a promo code to knock a few bucks off the price. Pixel Screenshots also supports a deep link capability where there will be a link back to the original screen if it's something that's still availability like a website. Granted, everything I saw involving Pixel Screenshot came in a canned demo designed to highlight the feature's strengths. Things may not go so smoothly in real-world use -- it may take longer to find what I'm looking for, it's unclear how conversational and informal your search terms can be and results may not always turn out to be as complete as what I saw in the demo. Still, it's an impressive addition to the Pixel 9, even if I do run into some growing pains during real-world use. The push toward AI features on mobile devices has introduced a number of new capabilities in the past year, some of which are flashier than others. There's nothing particularly breath-taking about retrieving screenshots based on their content -- it's just one of those features that removes some of the hassle of an everyday phone task. And I think that's what I like best of all about Pixel Screenshot.

Google's new Pixel Screenshots may be the feature that finally converts me to use AI

It won't turn my boring office background into a field of flowers, but being able to parse through hundreds of screenshots with a few clicks feels like a major win. According to the gallery app on my phone, I have 361 screenshots saved, including billing statements, important dates, memes, and web articles I've always wanted to revisit but didn't trust my browser to bookmark. If you asked me to find a specific image in the album, you'd have to give me a minute. Maybe longer. Also: Everything announced at Made by Google 2024: Pixel 9 Pro, Fold, Gemini, Watch 3, and more In theory, Google's new Pixel Screenshots feature should expedite that process, allowing me to simply type keywords or questions like "Con Edison bill for September" or "Which stadium is next week's soccer game taking place?" and the phone will populate the most relevant screenshots. It works like magic. Or, should I say, it works like AI? The AI part of Pixel Screenshots fits what I'll call the "old definition" better than the new one, meaning it has more to do with automated backend processing than content generation. You're not redefining a picture here, creating new emojis, or generating captions for images; instead, the Pixel 9's Gemini Nano extracts as much information as possible from screenshots, stores it, and then retrieves it when prompted. The whole Pixel Screenshots process happens on-device, so the internet and all its dangers are not involved when using the feature. Google tells me it plans to keep things that way for security and privacy reasons (read: the company would like to avoid a situation similar to Microsoft's Recall debacle), and I'm glad it is. Also: I went hands-on with every Google Pixel 9 model, and this is the one you should buy Besides screenshotting more things than I probably should -- because it's much easier to press two buttons simultaneously than to download a full webpage or save a URL in a notepad -- I often don't think about how much personal information ends up being captured in the frame. A skim through my phone's screenshots album reveals home addresses, usernames of close friends and family, contact numbers, and other trinkets of information that I'd rather not have fall into the wrong hands. I digress. From my brief demos of Pixel Screenshots, I'm most impressed by three things: the speed with which the phone pulls out image results (because this all works locally), the ability to upload and capture more images for the sake of future retrieval, and how seamlessly the feature works with natural behavior. To that last point, you don't have to purposely label or manually transfer screenshots for the feature to work; everything you capture automatically flows into the dedicated app. Also: Google's Loss of Pulse Detection feature on Pixel Watch 3 could save your life Some finer details tell me just how much thought was put into Pixel Screenshots, such as the ability to quickly launch the URL that the image was captured in on Chrome or YouTube and the option to set a reminder when you initially screenshot something. This feels bigger than just an archiving feature; it can potentially change the way we interact and bookmark digital content. At a time when smartphone makers are spending more time pitching AI features than camera hardware, Pixel Screenshots is a rare winner. I'm nearly sold on Google's AI vision here, and that may be just enough to reel me into the Gemini universe.

Google Pixel Screenshots App Isn't the Privacy Nightmare Like Microsoft Recall

Pixel Screenshots app will let you categorize, search through, and add notes to your screenshots. Best of all, it's not auto-screenshotting everything on your phone. The incoming slate of Pixel 9 phones and the Pixel 9 Pro Fold will be up to their gills in AI features. Of all the Magic Editor and Gemini mumbo jumbo, the one AI capability that stood out is a mere app to store your screenshots. Pixel Screenshots is a Google phone-exclusive application that will let you search through your screengrabs with some AI assistance. After watching it in action, I see it as a simplified version of Microsoft’s maligned Recall feature, and so much for the better. My screenshots are like barnacles adhered all over my organized Google Photos app. I take screen grabs as a catch-all for events, notes, and receipts I need to remember. I could remove them from Photos, but that would mean I might miss out on something I'll need later. Pixel Screenshots solves this issue by offering a specific app to search through my screen grabs. The feature uses Gemini Nano to identify text and comprehend images in the app. If I ask it to search for how much my friend owes me in drinks from last night, it should be able to pull up that info. Google told Gizmodo the entire process works on-device and doesn’t require any cloud-based work. Every time you take a screenshot, the app generates a title and summary for the image. You could add a note to the screenshot to help the AI-enhanced search find your content. Best of all, you can create categories for your various screenshots. When Microsoft shared details on Recall for its initial wave of Copilot+ PCs earlier this year, it created a wave of concern from privacy-minded consumers. Some were hesitant to buy a PC that, from the get-go, would start screenshotting everything they did on their computerâ€"including passwords and financial informationâ€"every few seconds and then storing that on their drive. Microsoft pulled the feature off the initial release of Copilot+ after researchers found the screenshots were easy to access without any real security layer. Both Google and Microsoft claim that all the AI screenshot analysis happens on-device, whether Recall or Pixel Screenshots. Google’s benefit is that users can choose what they screenshot and when. It also works with other Google apps. If it picks up that there’s a date or location in a screenshot, I can hit a button to add a location to Maps or an event to Calendar. Recall is supposed to be the always-on remembrance machine for when you forget what you were searching for the other day. But that feature automatically eats up a not-insignificant chunk of your storage to fill it with junk screenshots. Microsoft promises the feature won’t be enabled by default once they finally bring it to Copilot+, but I’d still be hesitant to allow it out of fear it will pick up some sensitive information while I scroll. Pixel phones will still save your screenshots into Google Photos, at least for now. My photos are for experiences, while my screenshots are useful as a tech reporter, so I’d like to keep them separate if possible. We’d also need to use the app more ourselves to offer a full impression, but as it stands, it's one of those features that makes far better use of AI than dull AI-generated text or awkward, empty AI art.

Google's new Pixel Screenshots takes an active (and safe) approach to organizing your digital life

During the recent Made by Google event, we got our first official look at many of the features coming to the Pixel 9 series, from Satellite SOS to Add Me. One of the standouts, though, was Pixel Screenshots, an app that aims to "help you save, organize, and recall information" collected on the smartphone. To explain how it works, let's say you're looking to update your wardrobe and find a nice pair of sneakers online. You take a screenshot of them and the app analyzes the image and places it into a digital library. From there, you can add a note to it for future reminders or save whatever screenshot you just took into a collection. These groups appear on the homepage of the Screenshots app for easy access. The app even houses its own mini search engine to help you navigate the library. Gemini Nano has been integrated into the software, allowing Screenshots to understand "complex queries." In the live demo, Google offers the example of a user typing "T-Shirt price" to locate an image of a shirt with the price tag in view. What's great is that the app also saves the URL inside an entry as below the main picture is a Chrome link to the source website. You no longer have to keep the tab open forever. Look down further, and it appears Gemini creates a short overview list of a product. If it's a shirt, the accompanying text tells you the item's name, its brand, where it was printed, and, interestingly enough, a brief description. It also appears users can share these entries with others and set up alerts for later viewing. If any of this sounds familiar, that's because Pixel Screenshots functions similarly to the highly controversial Microsoft Recall. Think about it: you're essentially feeding an AI a constant stream of screenshots displaying your internet history. That may sound scary on the surface, although the two have a major difference. Google's tech doesn't operate or record automatically. You have to give the app screenshots manually. It's a straightforward change that goes a long way in ensuring user privacy. There is no need to fear potential keylogging, which was a concern with Microsoft's tech. Pixel Screenshots is shaping itself up to be the next evolution for note-taking apps. It offers an incredibly convenient way to organize chaotic internet browsing. Hopefully, it means that image folders no longer have to be a total mess, thanks to Gemini doing all the work. There is still a lot we don't know about the software, like what kind of security measures are on board. The Tensor G4 chipset probably has data protection onboard, but how long do snapshots stay saved? Indefinitely? Or are old pictures deleted after a while? We would also love to know if there are plans to roll out Pixel Screenshots to older models. At the time of this writing, it's exclusive to the Pixel 9 series. Be sure to check out TechRadar's list of the top seven things we saw and learned at Made by Google.