Microsoft Enhances Windows 11 Copilot with AI-Powered Semantic File Search

Windows 11 test brings AI file search to the Copilot app

Microsoft is testing out a Windows 11 update that lets you use AI to search through your files and images directly from the Copilot app. This test is currently rolling out to Windows Insiders on Copilot Plus PCs and allows you to use more descriptive language when conducting your search. You can currently use the Copilot app to search for a file based on its name, when it was last opened, or file type. This update takes things a step further by allowing you to describe what you're looking for, such as "find the file with the chicken tostada recipe." Microsoft launched AI-powered Windows search on Copilot Plus PCs earlier this year. Microsoft is experimenting with a new Copilot home experience as well, which surfaces your recent apps, files, and conversations. If you select an app within the "get guided help" section, Copilot will automatically start a Vision session, where it scans everything on your screen and guides you through an app. You can also upload a recent photo to Copilot by selecting it from the left side of the screen and then asking the AI assistant for more information about it. These features are rolling out across all Insider channels via the Microsoft Store.

You can search for files in Windows using Copilot now - here's how

Microsoft continues to expand the power and reach of its Copilot AI to provide greater assistance, especially to Windows users. A new skill now rolling out will let you use the Windows Copilot app to search for your photos, documents, and other files using natural language. Also: Excel's new Copilot function turns your prompts into formulas - how to try it In a blog post published Wednesday, Microsoft announced the new "semantic file search" and described how it works. In short, you're able to take advantage of the enhanced search capabilities directly through the Copilot app instead of relying on the controversial Recall or other AI-based features. By using natural language, you can describe the type of file you need without having to remember its name or any of the specific words in the file. Sounds cool. But before you dive in, know that you must meet a couple of requirements. First, you'll need a Copilot+ PC, which Microsoft has been using as the platform to test its AI initiatives. Second, you'll have to be part of the Windows insider program, as the new feature is available only on the latest Windows 11 insider builds. If you qualify, here's how to try this. Fire up the Copilot app in Windows. At the prompt, type a description of the files you seek. Here are a few examples served up by Microsoft. Also: I use Edge as my default browser - but its new AI mode is unreliable and annoying If you're looking for a specific file, such as your resume, simply type "Find my resume." If you want a file with specific content but don't know the name, type something like "find the file with the chicken tostada recipe." And if you're searching for a group of photos or other images, type something like "find images of bridges at sunset." In response, Copilot should display links to the resulting files, which you can then select to open. Concerned about Copilot peeking at your personal files? Microsoft says that the feature works through the Recents folder, which shows all the recent documents and files you've opened. Rather than scan your entire PC, the AI checks only these recent files and only those stored locally. Also: Microsoft gives Copilot a face - here's how to try your new Appearance chat buddy Further, Copilot can find only certain types of files, specifically image files (.png, .jpeg, and .svg), Adobe PDF files (.pdf), Microsoft Word documents (.docx), Excel spreadsheets (.xlsx and .csv), and text files (.txt and .json). If you attach a file and submit it to Copilot as part of your prompt, the AI processes it but won't share it. You can also tweak the permissions to tell Copilot what it can access, retrieve, and read. To do that, head to the Copilot Settings under Permission settings. If you select a recent application via the "Get guided help with your apps" section in the new Copilot Windows app, a Copilot Vision session kicks off to answer your questions about what it sees on the screen. Also: Microsoft Copilot 3D turns your 2D images into 3D models for free - how to try Further, you can automatically send a file to the Copilot app for analysis. For this, select the file of your choice in the left pane in the app to upload it to Copilot. From there, the AI is able to summarize the file, identify items in a photo or other image, or simply chat with you about it. If you meet all the requirements but don't yet see the new search options, be patient. Microsoft says that the features are still rolling out, so not all insiders will get them right off the bat. To check your version of Copilot, open the app and select About. If the new AI file search has reached your PC, the version of Copilot should read 1.25082.132.0 or higher.

Microsoft Is Testing Semantic Search for Copilot App on Windows 11

Don't miss out on our latest stories. Add PCMag as a preferred source on Google. Microsoft is bringing semantic search to the Copilot app on Windows 11. With this update, Insiders will be able to search for files and images using natural language. In other words, you can just describe what you're looking for instead of remembering the exact file name or words mentioned in the file. Sharing examples, Microsoft says you can use queries like "find images of bridges at sunset on my PC," "find my CV," or "find the file with the chicken tostada recipe" to get to your desired items. To test the feature, Insiders need to have a Copilot+ PC and download the latest version of the Copilot app (version 1.25082.132.0 and higher) from the Microsoft Store. It currently works in English, Chinese (Simplified), French, German, Japanese, and Spanish. Semantic search was first added to Windows Search, File Explorer, and the Settings app on Copilot+ PCs earlier this year. At the time, Microsoft said it didn't require an internet connection and relied on a small language AI model that runs locally on your device's Neural Processing Unit (NPU). It also said that none of the data gathered would be "sent to any other computer or to Microsoft." It's unclear if all of these conditions apply as the feature comes to the Copilot app as well. We've reached out to Microsoft for clarification. However, to confirm what the Copilot app can access, retrieve, or read, you can go to Copilot Settings under Permission settings. The Copilot app update also adds two other features. A revamped homepage now displays your recent apps, files, and conversations. For now, it fetches PNG, JPEG, SVG, PDF, DOCX, XLSX, CSS, JSON, and TXT files from your standard Windows "Recent" folder. It's also getting a Vision update. According to The Verge, if you pick an app from the "Get guided help" section of the homepage, a Vision session will scan everything on your screen and guide you through the basics of the app. All of these features are packed into the latest version of the Copilot app for Copilot+ PCs, but they are "gradually rolling out, so not all Insiders will get the update right away," Microsoft adds.

Not a fan of the Copilot app in Windows 11? You might change your mind, as Microsoft just beefed up its powers

This introduces a powerful new natural language-based search Windows 11's Copilot app is getting a deeper and more powerful search functionality, and more besides - although this is only in testing for now. The Verge reports that Microsoft is rolling out an update for the Copilot app to Windows Insiders (those running test versions of Windows 11) who have Copilot+ PCs. Those devices have the necessary NPU to drive this new AI search, which allows you to use natural language in queries, and will dive into files to find results for you. So, as Microsoft points out, you can ask a conversational query such as 'find the file with the tiramisu recipe' and the Copilot app will hunt through the files on your PC to locate it. Or similarly, you could ask 'find pictures of my dog on the beach' and the AI will pick out those images specifically. Microsoft calls this functionality a 'semantic file search'. Microsoft is also bringing in a rejigged home page for the Copilot app which surfaces recently used files, apps, and conversations, a move designed to make it easy for you to jump back into whatever you were doing previously. You can also click on those recently-used files to query them with Copilot, or elect to get help with apps via a specific 'guided help' pane - this fires up a Copilot Vision session to guide you through said app. Testers who want to give the new Copilot app a whirl should grab the latest update from the Microsoft Store in Windows 11. Your immediate concern here may be privacy, and Microsoft has been quick to allay any fears along those lines in its blog post introducing these changes. The company makes it clear that Copilot surfaces recently-used files simply via the standard 'recent' folder that Windows 11 maintains - so the AI app isn't digging into your system any more deeply than the records which the OS keeps itself. Microsoft clarifies that: "Copilot doesn't scan your entire system or upload anything automatically." However, when you're directly querying a file with Copilot, in that case it is uploaded for processing, but "nothing is shared unless you explicitly do so". As for the semantic file-search capability, I assume that it functions similarly to the AI-supercharging of Windows 11's search itself on Copilot+ PCs (when Microsoft introduced natural language queries via the search box on the taskbar). Microsoft seemingly wants to beef up search with AI across the board, and so on Copilot+ PCs we have that powered-up Windows 11 search, as well as Recall (a screenshot-based AI search leveraging the activity on your PC), and now an improved natural language search within the Copilot app itself. What's the end game with pushing AI in search so strongly? Well, search is an obvious use case for AI, and I'd speculate that eventually, Copilot will take over all Windows search duties entirely. There will be no basic Windows search at all, in other words - if you want to find stuff on your PC, you'll ask Copilot, end of story. And hopefully it'll complete the search without showing you 15 related 'suggestions' or 'recommendations' about what else you might need to complete whatever task the AI thinks you're carrying out.

Copilot+ PCs Will Now Let You Search for a File By Describing It

The feature is currently only available in select languages Microsoft has started testing the semantic file search feature in Windows 11 with a larger user base. On Wednesday, the Redmond-based tech giant announced that two new artificial intelligence (AI) features are now available to all Insiders using Copilot+ PCs. The feature, powered by Copilot, allows users to search for certain files and images using natural language queries. Additionally, the company is also introducing a new Copilot home experience, where they can see the recently closed apps and files within the chatbot's interface. Semantic File Search Comes to Copilot+ PCs In a blog post, the tech giant announced the rollout of the new Copilot features. These features are being rolled out across all Insider Channels with the Copilot app update (version 1.25082.132.0 and higher) via the Microsoft Store. Notably, the semantic file search feature was first previewed in the Dev channel in January. Semantic file search in Copilot+ PCs Photo Credit: Microsoft Windows 11 typically supports keyword-based indexing, where typing the name of the file or keywords surfaces them. The new feature, however, uses Copilot's capability to offer semantic indexing, or description-based searching. This means users will be able to vaguely describe the file they're looking for, and the AI will still be able to surface it by understanding the context behind the query. Highlighting an example, Microsoft said, users can ask the AI to "find images of bridges at sunset on my PC" or "find the file with the chicken tostada recipe". Copilot can only surface files that are locally stored on the device. Additionally, users will be able to adjust permissions for what the chatbot can access, retrieve, or read via the Copilot Settings under Permission settings. The second feature, dubbed Copilot home experience, brings a user's recently closed apps, files, and conversations into the Copilot app's interface. Additionally, when a user clicks on a recently closed app in the "get guided help with your apps" section inside the Copilot app, the AI will automatically start a Vision session to assist with the issue. The home experience also allows users to quickly upload a file or photo to the chat window and ask queries about it.