Microsoft Copilot Unveils Major Upgrades: Voice Interactions, News Summaries, and More

I got my news briefing from Microsoft Copilot this morning, and I'd do it again

In its latest Copilot overhaul, Microsoft gave its AI assistant a much-needed facelift, including a cleaner user interface and exciting new features that help it better compete against popular alternatives like ChatGPT. My newfound favorite feature is Copilot Daily. Also: You can test Microsoft's experimental AI features in Copilot Labs now - but there's a catch Every morning, I catch up with the latest news as part of my daily routine. This often means checking the homepages of my favorite news outlets and reading up on the ones that most interest me. However, figuring out which sites and stories to read in a saturated media world can be challenging -- Copilot Daily made it an easier experience this morning. The feature, unveiled on Tuesday, summarizes the news and weather for you every morning and reads it to you in your favorite Copilot Voice, which sounds remarkably human. You can listen to a segment of my daily briefing this morning below. The stories that Copilot Voice reads are from Microsoft's partners, which include reliable news sources such as Reuters, Axel Springer, Hearst Magazines, USA TODAY Network, and Financial Times. Like most news wire services, Copilot Daily's content will be limited to news from its partners, which may be an inconvenience if you want perspectives outside the scope of the partnerships. Microsoft did add that more organizations will be coming in the future. Also: Gemini Live is finally available for all Android phones - how to access it for free The briefing is concise, about five minutes long, and features five stories, including some of the biggest headlines. For example, some of today's stories included Jimmy Carter's 100th birthday, the US Vice Presidential debate, and the World Health Organization's recommendations on protecting infants against RSV. If you want to learn more about a story, you can click on the tag that says the media outlet's name and then click on the story to open it up on the site and read it. You can also use the fast-forward icon at the bottom of the briefing to skip to the next story segment, which you can't typically do with a news broadcast. Even though five stories in five minutes wasn't comprehensive, it was a good way to catch up on topline news, and I expect this feature to get more helpful over time. Also: Copilot Voice beats Gemini Live and ChatGPT's Voice Mode in one big way Microsoft shared that it is working on ways for users to personalize the briefings and select from more personalization controls. I can also see the segments getting longer, growing into a more comprehensive news show as more partners join and the experience is developed further. Accessing Copilot Daily is entirely free; you only need a Microsoft Account. The feature is rolling out to users, and I was able to access it from the Copilot iOS app.

All you need to know about Microsoft's newest Copilot upgrades

Microsoft is rolling out Copilot Labs for some users to test beta features and provide feedback. Microsoft launched a slew of new AI upgrades to its assistant Copilot with features that read aloud a summary of the daily news, a new "intuitive" way to brainstorm and ask Copilot questions and beta 'skills' that 'think deeply' and analyse your screen, answering your questions. The announcements were made yesterday (1 October) at Microsoft's New York City event and brought about a new look across mobile, web and Windows versions of Copilots. Previously known as Bing Chat, Copilot is built into the sidebar of Microsoft's search engine Edge, as well as on Windows 10 and Windows 11. Since January this year, Copilot is also available as an app on Android, iOS and iPadOS. In a blog post yesterday, Microsoft AI's CEO Mustafa Suleyman, who has been in this role for half a year, said that his focus is on what users experience. "Great technology experiences are about how you feel, not what's under the hood," he wrote. He said Copilot will evolve to suit the mannerisms of its user. "Copilot will be there for you, in your corner, by your side and always strongly aligned with your interests. "Over time it'll adapt to your mannerisms and develop capabilities built around your preferences and needs. We are not creating a static tool so much as establishing a dynamic, emergent and evolving interaction." Here's a rundown of the new Copilot upgrades. Copilot talks 'naturally' Copilot Voice allows you to speak to, interact and brainstorm with Copilot. Calling this "intuitive" and "natural", Microsoft claims you can even vent to Copilot at the end of a tough day. As of now, the assistant is only available in English in the US, Canada, the UK, Australia and New Zealand. Copilot Daily summarises news from authorised sources that Microsoft has partnered with. Currently, it partners with Reuters, Axel Springer, Hearst Magazines, USA TODAY Network and Financial Times, and the company plans to expand on sources over time. "It's an antidote for that familiar feeling of information overload," Microsoft said. This feature is currently only available in the US and UK. Microsoft Edge users can now access Copilot directly from the browser by typing '@copilot' into the address bar. However, some newer PCs even have a dedicated keyboard key for Copilot. The feature is built on to Microsoft edge with the aim of helping answer questions, summarise page content, translate text or rewrite a sentence. Along with these upgrades, Copilot Labs gives users an opportunity to test beta features. Users can provide feedback and shape the end product that Microsoft creates. New features in beta At yesterday's event, Copilot Labs launched with two features called Copilot Vision and Think Deeper. Copilot Vision analyses what you're viewing on the screen and answers questions about it. It can also suggest next steps and help you "without disrupting your workflow", claims Microsoft. As a safety mechanism, Microsoft has made Copilot Vision sessions optional, adding that any data is deleted after the session. Microsoft also says that it has reduced Vision to only work on some safe and popular websites and won't work on paywalled pages. It is currently only available to a limited number of Copilot Pro subscribers in the US. While Think Deeper tackles challenging questions that require deeper analysing and more time. Microsoft says that users can ask Think Deeper to answer questions that need weighing between two options. And similarly, the skill is still under development and is only available to a limited number of Copilot Pro users in Australia, Canada, New Zealand, the UK and the US. In addition, Microsoft also rolled out Bing generative search in the US, optimising search results and changing how they appear as a result. Don't miss out on the knowledge you need to succeed. Sign up for the Daily Brief, Silicon Republic's digest of need-to-know sci-tech news.

New Microsoft Copilot features include natural AI voice interactions and daily news summary

Microsoft is revamping its Copilot personal AI assistant with a new design and features including a daily news summary, natural voice interactions, and the ability for the AI to act as a companion when browsing the web. The release points to its larger vision for consumer artificial intelligence services under Mustafa Suleyman, the DeepMind co-founder and former Google vice president who was named CEO of Microsoft AI earlier this year. "We are not creating a static tool so much as establishing a dynamic, emergent, and evolving interaction. It will provide you with unwavering support to help you show up the way you really want in your everyday life, a new means of facilitating human connections and accomplishments alike," Suleyman wrote Tuesday morning. Microsoft faces the difficult challenge of standing out in a field where core AI features are becoming table stakes, competing against the likes of Anthropic's Claude, Google Gemini, and ChatGPT, from its partner, OpenAI. But as with its separate Microsoft 365 Copilot service for businesses, the company has the advantage of leveraging its existing platforms to fuel Copilot adoption for personal use. Copilot is accessible from the Windows taskbar, as well as dedicated hardware keys on new Windows laptops and keyboards, and in the sidebar of Microsoft's Edge browser. Microsoft is rolling out the revamped Copilot personal interface and new features starting today in its apps for iOS, Android, Windows, and the Copilot web portal. Two new features stand out: In addition, the company says it's launching Copilot Labs, which will give users access to experimental features before they're widely released. Two features were announced under this umbrella: Microsoft is making Copilot Labs available to users of its $20/month Copilot Pro subscription service. The company also made a series of announcements related to new Copilot+ PCs, including improved Windows search capabilities, the ability to upscale image resolution in the Photos app, and a preview of a feature called "Click to Do," which puts an overlay on the box to suggest quick actions that the user can take. The flagship feature of Copilot+ PCs, Recall, was delayed due to privacy and security concerns, which the company sought to address in an announcement last week. Recall is now slated to roll out to Windows Insiders on Snapdragon-powered Copilot+ PCs in October, and on new Intel- and AMD-powered Copilot+ PCs in November.

A huge wave of new Copilot features has arrived, and you can use them right now

Copilot for the Web gets a sudden upgrade, including a new feature and voices Key Takeaways Microsoft unveils new Copilot features like Copilot Voice for natural conversations. Copilot Daily provides news and weather summaries in chosen voice, with personalized options. Personalized Discover uses user data to suggest activities, while Copilot Labs offers experimental features. We've heard rumors and rumbles about Microsoft revamping Copilot, but we didn't know all of the details. Fortunately, Microsoft has given us all the information on what it's planning with its AI assistant. Turns out, the Redmond giant kept a ton of features hidden under its sleeves, and now it's ready to show off all of the new features you can try right now. Related Microsoft Copilot: What is it, and how does it work? Is Microsoft Copilot the best AI chatbot available right now? Microsoft reveals a ton of new Copilot features, available to use today Image Credit: Microsoft In the press release, Microsoft reveals what we've suspected for the past few days now. It's planning to make Copilot less of a teacher, and more like a personal assistant: For Copilot, this means taking the first steps towards creating an AI companion, one that's always by your side and in your corner, helping you feel smarter and more supported through natural conversations. We're updating the core experience and adding a host of new features as we take the first steps on this journey. So, what does Microsoft have in store for this new "AI companion" plan? Let's dig in. A full list of all the new Copilot features is now available Image Credit: Microsoft First of all, Microsoft is adding Copilot Voice. This feature gives the AI companion a choice of four different voices to choose from, so it feels less like you're talking to a robot and more like an actual person. It's likely Microsoft's way of pushing the idea of having people perform "natural conversations" with the bot. It's available right now in Australia, Canada, New Zealand, the United Kingdom, and the United States, in English only. Next up is a feature we suspected Microsoft was taking with its AI assistant: Copilot Daily. There's a trend emerging within the AI space where companies are making their AI bots more like glorified news readers than passive assistants, and it seems Microsoft wants to take part with Copilot Daily: Copilot Daily helps you kick off your morning with a summary of news and weather, all read in your favorite Copilot voice, with more options like reminders coming soon. It's an antidote to that familiar feeling of information overload. Clean, simple, and easy to digest. Copilot Daily will only pull from authorized content sources. We are working with partners such as Reuters, Axel Springer and Hearst Magazine and plan to add more sources over time. We'll build additional personalization and controls in Copilot Daily over time. This feels like assistants on your phone, which accommodate your alarm with a weather forecast and a reminder breakdown, with the addition of using AI's ability to quickly summarize data to create a news feed for users. It'll be interesting to see how good that news is, given that Microsoft implies that it's pulling from a select whitelist of sources. This feature begins rolling out in the US and the UK starting today, with other countries coming later. Next up, we have Personalized Discover. This seems to be a way to get started with Copilot when you're unsure as to what you can do with it. Microsoft says that "with your permission," it can look at what you've done on other Microsoft services to get an idea of the things you want to achieve. Copilot in Microsoft Edge is getting a slight feature tweak. The company says that it'll act pretty much the same as it always has, except with this update, you can invoke it by typing @copilot into the address bar. Copilot Labs is a new Copilot Pro perk that allows people to check out some experimental features for the AI assistant. Next week, Microsoft hopes to get two features up and ready in the labs: Copilot Vision, which can analyze your desktop or your photos to help you come to a decision, and Think Deeper, which claims to help Copilot perform complex tasks like math problems. Copilot Vision will come "in the future," while Think Deeper should be available from today.

Microsoft Copilot can talk now: 3 new features that make it more like ChatGPT and Google Gemini

Say hello to Copilot Voice Credit: Mashable Composite: Microsoft / Canva Copilot, Microsoft's AI companion, got some big updates today, including advanced voice and vision capabilities. These new enhancements bring it more in line with other AI chatbots like Google Gemini and OpenAI's ChatGPT (though it's worth noting that the latter underpins Copilot). Some may even argue that the new-and-improved Copilot now outshines the aforementioned AI tools -- but that's up to you to decide. Curious about the new utilities that Microsoft introduced for Copilot? Stick around. Similar to the recently announced Gemini Live update and ChatGPT Advanced Voice Mode, Microsoft has debuted Copilot Voice, which allows users to have smooth, engaging verbal conversations with Copilot. Just like Gemini Live and ChatGPT Advanced Voice Mode, you'll have the ability to interrupt Copilot Voice. Plus, there will be four voices you can choose from. In an early demo Microsoft shared with Mashable, Copilot Voice spoke with filler words and disfluencies like "ums" and "likes." The voice had a slightly metallic tone, but it's too early to determine if this will carry over once it rolls out to customers. As you browse the web, Microsoft revealed that the upgraded Copilot can now "see what you see and hear what you hear." While most of the popular AI chatbots can handle text and images, Copilot Vision's edge is that it can interact with you while observing your browsing activity. In other words, Copilot Vision can process text and images on your screen in real time, ready to jump in with answers to your questions, suggest next moves, and keep things running smoothly without interrupting your workflow. One use case for Copilot Vision would be scrolling through Rotten Tomatoes and scratching your head on which film you should watch. With Copilot's help (again, it can see what you're doing), you can ask it which movie you should indulge in. Tapping into Copilot Voice, the AI chatbot can respond to you verbally by saying, for example, "Oh wow, there are lots of colorful posters and different genres on the Rotten Tomatoes homepage. Anything catching your eye?" After telling it that you have a desire to watch a comedy, Copilot Voice can chime in with funny movies that it has spotted on the page. You can even ask it for reviews. And don't worry -- Microsoft says it's serious about privacy and security. Your sessions are opt-in only, and once they're done, your data is wiped clean. No training, no tracking -- just real-time help. With the new Think Deeper feature, Copilot can now tackle more complicated questions and deliver detailed, step-by-step answers. However, it takes longer to respond because it's carefully weighing all the options before replying. Think Deeper is ideal for those tricky, everyday decisions -- like whether you should move to a new city or which car fits your lifestyle best. It's still in the experimental phase over at Copilot Labs, so Microsoft is counting on user feedback to help refine and improve this new skill. In addition to these three features, Microsoft is introducing Copilot Daily, a new perk that lets users get a daily digest -- a summary of news and weather -- all read in their favorite Copilot Voice. Microsoft said that it will only pull from authorized sources, including content from Hearst Magazine, Reuters, Axel Springer, and Financial Times. The Redmond-based tech giant says it plans to add more over time. Finally, Microsoft is making it easier for users to access Copilot in Microsoft Edge -- just type "@copilot" into the address bar. The updated Copilot is coming today to iOS and Android users, as well as the Copilot web experience at copilot.microsoft.com and the Copilot Windows app.

Microsoft's Copilot AI gets a voice and the ability to see websites you browse

The company is debuting a slew of new AI features, including experimental Copilot Labs offerings. Beyond debuting new features for Copilot+ AI PCs and Windows 11's 2024 update, Microsoft is also giving its Copilot AI a makeover on the web, mobile and desktop. That includes a slightly friendlier interface wherever you access it, along with new capabilities like Copilot Voice, which allows you to talk conversationally with the AI assistant. Ultimately, Microsoft is aiming for Copilot to be seen as more than just a party trick for generative AI search and image creation -- it's trying to make it a core part of your daily workflow. That starts with a cleaner and simpler UI that makes Copilot look different than a boring old search engine. That new look hits Copilot on the web, iOS, Android and Windows today. You'll also be able to access Copilot from within Whatsapp, which could be useful if you want to avoid Meta's AI assistant. Microsoft also plans to release more experimental AI features via Copilot Labs, similar to the way Google has given users access to early features with its own Labs. There's Copilot Vision, which lets the AI see everything you're viewing in Edge, and Think Deeper, a method for letting Copilot work harder on more complex dilemmas. Both features also have the potential to create new privacy and security issues, but Microsoft appears to have learned a bit from its messy rollout for Recall on Copilot+ PCs. Notably, these Copilot Labs offerings will only be available to people paying for Copilot Pro subscriptions. Microsoft says Copilot Vision sessions are "opt-in and ephemeral," and nothing is stored for AI training. Any data related to your session is deleted once you're done. The feature also won't work on all websites, only a "limited list of popular websites." And Microsoft notes that Copilot only interprets images and text that you're seeing on a website; it's not doing any major content processing. The company also plans to take feedback from early users, refine safety measures and "keep privacy and responsibility at the center of everything we do." There's no doubt that Microsoft has to re-establish trust with its users after the botched rollout for Recall, a feature that was meant to help you find anything you've done on your computer, but had some gaping security holes early on. "Ultimately, users want to have confidence with anything happening with sensitive data caches," Microsoft's Windows head, Pavan Davuluri, said when we asked him about the lessons the company learned from the Recall fiasco. He added that he believes the company has "gone above and beyond" to create that confidence by establishing new security standards for Recall. There are less controversial features coming, as well: Copilot Daily uses Copilot Voice to delivery a summary of news, weather and potentially reminders about tasks every day. Based on the demos we've seen, it's a lot like an early morning podcast built just for you. Copilot Discover, meanwhile, will give you tips about AI features to use (pictured in the screenshot above), depending on Microsoft services you've used in the past. Personally, I'm still a bit ambivalent about Microsoft's near-instant transformation into an AI company. That's particularly true since we're still dealing with inaccuracies and other issues from OpenAI's ChatGPT, which Microsoft relies on to power Copilot. These Copilot features seem a bit more useful than being a souped-up search engine, but the company still needs to prove it can build AI features with bulletproof security and privacy. As with practically any AI product these days, Microsoft is also staggering the release of these Copilot features. Copilot Voice and Daily launch today, but Voice will only be available in English in the US, Canada, Australia, the UK and New Zealand. Copilot Daily, meanwhile, only works in the US and UK for now. Copilot Vision will head to Labs over the coming days for a "limited number" of Copilot Pro subscribers, while Think Deeper is available for Pro users in the US, Canada, Australia and New Zealand.

Microsoft Just Dropped a Bunch of New Copilot Features

Microsoft is updating its Copilot chatbot to match other AI bots. After an underwhelming launch in June, Microsoft is retooling Copilot and Copilot+ to make it match up to its competitors a bit more. It's a mixed bag of an update though: It's losing some of its initial, unique promises -- there don't seem to be any tools for adjusting your Windows settings included in the new version -- even as it gains multimodal features (like ChatGPT) and AI generative fill (like Photoshop). The Copilot chatbot gets a voice In a series of videos and a blog post, Microsoft revealed that Copilot now comes with a new, more neutral look as well as the ability to chat over voice -- and soon, it will also take input from video. It's similar to what companies like Google are doing with Gemini Live, with a few Microsoft touches. Most obvious are the cards that pop up when you open the new Copilot app or website, which resemble Microsoft's live tiles UI from older versions of Windows and give you suggestions for what to do with Copilot based on your past conversations with the chatbot. More interesting is the Copilot Daily feature, which presents a brief AI-generated summary of recent news headlines. These are taken from approved partners like Reuters or Financial Times, to prevent copyright issues, and are presented in a sort of TV news anchor voice. It could be helpful to quickly get caught up when waking up, although I'd caution against using a freshly developed robot as your only news source. The new look and voice support are available for iOS, Android, and the web today, while Vision, which will let you point your camera at your surroundings in order to use them when asking the AI questions (say, "What breed of dog is this?"), is currently part of the experimental Copilot Labs program. Also in Copilot Labs is "Think Deeper," which will let you give the AI more time to answer your questions, hopefully improving your results. Copilot Labs features will be available to a limited number of Copilot Pro subscribers first -- it's not something you can just sign up for like a Windows Insider account. At this point in the lifecycle of AI, none of these reveals are especially revolutionary, with even Copilot Daily having similar competition in apps like Perplexity. The unique but controversial Recall feature that will take continuous screenshots of your Copilot+ PC is still in testing, although Microsoft says it will come to testers with Qualcomm's Copilot+ chips soon. In the meantime, Copilot+ laptops are getting some additions to Paint and Search to hold them over. Copilot+ Laptops may soon finally be worth it Building on the existing Cocreator in Paint feature, which uses generative AI to make images based on your sketches, Generative Fill and Generative Erase allow Copilot+ laptop owners to adjust their images without having to draw anything first. Fill will simply let you create and place AI imagery from text, while Erase works a bit like Google's Magic Eraser, letting you paint over elements you want to remove from a photo with a brush, then watching as they get erased and replaced with imagery the AI thinks is appropriate. There's also AI upscaling in the Copilot+ version of the Photos app, with which on-device AI will fill in the blanks on low-res photos to increase their resolution as much as eightfold. Microsoft says you can get 4K results "within seconds," although I'll need to see more examples of its output before I can really recommend the tool. Meanwhile, Windows Search on Copilot+ PCs will soon use AI to discern the content of your files, not just their names. That means you could search for "cat" and it could bring up photos of cats, even if "cat" is not in the file name. While Microsoft doesn't say that the feature is in testing, the company seems to think it's not ready yet, saying it will be coming first to File Explorer next month, and to Windows Search and Settings "in the coming months." Perhaps this extra time is to address any privacy concerns that could come from Microsoft snooping through your files, as happened with Recall, although Microsoft does say the feature will work offline. One feature that Microsoft does say is in testing is "Click to do." Click to Do seems sort of like an AI-powered right click, with Copilot analyzing what you're currently doing on your PC and suggesting appropriate actions in a context menu. It's a bit more involved, since you'll need to access the Snipping Tool or Print Screen to do it, but it's the one new feature that hearkens back to Copilot's initial promises of, well, helping you pilot through Windows. Click to Do will be available to Windows Insiders on Copilot+ laptops next month. Overall, today's changes and announcements are hefty, in the sense that there's a lot to dig into, but it might be a while until they actually impact your workflows, as some are in testing and you'll need a Copilot+ laptop to access most of them. Whether that'll make Copilot+ worth the investment remains to be seen, but in a best-case scenario, features like Generative Fill could potentially save you the cost of a Photoshop subscription.

Microsoft Copilot now reads, thinks, and speaks

In a new update to their Copilot AI collection, Microsoft is launching a range of improvements expected to make interacting with devices easier. Among them, Copilot Vision, Think Deeper, Copilot Voice are further designed to offer user-focused, timely and context-aware experience on Windows, iOS, Android, and Web. All these features are in one way or the other belonging to the broader approach that Microsoft is taking to have Copilot in the middle of people's daily activities alloting AI solutions merged with the protection of the users' privacy. Among new features, there is Copilot Vision, which lets users ask all sorts of questions regarding the content on their screen - be it the text or images on a web page. For example, it is possible to ask for a particular recipe that is depicted in a photo or for furniture advice while using the site. Copilot Vision then operates in manners where it scans through the contents that the users are consuming and responds to the user in real-time, based on the content. Accessible via Microsoft Edge by typing "@copilot" in the address bar, Copilot Vision is available as part of Copilot Labs, an exclusive opt-in program that offers experimental features to a select group of users. While this feature is not entirely groundbreaking -- Google's Android platform offers similar capabilities -- Microsoft claims that Copilot Vision takes things a step further by ensuring that no data is stored or used to train AI models. Microsoft's complicated Copilots explained in detail Microsoft has been keen to emphasize its commitment to privacy with this tool. In a recent blog post, the company highlighted that Copilot Vision deletes all processed data immediately after a conversation, with no audio, images, or text being stored. This is likely in response to growing concerns about data misuse in AI systems, a point Microsoft has been careful to address following legal scrutiny over past incidents involving its AI products. However, Copilot Vision comes with certain limitations. Currently, the tool cannot analyze content from paywalled or sensitive websites, restricting its functionality to a pre-approved list of web properties. This restriction seems designed to avoid legal challenges similar to the one Microsoft faces with The New York Times, which accused the company of allowing its AI tools to bypass paywalls and access premium content. Another significant addition to the Copilot suite is Think Deeper, a feature that leverages advanced reasoning models to help users navigate complex tasks. Whether it's solving intricate math problems or comparing multiple home renovation options, Think Deeper is designed to break down and analyze complicated scenarios. Microsoft has described Think Deeper as a tool that provides step-by-step guidance, making it useful for both everyday challenges and more specialized problems. The company noted that the feature is powered by "reasoning models" that take additional time to respond, ensuring more thorough and accurate answers. Though Microsoft has remained tight-lipped about the specific models used, it is likely a customized version of OpenAI's o1 model, which Microsoft has fine-tuned for this purpose. Currently, Think Deeper is available only to a select group of Copilot Labs users in countries like the U.S., U.K., Canada, Australia, and New Zealand, with plans for broader availability in the future. The last in the line of Microsoft's AI features is Copilot Voice, the ability to address Copilot and issue commands. Launched in the USA, UK, Canada, Australia, and New Zealand first, Copilot Voice extends the ethos of Copilot with answers verbalized. Like OpenAI's voice in ChatGPT, Copilot Voice is capable of sensing voice tone, and respond accordingly. Some features of the application make it even more engaging: for example, users can pause Copilot during the conversation or interrupt its answers in the middle of the sentence. The feature of the application is the four synthetic voice which are very natural and realistic to the touch. However, there are limits on how much users can engage with Copilot Voice. Microsoft has implemented a time-based usage limit, with Copilot Pro subscribers gaining access to additional minutes depending on overall demand. In addition to the major feature updates, Microsoft is also introducing personalization options to Copilot. These settings allow the AI to tailor its suggestions based on the user's past interactions, drawing insights from activity across other Microsoft apps and services. According to Microsoft, this will help Copilot offer more relevant recommendations and faster access to useful features. However, the rollout of personalization is being delayed in certain regions. For now this feature is not going to be available in EU or UK anymore as Microsoft continues to weigh options on how to adapt to the tight data protection laws in these regions. It follows the strategy that leading IT enterprises like Apple and Meta as well as other firms have executed when rolling out AI solutions in the European Union. Microsoft has expressed that the firm is looking forward to providing personalization incorporated into the users in these areas and at the same time satisfying the legal provisions such as the Digital Services Act, which regulates the use of AI and data protection in EU. In the past, the company has received certain regulatory issues, particularly in the assortment EEA's recent admonition to Microsoft that it might be penalized for not giving enough details about its generative AI goods.Privacy and responsibility at the core Throughout the rollout of these new features, Microsoft has maintained that privacy and user feedback are at the heart of its development process. The company has stressed that none of the data processed by Copilot Vision, Think Deeper, or Copilot Voice is used for AI training or stored beyond the immediate session.

Microsoft Copilot just got a big update -- here's all the new AI features

Microsoft has begun rolling out updates to Copilot, introducing several new features and enhancements to the company's Windows 11 AI platform. This includes advanced capabilities like Voice and Vision for more natural interactions, improved Windows search, detailed answers to questions, and more. Copilot+ PCs are also receiving various updates this fall. Copilot has been a major talking point for Microsoft for more than a year. But as competing machine learning programs like ChatGPT become more sophisticated, Microsoft's AI was in danger of falling behind. Though we'll have to go hands-on with the updated Copilot to see how it stacks up to its previous iteration and its competitors, the upcoming changes could give it a nice boost. Here are all the big changes coming to Microsoft Copilot and Copilot+ PCs. Whether it's on your phone or PC, the redesigned Copilot app should be more useful than ever. Here are some of the most notable features. The new Copilot Voice lets you have smoother and more natural conversations with the AI companion. This way, it will feel more like speaking with a person than a robot. Responses are faster and you'll even be able to interrupt the chatbot and redirect the conversation. This will have four voice options to choose from. Expect Copilot Voice to roll out in the US and UK first before hitting other countries. Copilot Daily exists to help you kick off your day with a summary of news and weather. This information comes from sources like Reuters, Financial Times and Hearst Magazine, among others. Expect additional options like reminders and other personalization and controls to arrive on Copilot Daily over time. Your chosen Copilot voice will deliver these daily summaries. Like Copilot Voice, this feature is only available in the US and UK but will come to other countries in the future. Think Deeper is for those who need Copilot to help with complex matters such as deciding which city to move to or the best type of car to suit your needs. It can also help solve complicated math problems or write more advanced code. This feature is still in its infancy and undergoing development, so Microsoft has placed it in its experimental Copilot Labs to test and get feedback from folks. Copilot Vision is another feature under development that could prove useful for desktop users. Built natively into the Microsoft Edge browser, Copilot Vision sees whatever you're looking at on your computer and can talk to you about it in real time. It understands the web page you're viewing (both text and images) and can answer questions about the content, suggest next steps, and so on. For example, if you're on Rotten Tomatoes and need a recommendation for what to watch, Copilot Vision sees what's on the screen and makes suggestions based on your preferences. Copilot+ PCs will receive updates this fall that will help streamline tasks and provide a more personalized experience thanks to AI. This not only includes existing Copilot+ PCs packing Snapdragon X Plus and Snapdragon X Elite, but also upcoming laptops featuring Intel Lunar Lake and AMD Ryzen AI 300 processors. Here are some of the new features and updates. Recall returns after the company recalled the feature back in June. The biggest change is that Recall is now an opt-in feature and isn't enabled by default. You'll have to manually enable it when setting up a new Copilot+ PC for the first time. If you don't, Windows Recall will be turned off and no snapshots will be taken or saved. Likewise, you also have the option to remove it entirely using the optional features settings in Windows 11. Click to Do understands everything it sees on the screen and then enables useful shortcuts to actions to help you quickly search, learn, edit, shop, or act on items. It works on any image, document or video. For example, if you're watching a furnishing video and want information about the decor, you can bring up Click to Do, click on an item in the video, and get actions like share, save, or do a visual search with Bing. Finding files on your computer will be easier with descriptive search. Thanks to this feature, you won't have to remember specific file names or even worry about spelling. You can type "barbeque pictures" in Windows search and receive all related photos, documents, and so on. Given how finicky Windows Search can be, this should be a super useful feature. Speaking of photos, it'll be easier to retouch and reimagine your photos thanks to AI. For instance, you'll be able to enhance low-resolution photos up to 8X the original resolution. Generative Fill and Erase in Paint lets you add or remove elements from your photos. If you have a photo of a beach, you can add a lighthouse by using a text prompt. New Copilot features like Vision, Voice and Think Deeper should make Windows Copilot more useful and personalized. Again, we'll have to pit it against existing AI programs like ChatGPT to see how it stacks up. But based on Microsoft's promises, Copilot's new features and enhancements could give it a second wind.

Copilot's AI will be able to 'see' and talk to you, Microsoft says

These new Copilot services will use the cloud, not your PC, for processing. Some will require a Copilot Plus subscription. Microsoft is beginning to roll out its next feature update of Windows 11, the Windows 11 2024 Update, beginning today. But Microsoft obviously isn't done yet, and it's offering a sneak peek at new Copilot experiences which will debut this fall, including Copilot Voice, Copilot Vision, and Copilot Daily, among others. On the surface, the new additions to Copilot sound similar to multimodal ChatGPT (or GPT-4o) that OpenAI launched earlier this year, where ChatGPT can now "see" and an Advanced Voice feature means that you can have conversations with it. But there are some key differences between what Microsoft and OpenAI are offering, and only some of Microsoft's Copilot innovations will be available right away. It's probably safe to say, though, that Copilot Voice will be the most important addition -- and Copilot Vision may not be. In fact, some might see Copilot Vision as just another version of Recall, Microsoft's controversial search feature that the company has delayed while users test it more thoroughly. Here's what's going on: Microsoft may have been one of the first to market with Bing Chat, later dubbed Copilot. But Google, OpenAI, Meta, Anthropic, and more have debuted their own LLMs and subscriptions to go along with them. Microsoft, of course, has to keep justifying its Copilot Plus subscription and several of these new features -- Copilot Vision and "Think Deeper" -- will only be available in Copilot Labs, which in turn will be restricted to Copilot Plus subscribers. Does Copilot Vision sound similar to the controversial Recall feature? It does indeed and that's why Copilot Vision will be slow to deploy. But other features -- Copilot Voice, Copilot Daily, an updated Copilot for Microsoft Edge, and a Personalized Discover feature -- will be entirely free and will debut soon. You'll see these new features as part of new Copilot mobile apps today for iOS and Android, plus an updated Copilot app for Windows and a new look for copilot.microsoft.com, too. Microsoft is also beefing up generative search on Bing, presumably following Google's lead. However, these new Copilot features will not require a Copilot+ PC. Instead, these Copilot features run in the cloud. Microsoft is bringing new AI features to Windows that do depend on the NPU inside of a Copilot+ PC. And they're cool, too, including AI upscaling of Photos and generative fill within Paint. We've covered these as part of a separate story. Copilot Voice certainly seems like the new feature that will impact you the most. At least in a demonstration, Copilot Voice looked very similar to ChatGPT's voice feature. Microsoft showed it on a smartphone, with a hypothetical exchange where the user confessed that she was at the grocery store and didn't know what to bring a friend to their party. The conversation included interruptions -- the feature that helped ChatGPT seem like a real person -- explanations (what is a fancy olive oil?) and more. If this is representative of the final product, it could be useful. Like Copilot Voice, Microsoft showed off the new Copilot experiences running on a smartphone. Presumably, Microsoft wants you to start your day listening to Copilot Daily, an audible news summary of headlines Microsoft's AI thinks will be relevant to you. (Presumably you'll be able to access these things on a PC, too). Copilot Daily is just another iteration of Microsoft trying to be helpful. A decade ago, Cortana would try and deliver a snapshot of your day. In 2020, Cortana's Daily Briefing would send you an email outlining your meetings and what you would need to do to prepare. Copilot Daily's short demo did none of that and instead summarized news stories about the Alps, U.S. holiday sales, and surfing dogs. Copilot Daily is basically Microsoft's Start in an audio format with partners such as Reuters and Axel Springer. It's hard to say if Personalized Discover will be a more productive replacement. Microsoft's quick look at the technology made it look like just a more informed version of a new tab page on Edge, with subjects that "will be further personalized over time according to your chat history." Again, this feature looked pretty ephemeral. So does the update to Copilot for Microsoft Edge, which will essentially allow you to address Copilot like a coworker in a chat, by addressing it as "@copilot" in the address bar. Interestingly, Copilot Vision, a paid feature, is not like the multimodal AI you may have seen before from Google's Project Astra or competitors. It runs entirely on the PC, and simply interprets what you're already seeing on the screen. In fact, it was probably one of the most underwhelming demos I've seen in some time. In one example, a user browsed Rotten Tomatoes while asking Copilot for movie recommendations. The conversation was performed audibly via Copilot Voice, but all the chatbot did was read what the screen was already displaying! Perhaps it made sense that it could summarize the page, but it was a poor example. In another demonstration, however, a user browsed a recipe a grandmother had authored. Copilot Vision intelligently suggested how long it would take, using knowledge not gleaned from the page. That is hopefully what Copilot Vision will provide. Some will see this as just another version of the controversial Recall, which Microsoft has already taken pains to allow users to opt out of or not install at all. Microsoft says that Copilot Vision will be entirely opt-in, and that data collected by Copilot Vision will be "ephemeral," in that (at least in the preview) data will not be stored and will be immediately discarded. Microsoft has also "put boundaries on the types of websites Copilot Vision can engage with" and will whitelist a set of sites. Finally, Copilot Vision won't work on "paywalled and sensitive content" for this preview. Whether that includes things like financial websites or private browsing hasn't been made clear. "Before we launch broadly, we'll continue to take feedback from early users in Copilot Labs, refine our safety measures, and keep privacy and responsibility at the center of everything we do," Microsoft said of Copilot Vision. Microsoft hasn't said whether or not its upcoming Think Deeper feature is essentially GPT4-o, an AI that can ponder or reason. Think Deeper takes more time before responding. "Using the very latest in AI technology, it can help you with anything from solving tough math problems to weighing up the costs of managing home projects," Microsoft says. Copilot Voice will be available in the U.S., Canada, the U.K., and other regions, presumably today. Copilot Daily and personalization options are available today, too. Ditto for Think Deeper, which will be pushed to Copilot Pro users. But Copilot Vision is coming to Labs "in the future," and to a limited number of users.

Microsoft Copilot can now read your screen, think deeply, and speak aloud to you

A week after announcing a wave of updates for its enterprise suite of Copilot AI-powered products, Microsoft is launching new Copilot capabilities on Windows for all users, including a tool that can understand and respond to questions about what's on your screen. Refreshed Copilot apps for iOS, Android, Windows and the web are rolling out today, and all feature a Copilot with a more "warm" and "distinct" style, as Microsoft describes it. Microsoft is also bringing the chatbot to WhatsApp, letting users chat with Copilot via DM, similar to the experience you get with other bots on Meta's messaging platform. Copilot Vision Copilot Vision has a view of what you're viewing on your PC -- more specifically, a lens into the sites you're visiting with Microsoft Edge. Gated behind Copilot Labs, a new, Copilot Pro-exclusive opt-in program for experimental Copilot capabilities, Copilot Vision can analyze text and images on webpages and answer queries (e.g., "What's the recipe for the food in this picture?") about them. Vision, which can be pulled up by typing "@copilot" in Edge's address bar, isn't exactly a technical marvel. Google offers similar search technology on Android, and recently brought bits and pieces of that tech to Chrome as well. But Microsoft suggests that Copilot Vision is more powerful and conscious of privacy than previous screen-analyzing features. "Copilot Vision can ... suggest next steps, answer questions, help navigate whatever it is you want to do, and assist with tasks, all while you simply speak to it in natural language," Microsoft wrote in a blog post shared with TechCrunch. "Imagine you're trying to furnish a new apartment. Copilot Vision can help you search for furniture, find the right color palette, think through your options on everything from rugs to throws, and even suggest ways of arranging what you're looking at." No doubt eager to avoid another round of bad press from AI privacy fumbles, Microsoft is stressing that Copilot Vision was designed to delete data immediately following conversations. Processed audio, images or text aren't stored or used to train models, the company claims -- at least not in this preview version. Copilot Vision is also limited in the types of websites that it can interpret. For the time being, Microsoft's blocking the feature from working on paywalled and "sensitive" content, limiting Vision to a pre-approved list of "popular" web properties. What does "sensitive" content entail, exactly? Porn? Violence? At this juncture, Microsoft wouldn't say. Accusations of circumventing paywalls with AI tools have landed Microsoft in legal hot water in the recent past. In an ongoing lawsuit, The New York Times alleged that Microsoft allowed users to get around its paywall by serving NY Times articles through the Copilot chatbot on Bing. When prompted in a certain way, Copilot -- which is powered by close Microsoft collaborator OpenAI's models -- would give verbatim (or close-to-verbatim) snippets of paid stories, according to The Times. Microsoft said that Copilot Vision, which is U.S.-only at the moment, will respect sites' "machine-readable controls on AI" -- like rules that disallow bots from scraping data for AI training. But the company hasn't said precisely which controls Vision will respect; there are several in use. We've asked Microsoft for clarification. Many major publishers have opted to block AI tools from trawling their websites not only out of fear their data will be used without permission, but also to prevent these tools from sending their server costs soaring. If the current trend holds, Copilot Vision may not work on some of the web's top news sites. Microsoft said it's committed to "taking feedback" to allay concerns. "Before we launch broadly, we'll continue to ... refine our safety measures and keep privacy and responsibility at the center of everything we do," Microsoft said in the blog post. "There is no specific processing of the content of a website you are browsing [with Copilot], nor any AI training -- Copilot Vision simply reads and interprets the images and text it sees on the page for the first time along with you." Think Deeper As with Vision, Copilot's new Think Deeper feature is an attempt to make Microsoft's assistant more versatile. Think Deeper gives Copilot the ability to reason through more complex problems, Microsoft said, thanks to "reasoning models" that take more time before responding with step-by-step answers. Which reasoning models? Microsoft was a bit cagey when I asked, saying only that Think Deeper uses "the latest models from OpenAI, fine-tuned by Microsoft." Reading between the lines, it's a safe bet that they're a customized version of OpenAI's o1 model. "We've designed Think Deeper to be helpful for all kinds of practical, everyday challenges, like comparing two complex options side by side," Microsoft wrote in a blog post. "Think Deeper can help with anything from solving tough math problems to weighing up the costs of managing home projects." Microsoft talked up Think Deeper's potential quite a bit in its press materials. But assuming the model underneath is o1, it will most certainly fall short in some areas. We're curious to see what sort of enhancements Microsoft made to the base model, and how forthcoming Think Deeper is about its limitations. Think Deeper will be available from today to a limited number of Copilot Labs users in Australia, Canada, New Zealand, the U.S. and the U.K. Copilot Voice A new Copilot feature generally available today is Copilot Voice (not to be confused with GitHub's Copilot Voice). Launching in English in New Zealand, Canada, Australia, the U.K. and the U.S. to start, Voice adds four synthetic voices, letting you talk to Copilot and have its responses be spoken aloud. Like OpenAI's Advanced Voice Mode for ChatGPT, Copilot Voice can pick up on your tone during conversations and respond accordingly, and you can interject at any point while Copilot Voice is answering. A Microsoft spokesperson told me that the mode uses "the latest voice technology with new models that have been fine-tuned for the Copilot app." What tech? Which models? On the specifics, mum's the word. One thing to be aware of: Copilot Voice has a time-based usage limit. Copilot Pro subscribers get more minutes but the number is "variable," Microsoft told me, depending on demand. Personalization Copilot will soon become more tailored to your likes and preferences, Microsoft said, thanks to a new personalization setting. When the setting is enabled, Copilot will draw on your past interactions and history, as well as your interactions with other Microsoft apps and services (Microsoft won't say which) to recommend ways to use Copilot. "This helps you get going," Microsoft wrote in a blog post, "offering both a handy guide to Copilot's useful features and conversation starters." Personalization in Copilot, which can be switched off in the Copilot settings menu on Windows, isn't slated for the U.K. or EU anytime soon. But users elsewhere should begin to see the setting this afternoon. Microsoft and the EU have had a testy relationship where it concerns the company's AI product rollouts. In May, the EU warned Microsoft that it could be fined up to 1% of its global annual turnover under the bloc's online governance regime, the Digital Services Act, after the company failed to respond to a request for information that focused on its generative AI tools. A number of tech giants beyond Microsoft, including Apple and Meta, have taken a cautious approach to launching AI tools in the EU, wary of running afoul of the bloc's laws governing data privacy and model deployment. "For users in the European Economic Area (EEA) and a limited number of other countries, we are evaluating options before offering this level of Copilot personalization for those users," a Microsoft spokesperson told TechCrunch. "Some features will not be available in the EEA until a later date."

Every new Microsoft Copilot feature and AI upgrade coming soon to your Windows PC

Microsoft is doubling down on AI with new Copilot features and other AI tools. Here's who gets them first and when you can expect to see them. For anyone running Windows 11 on standard PC hardware, the new 24H2 release will be a ho-hum event, with mostly minor tweaks to the Windows experience. But that's emphatically not the case for anyone with one of the new Copilot+ PCs that debuted earlier this year. Also: What's new in Windows 11 24H2, and when will your PC get it? On those devices, the Windows 11 2024 Update will enable a handful of new AI-focused features that will roll out gradually via Windows Update and through app updates in the Microsoft Store. Those new features won't go to the general public right away. Instead, they'll go to members of the Windows Insider Program, starting with Qualcomm-powered Arm devices in October and broadening in November to Copilot+ PCs built using Intel's new Core Ultra CPUs and AMD's Ryzen AI hardware. Also: How to upgrade your 'incompatible' Windows 10 PC to Windows 11: Two options What they have in common is that they take advantage of the built-in neural processing unit (NPU) on Copilot+ PCs to perform those AI-related tasks locally. That saves bandwidth and energy, and it also keeps personal information private instead of sending it to the cloud. The first AI feature to arrive will be a preview version of the controversial Recall capability, which promises the ability to give your PC a "photographic memory." Microsoft had originally planned to debut this feature in June with the first wave of Copilot+ PCs but canceled those plans after a flood of negative feedback from security experts. The revised code leverages core Windows 11 features like virtualization-based security filters to keep sensitive information from being collected. Recall will be turned off by default and only available for customers who opt to enable the feature. For more details, see "Microsoft announces sweeping changes to controversial Recall feature for Windows 11 Copilot+ PCs." Microsoft pitches this feature as a way to enhance productivity by adding access to AI-based actions to a shortcut menu that appears over images or text. After taking a screenshot with Snipping Tool, for example, you can press the Windows key and right-click to see a menu that offers to blur the background or erase objects from a photo. Over a block of text, the shortcut menu might offer to rewrite or summarize the text in line. As the "preview" label suggests, this is another feature that's likely to be of limited value initially. Microsoft promises that it will add more "rich actions" over the coming months. The trouble with the search tools built into Windows is that they're extremely literal. This batch of improvements promises to eliminate that frustration by allowing you to describe a file, photo, or setting you're looking for without having to worry about file names or search syntax. The feature also claims to work around spelling errors and can find photos and files stored locally or in OneDrive. Also: Have a Windows 10 PC that can't be upgraded? You have 5 options before support ends next year Although it's not billed as a preview, this feature is also starting out with a limited scope. It will be available in the File Explorer search box initially and will expand to Windows Search and Settings later. The three features mentioned earlier will arrive via Windows Update. Microsoft is also delivering some of its AI enhancements via updates from the Microsoft Store to apps included with Windows. The new Super Resolution feature in the Photos app makes a bold promise that it can "reimagine and enhance" old, low-resolution photos. A slider can reportedly increase the resolution by up to 8X, using the NPU to perform the task locally within seconds. Meanwhile, an update to the Paint app will add generative fill and generative erase features to "remove unwanted or distracting elements in your image or add new ones, exactly where you want them." Microsoft really wants you to tap the Copilot key on that new Copilot+ PC and to sweeten the deal it's promising to infuse the AI companion with a megadose of feelings. The new Copilot will be "always by your side and in your corner, helping you feel smarter and more supported through natural conversations," the company says in a slightly overwrought blog post announcing the changes. Also: Microsoft to start charging for Windows 10 updates next year. Here's how much "It now adapts to you with a warm tone and a distinct personality, providing not only information but encouragement, feedback and advice as you navigate life's everyday challenges - no matter how big or small." The updated Copilot will be available on mobile apps (iOS and Android), on the web, in the Copilot app for Windows, and as a feature in WhatsApp. It will include a half-dozen or so new and enhanced features. You'll be able to chat using Copilot Voice (with four voice options available for the companion), and you can request a daily summary of news and weather, read in that voice, using the Copilot Daily feature. If you opt in to the Copilot Discover feature, you'll see cards personalized from your chat history that offer conversation starters. In Microsoft Edge, you can access Copilot features using the @copilot shortcut. Two new experimental features are also debuting as part of the Copilot Labs program for paid subscribers. Copilot Vision allows you to discuss the contents of a web page (images and text) with the AI companion in real time. You can ask questions and get suggestions based on that content. Also: Is OneDrive messing with your files? How to get your Windows storage under control If it sounds like that could lead to some problematic scenarios, you're right. There's no question that Microsoft was stung by criticism of its initial Recall launch, and today's announcement goes out of its way to emphasize "safety and security" guardrails. None of the content you engage with in this feature is stored or used for training, it promises: "The moment you end your session, data is permanently destroyed." There are also "boundaries on the types of websites Copilot Vision can engage with." (It doesn't take much imagination to see where that could lead.) For now, the feature is limited to a short list of popular websites, and it won't be available for paywalled and sensitive content. Also: The Windows laptop I recommend to most people undercuts the MacBook Air (but is just as capable) Finally, there's a Think Deeper feature designed to reason through more complex questions, including tough math problems. The new features are rolling out in limited markets initially.

Microsoft ushers in several new Copilot, Copiliot+ features alongside Windows 11 24H2

AI everywhere, but also new battery-saver functionality, compatibility with hearing aids, and Wi-Fi 7 support. Microsoft is updating both Windows 11 and Copilot today, announcing a slew of new features for both the operating system and the virtual assistant. Some of these features are limited to Copilot+ PCs (and whether that's Qualcomm, or also Intel and AMD is a bit muddled at the moment), such as the debut of the controversial Recall feature, which has gotten a security upgrade. The Windows features are rolling out broadly, starting with PCs that are eligible devices running Windows 11 22H2 and 23H2 and those who opted in settings to get updates first, and then through Microsoft's typical phased approach. All eligible Windows PCs will get Windows 11 24H2. (If you want to get a head start, go to Settings > Windows Update and turn on "Get the latest updates as soon as they're available."). John Cable, Microsoft's vice president for Windows servicing and delivery, writes in a blog post that the update "is a full operating system (OS) swap that contains new foundational elements required to deliver transformational AI experiences and exceptional performance." The update includes a handful of features. There's a new Energy Saver mode that you can use to extend battery life (or, even if the device is plugged in, attempt to reduce your energy usage). This is accessible in the system tray, or you can set it up to turn on at a specific battery percentage. There is new support for audio devices, including hearing aids and cochlear implants, as long as both your PC and your hearing device support Bluetooth LE. These will allow people to gain access to better audio presets, customized sound enhancements, and ambient sounds, as well as stream audio direct to their aids. Windows is also getting Wi-Fi 7 compatibility for faster speeds, as well as HDR background support on computers with screens that can use the technology. Microsoft will also be making changes to the File Explorer, Settings, taskbar, and system tray. Some features, however, are limited to Copilot+ PCs. Unlike the initial features, which went only to Qualcomm Snapdragon X computers, Microsoft tells me that these features will now also come to Intel's Ultra Core Series 2 and AMD's Ryzen AI 300. We're still waiting to learn which features will be deployed widely and which ones may go through Windows Insider preview first. We'll update this story when Microsoft responds. The feature sure to make the most noise is the return of Recall, which had been pulled from its June launch date due to privacy concerns. Now it's back with a new security architecture (and as an opt-in feature that you can remove altogether).It will continue to work by taking snapshots of your computer usage for you to search through later, but now with better filtering to prevent Recall from scanning certain websites and apps, as well as filtering out personal information like credit card numbers and ID numbers. Recall will launch in Windows Insider in October for Qualcomm Snapdragon Copilot+ PCs, followed by AMD and Intel Copilot+, with wider availability to "be shared soon." Another preview update is Click to Do, which suggests tasks based on items you click. For instance, clicking on a picture with Windows and left click might recommend removing the background, searching with Bing, or using background blur in the Photos app, while selecting the text in an email may suggest rewriting or auto-summarizing it. It definitely feels like a way to suggest other Copilot+ AI features, but we'll have to get our hands on it to see if it feels useful. One feature that I think may be more broadly beneficial is "Improved Windows Search," which will allow for natural language to look for photos and files. Microsoft is using the example of finding photos from a recent barbecue by searching for "BBQ Party" in File Explorer, without worrying about actual file names folders. In the future, this should show up in the Windows Search Box or in Settings (the latter in which the company demonstrated the idea of searching for a place to add headphones being directed to Bluetooth and audio settings). The photos app is getting a new Super Resolution feature, which uses on-device AI to increase resolution on old or low-res photos by up to eight times. (For instance, a 720p photo could go up to what I suppose you could call 5760p). A slider in the Photos app lets you decide how far you want to go, and Microsoft suggests the whole process can "upscale a photo up to 4K within seconds". Lastly, Paint, which previously got the Cocreate feature, is now getting "Generative Fill and Erase" allowing you to select a part of a picture and then use natural language to describe what you want added -- or removed -- from a photo. In some ways, this is a bit like Google Photos' Magic Eraser on Android, letting you remove people from the backgrounds. But you can also add elements to more creative drawings (Microsoft showed press a demo of adding a lighthouse to a beach scene), similar to what Adobe added to Photoshop recently. In a Q&A with journalists, Pavan Davuluri, the head of Windows and Devices at Microsoft, stated that features from Qualcomm's Snapdragon Copilot+ PCs, such as Cocreate and Live Translation "will come" to AMD and Intel PCs, but that Microsoft needs to work on the best way to ship the features. He also said that Microsoft has learned "users want to have extreme confidence" in features with sensitive data, referring to Recall, and that the company will continue to push updates through Insider for feedback. Separate from Copilot+ updates, Copilot, the AI assistant, is getting its own new features. Many of them are set to launch today on the web, the Windows Copilot App, and updated Android and iOS mobile apps. The first is Copilot Voice, which will let you talk to Microsoft's AI in natural language. Microsoft says that this is the easiest way to talk to Copilot, and that voice is "the most intuitive and natural way to brainstorm on the go, ask a quick question, or even just vent at the end of a tough day." I'm not sure if I want to rant at Copilot after a long day at work, but I suppose I'll have to try it. There will be four voice options to choose from. The voice feature will launch in English in the U.S., United Kingdom, Australia, Canada, and New Zealand, and will be available in more countries and languages "soon." Microsoft is also adding Copilot Daily, a "summary of news and weather" that sounds like features you could use with Amazon's Alexa speakers for years. Microsoft is working with media partners like Axel Springer (parent company of Business Insider, among others), Hearst Magazines, and Reuters, with plans to add more publications and personalization features over time. Some new features will be in Copilot Labs, a program for Copilot Pro customers to try experimental features that are being worked on. The first is Copilot Vision (which, like Recall, Microsoft points out is "entirely opt-in." Vision is built into the Edge browser, and can read the images and text on the page, allowing you to ask questions and have Copilot answer them using information from the page. Microsoft is stressing that, "in this new preview," data isn't stored. There's also a restriction on certain types of websites, and the company is using a "limited list of popular websites" and that Copilot Vision won't allow responses on paywalled or "sensitive" content. The second is "Think Deeper," which goes into standard Copilot chat to work with complicated requests. In this mode, Copilot will take more time before responding, which the company says will allow it to do better math, write improved code, or offer more detailed answers to problems.

Microsoft doubles down on AI with new Copilot features - here's who gets them first

What's new in Windows 11 24H2? Lots and lots of AI features, that's what. Most will be limited to the latest hardware, but a few are coming to a PC or mobile device near you soon. For anyone running Windows 11 on standard PC hardware, the new 24H2 release will be a ho-hum event, with mostly minor tweaks to the Windows experience. But that's emphatically not the case for anyone with one of the new Copilot+ PCs that debuted earlier this year. Also: What's new in Windows 11 24H2, and when will your PC get it? On those devices, the Windows 11 2024 Update will enable a handful of new AI-focused features that will roll out gradually via Windows Update and through app updates in the Microsoft Store. Those new features won't go to the general public right away. Instead, they'll go to members of the Windows Insider Program, starting with Qualcomm-powered Arm devices in October and broadening in November to Copilot+ PCs built using Intel's new Core Ultra CPUs and AMD's Ryzen AI hardware. Also: How to upgrade your 'incompatible' Windows 10 PC to Windows 11: Two options What they have in common is that they take advantage of the built-in neural processing unit (NPU) on Copilot+ PCs to perform those AI-related tasks locally. That saves bandwidth and energy, and it also keeps personal information private instead of sending it to the cloud. The first AI feature to arrive will be a preview version of the controversial Recall capability, which promises the ability to give your PC a "photographic memory." Microsoft had originally planned to debut this feature in June with the first wave of Copilot+ PCs but canceled those plans after a flood of negative feedback from security experts. The revised code leverages core Windows 11 features like virtualization-based security filters to keep sensitive information from being collected. Recall will be turned off by default and only available for customers who opt to enable the feature. For more details, see "Microsoft announces sweeping changes to controversial Recall feature for Windows 11 Copilot+ PCs." Microsoft pitches this feature as a way to enhance productivity by adding access to AI-based actions to a shortcut menu that appears over images or text. After taking a screenshot with Snipping Tool, for example, you can press the Windows key and right-click to see a menu that offers to blur the background or erase objects from a photo. Over a block of text, the shortcut menu might offer to rewrite or summarize the text in line. As the "preview" label suggests, this is another feature that's likely to be of limited value initially. Microsoft promises that it will add more "rich actions" over the coming months. The trouble with the search tools built into Windows is that they're extremely literal. This batch of improvements promises to eliminate that frustration by allowing you to describe a file, photo, or setting you're looking for without having to worry about file names or search syntax. The feature also claims to work around spelling errors and can find photos and files stored locally or in OneDrive. Also: Have a Windows 10 PC that can't be upgraded? You have 5 options before support ends next year Although it's not billed as a preview, this feature is also starting out with a limited scope. It will be available in the File Explorer search box initially and will expand to Windows Search and Settings later. The three features mentioned earlier will arrive via Windows Update. Microsoft is also delivering some of its AI enhancements via updates from the Microsoft Store to apps included with Windows. The new Super Resolution feature in the Photos app makes a bold promise that it can "reimagine and enhance" old, low-resolution photos. A slider can reportedly increase the resolution by up to 8X, using the NPU to perform the task locally within seconds. Meanwhile, an update to the Paint app will add generative fill and generative erase features to "remove unwanted or distracting elements in your image or add new ones, exactly where you want them." Microsoft really wants you to tap the Copilot key on that new Copilot+ PC and to sweeten the deal it's promising to infuse the AI companion with a megadose of feelings. The new Copilot will be "always by your side and in your corner, helping you feel smarter and more supported through natural conversations," the company says in a slightly overwrought blog post announcing the changes. Also: Microsoft to start charging for Windows 10 updates next year. Here's how much "It now adapts to you with a warm tone and a distinct personality, providing not only information but encouragement, feedback and advice as you navigate life's everyday challenges - no matter how big or small." The updated Copilot will be available on mobile apps (iOS and Android), on the web, in the Copilot app for Windows, and as a feature in WhatsApp. It will include a half-dozen or so new and enhanced features. You'll be able to chat using Copilot Voice (with four voice options available for the companion), and you can request a daily summary of news and weather, read in that voice, using the Copilot Daily feature. If you opt in to the Copilot Discover feature, you'll see cards personalized from your chat history that offer conversation starters. In Microsoft Edge, you can access Copilot features using the @copilot shortcut. Two new experimental features are also debuting as part of the Copilot Labs program for paid subscribers. Copilot Vision allows you to discuss the contents of a web page (images and text) with the AI companion in real time. You can ask questions and get suggestions based on that content. Also: Is OneDrive messing with your files? How to get your Windows storage under control If it sounds like that could lead to some problematic scenarios, you're right. There's no question that Microsoft was stung by criticism of its initial Recall launch, and today's announcement goes out of its way to emphasize "safety and security" guardrails. None of the content you engage with in this feature is stored or used for training, it promises: "The moment you end your session, data is permanently destroyed." There are also "boundaries on the types of websites Copilot Vision can engage with." (It doesn't take much imagination to see where that could lead.) For now, the feature is limited to a short list of popular websites, and it won't be available for paywalled and sensitive content. Also: The Windows laptop I recommend to most people undercuts the MacBook Air (but is just as capable) Finally, there's a Think Deeper feature designed to reason through more complex questions, including tough math problems. The new features are rolling out in limited markets initially.

Microsoft Copilot, Copilot+ PC, Windows 11 AI Updates Start Rolling Out

New Copilot+ PC and Windows 11 features will come out this month for Windows Insider Program (WIP) members. Microsoft has started rolling out new features and previews for its Copilot artificial intelligence assistant offerings and for Windows 11 and Copilot+ AI PC users, including a way to interact with Copilot by voice, a preview of an AI capability that understands text and images on web pages and a PC experience in preview that gives quick action suggestions based on images and text on screen. Some of the Copilot upgrades will become available through the Redmond, Wash.-based vendor's new Copilot Labs program for limited numbers of Copilot Pro users to test new features in a preview environment. Two features coming to Labs for a limited number of users for testing "next week" are Copilot Vision and Think Deeper. New Copilot+ PC and Windows 11 features will come out this month for Windows Insider Program (WIP) members, with a phased rollout to certain devices and markets starting Nov. 1, according to Microsoft. A newer version of Microsoft Recall than the one that stirred controversy earlier this year is among the features that will become available for WIP members this month. WIP members will see Recall on Arm for Qualcomm-powered Copilot+ PCs this month. Kevin Beaumont, a cybersecurity professional who criticized the initial version of Recall, called the planned changes "much more robust" in a post to social media website Mastodon on Friday. "Some of the things are boomerangs - eg they said it wasn't uninstallable weeks ago, but it is now," Beaumont said. "Also they said it wasn't developed under Secure Future Initiative a few months ago.. but now say it was originally under SFI. The proof is in the pudding obviously so hands on tests will be required. They've locked it to Copilot+ PC systems now, which will limit research." Users with these AI PCs powered by Intel and Advanced Micro Devices (AMD) will see Recall in WIP in November. More specific availability dates are expected "soon," according to Microsoft. The news comes soon after a Microsoft virtual event detailing a new wave of Copilot updates centered on a new Copilot Pages canvas offering, more integrations with existing productivity applications and capabilities around AI agents. Microsoft bills the new Copilot Voice interaction method as a way to brainstorm, ask a quick question and talk to Copilot. Users have four voice options. Available now is Voice in English in the U.S., U.K., New Zealand, Canada and Australia, according to Microsoft. More regions and languages are coming "soon" for Voice. Copilot Daily is a summary of news and weather read aloud. Microsoft said to expect more sources, personalization and controls in Daily over time. Daily starts rolling out today in the U.S. and U.K., with more countries and an option for reminders coming "soon." The feature will only pull from authorized content sources, with current content partners including publishers Reuters, Hearst Magazines (whose brands include Cosmopolitan, Elle, Esquire, Seventeen, Popular Mechanics) and Axel Springer (owner of Politico and Business Insider). Two features coming to the Copilot Labs testing program "next week" are Copilot Vision and Think Deeper. Copilot Vision-only coming to select testers in the U.S.-will allow AI to see everything on a web page and talk to the user about it in real time. Vision is integrated into the Microsoft Edge browser and understands the text and images on the page a user is viewing, according to Microsoft. Vision can answer questions about the content, suggest next steps and help with tasks without disrupting workflow. Vision won't store or use content it engages with for training while in preview, according to Microsoft. Data is permanently discarded once a session ends. Think Deeper will only come to select Copilot Pro users in the U.S., U.K., New Zealand, Canada and Australia. This capability allows Copilot to reason through more complex problems, from higher-level math to the cost of a home project. This feature takes more time before a response to provide detailed answers with steps. Rolling out for Microsoft's Bing search engine starting today is generative search for dynamically generated, curated answers. Users can search "Bing generative search" for query examples. Microsoft said the feature is considered still under development. Also starting today are updated Copilot applications for iOS and Android, the Copilot web experience, a Copilot experience for WhatsApp and the Copilot Windows app, according to Microsoft. Copilot Discover is meant for users not sure where to start with the AI tool. Discover can present cards based on interactions with other Microsoft services with permission from the user. The cards become more personalized over time based on chat history. And Microsoft Edge browser users can access Copilot by typing "@copilot" in the address bar. In Edge, Copilot can answer questions, summarize page content, translate text, rewrite sentences and perform other tasks. Turning to the new features for Copilot+ PCs and Windows 11 users, Click to Do is Microsoft's name for an experience in preview that places an interactive overlay on top of PC screens to help simplify user workflow. With the overlay, quick action suggestions will appear over images and text, connecting users to tools and actions to accomplish a task. For example, users might receive a suggestion for a visual search with Microsoft Bing, a suggestion to erase an object in a photo, an offer to remove a background with Microsoft Paint, an offer to send an email with a recognized email address, and offer to open a website from a recognized web address or an idea for a way to rewrite or summarize a text. Click to Do is context-aware and accessible from any Copilot+ PC screen with the Windows key and clicking the mouse to find the feature in Snipping Tool or Print Screen, according to Microsoft. The vendor also promises more actions coming for Click to Do "in the coming months." Windows 11 24H2 update started rolling out to Windows 11 users today. The update includes a feature to extend battery life and reduce energy use even when devices are plugged in. The update also adds improved hearing aid support with Bluetooth LE Audio, Wi-Fi 7 compatibility and high dynamic range (HDR) background support, according to Microsoft. The vendor will use a phased rollout starting in "the coming weeks" with eligible devices running Windows 11 version 22H2 and 23H2 if the user is opted in to experience new enhancement. In File Explorer is an improved search capability where users can describe what they are looking for instead of remembering file names, settings locations and writing with perfect spelling, according to Microsoft. "In the coming months," improved search will expand to Windows Search and Settings. The improved search works when users aren't connected to the internet as well due to Copilot+ PCs' 40 trillion-plus operations per second (TOPS) neural processing units (NPUs), according to the vendor. The NPUs will allow for super resolution in photos for upscaling up to 4K in seconds for free, according to Microsoft. The super resolution capability also promises to change older, lower-quality images into high-quality ones without blurring or pixelation with a slider. Microsoft Paint has gained generative fill and generative erase features for editing and altering images. The underlying diffusion-based model has received an upgrade for better results faster with built-in moderation, according to the vendor. Native Windows on Arm applications "on the horizon" include Google Drive, Arc Browser, Sketchbook Pro, Fantastical, Vegas Pro and NordVPN, according to Microsoft.

Microsoft delivers new Copilot+ AI PC features with Windows 11's 2024 update

It starts rolling out today, and Copilot+ PC owners can test new features soon. Now that we're a few years into Microsoft's obsession with AI and its Copilot assistant, it's clearer than ever that Windows 11's role is to show off the company's artificial intelligence prowess. At least, that's the message I took away from today's announcement that the Windows 11 2024 (version 24H2) update is now beginning to roll out. While the OS itself is getting a few new features, like long overdue File Explorer tweaks, Microsoft is far more eager to hype up new Copilot+ AI PC capabilities. There's "Click to Do," which triggers Copilot to assist you with whatever is on your screen, like removing an object in the Photos app, or summarizing a long article. It's seemingly easy to use -- just hold the Windows button down and click your target -- and Microsoft says it'll offer contextually relevant tips. As with so many of the company's AI features, you can think of Click to Do as an on-demand super-powered version of Clippy, its infamous software helper from years past. But since Click to Do appears at your whim, and there are actually some potentially useful AI features, it should hopefully be far less annoying than that damn paperclip. You'll also be able to upscale pictures in the Photos app by up to eight times their resolution. A slider will let you adjust precisely how much you'd like to increase the quality. This isn't particularly new -- Adobe has its own AI-powered Super Resolution capability, as does the popular Mac photo editing app Pixelmator. But at least it's helpful to have it built directly into your OS. Generative AI-based fill and erase options are also coming to Paint, allowing you to remove objects or easily create new ones. Microsoft has hinted at some AI-powered search improvements in the past, and it looks like those are finally arriving with the Windows 11 2024 update. Now you can find files using your own words, without worrying about esoteric search syntax. If you want to find pictures of your dog by the beach, you can just type that. And of course, there's Recall, the company's debut Copilot+ feature for retrieving anything that happens on your computer. It was immediately criticized for being surprisingly insecure -- researchers discovered that hackers could access the Recall screenshot database without administrator privilege. Microsoft immediately delayed Recall to revamp its security model: It's now making the feature completely opt-in, instead of flipping it on by default. Windows Hello biometric authentication is required to use Recall, and it's also encrypting the screenshot database and other interactions. Last week, the company detailed more of its security methods, including using VBS Enclaves to further isolate Recall from hackers. David Weston, Microsoft's VP of OS and enterprise security, noted in a blog post that "you are always in control" of the Recall experience. That reassurance may not be enough for users turned off by Microsoft's initial security flubs, though. If anything, Recall's rollout was a clear example of how the company was cutting corners to move quickly and be seen as an AI industry leader. "Copilot will be there for you, in your corner, by your side, and always strongly aligned with your interests.," Mustafa Suleyman, the company's head of AI, wrote in an overly enthusiastic blog post. "It understands the context of your life while safeguarding your privacy, data and security, remembering the details that are most helpful in any situation." Security will undoubtedly be a major concern for Copilot users moving forward, and according to Microsoft's Windows head, Pavan Davuluri, the company has learned from its troubled Recall launch. "Ultimately, users want to have confidence with anything happening with sensitive data caches," he said in a briefing with press, adding that he believes the company has "gone above and beyond" to create that confidence. He also acknowledged that AI features enable "new attacks and defenses." For example, Windows has traditionally allowed administrator accounts to have total access to everything on a machine, but now users may want to see "additional rings of protection" with AI features. Davuluri stressed that community feedback helped Microsoft make Recall more secure for users, and the company will continue listening as it carefully rolls out more AI-infused capabilities. That's also why all of the above Copilot+ features won't be rolling out to everyone immediately -- they'll be available to Windows Insiders first, and then arrive via a phased rollout to "select devices and markets" in November. The Windows 11 2024 update will also have a staggered rollout starting today. If you're eager to snag it, make sure you've enabled "Get the latest updates as soon as they're available" in Windows Update. And what about features from this Windows 11 update that don't require Copilot+ PCs? They appear at the tail-end of Davuluri's blog post today, almost as an afterthought: There's Wi-Fi 7 support, HDR backgrounds, Energy Saver improvements to prolong battery life and better hearing aid support using Bluetooth LE. They sound like the Windows upgrades we used to see before Microsoft became completely AI-pilled, but they're certainly not as exciting as something like Click to Do. Of course, that's all intentional. If you want to join the Copilot+ AI PC party, you'll probably have to get a new computer. And that's precisely what Microsoft and PC makers want.

Microsoft outs Windows 11's AI roadmap: Smart search, upscaling, more

What's next for Windows? Microsoft may have just shipped the Windows 11 2024 Update (24H2), but the company is already disclosing its plans for next-gen Windows apps -- and there's some very interesting AI-powered features due by the holidays. Microsoft disclosed that it is working on several AI additions to Windows and Windows apps: improved Windows search using natural descriptive language, super resolution in Photos, generative fill and erase in Paint, as well as the debut of Recall. All (save Recall) will appear in October as part of the Windows Insider program, with an expected launch in November. All of these features will depend on the NPU inside of Copilot+ PCs, which will now include PCs that feature Qualcomm Snapdragon X Elite processors as well as AMD's Ryzen AI 300 and Intel's Lunar Lake. Microsoft also plans several more Copilot features that will run in the cloud, which will include Copilot Voice and Copilot Vision, similar to the innovations used in competing AI services. The timing of these new features will vary by platform, however, as Snapdragon X PCs have shipped for several months now; Microsoft will enable the Copilot+ capable PCs from AMD and Intel with their own updates. Microsoft has already disclosed more details of its Recall revamp, which the company now says can be bypassed when setting up a new PC or removed later. Windows Recall takes snapshots of your screen every so often, extracting the data and then storing it in case you need it later. The feature came under fire because of claims that it invaded user privacy and was insecure. Now, Microsoft says that it's storing the screenshots used in Recall and the extracted data inside an encrypted enclave. Security researchers have previously said that the data was stored unencrypted. Insiders who own Snapdragon X PCs will be able to experience Recall via the Windows Insider program beginning in October. If you own a Copilot+ PC with an AMD or Intel chip inside, the Insider trial will begin in November instead. When will Microsoft release Recall to the public? For now, that's still a mystery. Microsoft said that it plans to improve search on your PC by using more natural language when searching for files on your PC. You may have already seen this in apps like Microsoft Photos, or Google's Photos app, too; if you search for "beach," for example, the apps will use AI to identify beach scenes. Microsoft will apply the same techniques in File Explorer, though it's not clear what folders or files they'll apply to. The improved Windows search does seem to do a better job at identifying scenes than it did before: "BBQ party" was listed as an example search term in a demonstration, below. "You no longer need to remember file names, settings locations, or even worry about spelling -- just type what's in your mind to find it on a Copilot+ PC," Microsoft says. It seems unlikely, however, that you'll be able to find a particular .ini file in the depths of your user folder with the same ease that you'll find a photo of your aunt's wedding. Improved Search will begin with File Explorer, then expand to Windows Search and Settings "in the coming months." Super Resolution within Photos may be my favorite potential app, if only because a.) I have many old photos from older, crappy digital cameras, and b.) journalists often are supplied with low-resolution photos that need to be expanded or upscaled for publishing. In either case, the new Super Resolution feature promises to solve those problems. Microsoft announced Automatic Super Resolution as a way to improve games, but Super Resolution for Photos appears even more practical. There are numerous sites and applications that promise upscaling, and it's not clear whether this new app will outperform those. Still, Super Resolution for Photos will be free. Tapping the Copilot+ PC's AI TOPS, you'll be able to increase the resolution up to eight times, Microsoft says. Super Resolution will be part of Photos, which can already automatically adjust lighting and tint, remove a background, add generative elements, and more. Speaking of generative AI, you knew that Microsoft had more up its sleeve. In Paint, Microsoft already added Image Creator and Cocreator, which allows for AI-generated images that can be added via layers -- also part of Paint. In Photos, generative erase is also a current feature. Photos previously had spot erase, which was wonderful for digitally removing dust that evaded my cleaning cloth when taking a photo of a laptop or docking station. Generative erase, also already in Photos, allows you to define an entire region -- such as a power cord, say -- and remove that, too. (You know what's weird? Generative erase in Photos doesn't demand an NPU to work. But Microsoft is saying that generative erase in Paint, will.) Generative Fill basically allows you to define an area within Paint and then create a piece of generative art to fill that region, intelligently working its way into the background. You could sort of already do this in Paint, by creating a second layer, then a piece of AI art, then combining the two. This new feature adds smarts: It instructs the AI to accommodate both your defined region as well as the background when creating the art, making a seamless whole. The final feature, called Click to Do, is a bit of a puzzle. Normally you can hover your mouse cursor over any part of a web page, right click, and receive any number of options. Now, Microsoft wants you to click the Copilot button, then (left) click somewhere on the screen to receive a list of options that include Copy, Save, Blur background, and more. "At your request, Click to Do seamlessly connects you to tools and actions to accomplish tasks, like suggesting help with Visual Search with Bing, background blur or erasing objects in Photos, removing backgrounds with Paint, and more," Microsoft explained. "It also assists with text-related actions such as rewrite, summarize or explain text in line, opening in a text editor, sending an email (with a recognized email address), web searches, and opening websites (with a recognized URL)." Additional actions will be added in the coming months, Microsoft said. Microsoft demonstrated Click to Do "analyzing" a web page when a user used the appropriate click command. What's unclear is whether users will remember to actually use the "Copilot click" to do things, or just fall into old habits.

Windows 11's 2024 Update: 5 big changes I really like (and more)

Windows 11's annual update will be rolling out over the next few weeks. The big Windows 11 2024 Update (also known as Windows 11 24H2) is both a brand-new operating system but also one that's been out for several months now. It's Windows 12 -- but it's also not. And its best features are really reserved for those who have invested in a next-gen Copilot+ PC powered by chips from Qualcomm, Intel, and AMD. These seeming contradictions are at the heart of Windows 11 24H2, which begins rolling out today in a "phased" rollout that will last several weeks. But when you get it and what you get with it will all depend on whether you own a Copilot+ PC. In other words, there's a set of basic features that everyone will receive (including new energy-saving features for laptops and desktops, improved smartphone integration, plus support for Wi-Fi 7 and the upgraded 80Gbps capabilities of USB4), along with more advanced features that are only available to Copilot+ PC users. Microsoft describes the Windows 11 2024 Update as a "full operating system swap," which signals that it's more than just a minor update. Rather than exhaustively listing every new tweak, quirk, and detail, I'm just going to hit the biggest features and changes that I really like in Windows 11 24H2. Let's start with the best and move down. Phone Link has been one of the more interesting features in Windows that, for whatever reason, never quite caught on. But it's getting a killer new feature that could be disastrous for the webcam industry: the ability to use your phone's camera as a webcam. Under Settings > Bluetooth & devices > Mobile devices, you now have the option to Manage devices. You'll first have to set up a phone via Phone Link if you haven't already, but then you'll have the ability to do things like view the photos on your phone, respond directly to text messages, and more. (Phone Link is geared toward Android devices.) But you can now do two more things: get notifications when photos are taken, and use your phone as a "connected camera" (i.e., webcam). The latter option seems to work just fine, which means that instead of using a crappy 1080p webcam, you have access to the high-tech camera in your smartphone and all that it offers. You'll probably need some sort of a mount for your phone, though. Also worth noting is that Windows will record you in 720p at 30 FPS, or at least it did under my tests. By comparison, the native camera on my laptop recorded in 1080p at 30 FPS. My phone's camera was far better than the built-in webcam on my test Surface Laptop 7, but your mileage may vary depending on what you have. Support for a technology like Wi-Fi 7 requires coordination on all fronts: access points, controllers, radios, laptops, drivers, and operating systems. We've had Wi-Fi-7-capable hardware for months now, but without the operating system support necessary to enable it. I don't own any Wi-Fi-7-capable hardware yet, but see below for an example screenshot of what it should look like on your system. Wi-Fi 7 enables theoretical speeds up to 46Gbps, which should be enough for most people. Learn more in our Wi-Fi 7 deep dive. In January, Microsoft also began testing support for the upgraded 80Gbps version of USB4, sometimes known as USB4 Gen 2. Very similar to Thunderbolt 5 (of which we've already seen the first docks), USB4 should allow for high-speed I/O, even up to 120Gbps in some cases. You'll still need laptops and docking stations that support the new technology, however, and those are currently in short supply. Look for that to improve over time. Bluetooth Low Energy Audio is one of those "if you know, you know" technologies that I, thankfully, don't know very well yet because my ears are fine. But it's an important technology because some hearing aids and cochlear implants now ship with Bluetooth Low Energy Audio support, allowing them to stream music as well as aid in conversation. Windows 11 24H2 enables support for Bluetooth LE Audio on PCs, and it allows you to adjust audio presets and ambient sound volume (using settings determined by your audiologist) within Settings. For this, you'll need both a PC and hearing aids that support Bluetooth LE Audio. Fortunately, a 2022 executive order by President Biden now makes hearing aids available over the counter, and Bluetooth 5.3 includes the LE Audio specification. That means that devices like Apple's AirPods Pro 2 (which use Bluetooth 5.3) can be controlled by your PC. If you've followed our coverage of the Qualcomm Snapdragon X Elite and Intel Lunar Lake chips, you're probably giggling at the thought of even more battery life. We're already at 17 hours or so under load. Extending that even further might feel like overkill. However, the new Energy Saver control -- which dials down screen brightness by 30 percent, disables transparency effects, and throttles some background tasks -- can be set to operate while the PC is plugged in, making it useful for both laptops and desktops for saving power. It can turn on automatically, but you can also set it to always run, too. (Energy Saver doesn't let you alter performance settings while it's on, though, so don't think of it as a cool way to undervolt your PC.) While many desktops already feature a "quiet mode" or other alternatives for lowering power and performance, Energy Saver promises to provide overarching controls that accomplish much of the same. If you're like me and live in the Western US where "summer" can last through October, cutting power usage can save on energy and lower heat generation. You don't really need the full power of a desktop if you're just browsing the web or performing office work. One of the things I really disliked about the first Windows 11 builds was the right-click context menu, for two reasons: first, it extended into a second menu because it couldn't fit everything into the main menu, and second, Microsoft made the foolish decision to replace simple actions (like copy, paste, and rename) with icons. Now, three (!) years later, the icons finally have text labels to go along with them. Sure, I've learned what they mean by now -- and you probably have, too -- but for the millions of users who will be forced to migrate to Windows 11 once Windows 10 hits end-of-support next year, this is going to be an important change for usability. The right-click context menu still extends into the secondary "Show more options" menu, however. Oh well. Windows 11's 2024 Update removes some legacy apps. On a Surface Laptop 7, the Mail and Calendar apps came preinstalled but don't work. Instead, Microsoft is replacing them with the "new" Outlook app, a web app that simply isn't as functional as its predecessors. Microsoft hasn't provided any links to the older Mail or Calendar apps, but links (at least on the web) still remain to the "old" Outlook app, which provides a greater number of features and runs as a dedicated app. Cortana is gone (replaced by Copilot), and Movies & TV as well as Maps have been removed. WordPad has, too. (You can get WordPad back, however, and the replacement Windows Media Player is quite nice.) As a rule of thumb, any AI feature that includes "Copilot" in the name depends upon the cloud for processing. But there are several AI features that rely on your PC's NPU. In Paint, Microsoft has two AI tools: Microsoft Image Creator, the scaled-down version of Microsoft's Bing Image Creator/Microsoft Designer, and Cocreator, which works in conjunction with Paint. Image Creator uses a cloud-based generative AI art tool, though it requires an internet connection to ensure you aren't creating anything naughty. Cocreator involves providing a reference image, which can "tell" the AI art engine far more than a text prompt can. Microsoft Photos adds a similar feature called Restyle, which takes an existing photo and "restyles" it in some other art style. When you open Photos, Microsoft also alerts you that it's using AI for face detection to identify the people in your photos. The more practical uses of AI are new additions to Windows Studio Effects, which we saw first inside the Surface Pro 9 (5G). The latest additions include a couple of (largely ineffective) background blurring effects and new Eye Contact modes that make it seem like you're reading off a teleprompter. (The idea here is that you can move your eyes around but the subject will see an "AI" version of your eyes that focus on the camera or move back and forth like you're reading words. I doubt you'll notice a substantial difference.) Microsoft's Voice Clarity voice filtering was originally designed as an NPU-specific feature, too, but the company said in January that it refined the model enough that it can run on non-AI PC processors. Everyone will have access to it in Windows 11 24H2. AutoSR (Automatic Super Resolution), a nifty little way to improve your PC's graphics, is really an unknown for me at this point. We've largely concluded that the first generation of Snapdragon X PCs aren't suited for games, but they're the only ones to include both an NPU and Windows 11 24H2 to play games with. AutoSR does add AI-generated frames, but the real test will probably be the first Copilot+ PCs from AMD and Intel, where those frames will make a substantive difference. AutoSR does seem to work, however. In a test of Cyberpunk 2077 -- a game that's largely out of reach of the Surface Laptop's Adreno GPU -- simply enabling AutoSR raised frame rates from 26 FPS to 32 FPS, an improvement of 23 percent. (I tested at 1080p, Low settings, without any in-game frame generation options on.) A frame rate of 32 FPS isn't really playable, but the performance improvement seems real. That's not to diminish AutoSR, though. AutoSR is designed to improve existing games by first downscaling them to decrease resolution (and improve the frame rate) and then sharpening those frames to improve the visuals. It all runs on the NPU, with the game blissfully "unaware" of what's being done to it. AutoSR could very well be significant for laptop gaming; we just can't see all of the benefits yet. Oh, and Recall? You'll see Microsoft's controversial AI-powered search feature eventually, but not after a period of testing. If you don't want it, that's fine, too! Microsoft now says that you can refuse to install Recall or uninstall it later after trying it out. Microsoft has also added some minor improvements to the Windows 11 2024 Update. They include:

A huge Windows 11 2024 update dropped today: 5 new AI-powered features, including Recall

Microsoft is rolling out the new Windows 11 2024 update today -- and it's arguably the most highly anticipated version of the OS this year. Why? Among a slew of other new features, it's expected to come with the new-and-improved (and controversial) Recall feature that attracted scrutiny from security experts. However, as we reported on Friday, the Redmond-based tech giant announced new privacy-focused protocols to ease concerns about the up-and-coming AI-powered feature. Curious about what Microsoft has up its sleeves with the new Windows 11 2024 update (also known as Windows 11 24H2)? Stick around. As mentioned, one major feature coming to the next big update is Recall. For the uninitiated, Recall is an AI-powered feature that keep tabs on your PC activities by consistently taking screenshots. It's designed to help the forgetful among us rediscover lost webpages, or anything else we've done in our past, using a timeline scrubber and/or AI-powered natural language search. On Friday, Microsoft quieted concerns about security by announcing that Recall will be secured with Windows Hello (a security feature that lets you access Recall with fingerprint ID, facial recognition, and other credentials). Plus, Microsoft promised encryption, the option to disable and uninstall, and more. Click-to-do, after hitting the Windows Key and clicking your trackpad or mouse, places an an interactive overlay on top of your PC screen. This will give you easy access to the following: This list is non-exhaustive, though, so there's more to explore. Click-to-do will also help users with text-related functions, including rewriting, summarizing, and explaining in-line text. Windows Search has gotten smarter. According to Microsoft, you no longer need to worry about remembering file names. You can simply type what's on your mind and Windows 11 will do its best to find the file for you. For example, if you have some pictures of you having a blast at a July 4th gathering, you can type "BBQ party," and Windows 11 will find what you're looking for. This new-and-improved search functionality will come to File Explorer first before expanding to Windows Search and Settings. Have you ever looked an a low-resolution photo and thought, "Gee, I wish I could just make this look sharper and clearer?" Well, with the new Super Resolution feature in the Photos app, you can do just that. Thanks to on-device AI, Super Resolution can upscale your photos up to 4K in seconds -- and you don't need to spend a dime to make it happen. In late June, we showed off how Microsoft Paint now lets you use text prompts to help you create stunning AI-generated masterpieces in the rudimentary graphics program. With the Windows 11 2024 update, Microsoft has improved the model that underpins Paint to deliver faster results. Plus, with new generative fill and erase tools, the Microsoft boasts that you can now use an adjustable brush to remove "unwanted or distracting elements" in your image. You can also add new elements exactly where you want them. Other new features include Energy Saver (extends battery life even when your PC is plugged in), improved hearing aid support with Bluetooth LE Audio, and Wi-Fi 7 compatibility. This update, also called Windows 11 24H2, will start rolling out today. Windows Insiders will get to experience the features first before a wider rollout in November. To be among the first to experience the new features, navigate to Settings > Windows Update and turn on "Get the latest updates as soon as they're available." (Eligible devices must be running Windows 11 version 22H2 or 23H2.)