Microsoft Introduces AI-Powered Gaming Copilot to Windows Game Bar

Microsoft is bringing its Xbox Copilot to the Windows Game Bar

Microsoft started testing its Xbox Copilot on iOS and Android earlier this year, and now it's bringing the chatbot to the Game Bar feature on Windows 11. Xbox Insiders can start testing a preview of the Gaming Copilot integration in the Game Bar today, and Microsoft is planning "further optimizations" for this new feature on the upcoming Xbox Ally handhelds. "Gaming Copilot knows what you're playing and understands your Xbox activity to provide an experience centered around you," says Taylor O'Malley, principal program manager at Xbox. You'll be able to ask the Copilot for help with the game you're playing, as well as access achievements and play history. The Gaming Copilot will be available as a widget inside the Game Bar, and it includes a voice mode so you can talk to the AI assistant without having to alt + tab out of a game. The Gaming Copilot will even use screenshots of your game to help answer questions about it, without you having to describe things in detail. That could be as simple as asking "what am I looking at?" while facing a boss in a game, and Copilot should respond with details about the boss and perhaps even methods to take it down. This is the first step in Microsoft's vision to turn its Xbox Copilot into an AI gaming coach. Microsoft is positioning the Gaming Copilot as a sidekick, one that will accompany you through games and offer tips and information about game worlds. Microsoft has previously demonstrated using the Gaming Copilot with Overwatch 2, letting it recommend good hero picks to complement the rest of the team. This early preview is only available in English and limited to the US, Australia, New Zealand, Japan, Singapore, and a few other regions. Microsoft isn't making it available in the UK or any other European markets yet. It's planning to release the Gaming Copilot in additional regions at a later date, but it's not clear yet when exactly that will happen.

Microsoft brings AI-powered assistance to gaming -- Copilot Gaming (beta) and Copilot 3D creative features went live this week

Microsoft has been pushing its Copilot AI technology to more users, with the company releasing features for both gamers and creatives in recent days. According to the Xbox Wire, Gaming Copilot is now available in Beta to Xbox Insiders on Windows PC, with the feature also rolling out to handhelds, but with limited capabilities. Redmond introduced Copilot for Gaming to the Xbox mobile app earlier this year, and it's now expanding its capabilities to the Game Bar. The AI gaming assistant knows what you're playing and understands what's happening on your screen, and it now also gets a Voice Mode, so you can ask it for tips and tricks if you're stuck at a particular level. If you want to try out its features, you need to be an Xbox Insider enrolled in the PC Gaming Preview. You must also be at least 18 years old and located in the U.S., Australia, New Zealand, Japan, Singapore, and a few other regions. Those who are already enrolled can try it out by pressing Windows + G to open the Game Bar and then clicking on the Gaming Copilot icon in the Home Bar to open the widget, where you will be asked to log into your Xbox account. Creatives are also getting the new Copilot 3D feature in Copilot Labs. This easily converts a flat 2D image into a 3D model, so you don't have to spend hours and hours building a custom 3D file. More importantly, it's free and is available to anyone with a Microsoft Account across the globe. Copilot 3D isn't just a novelty tool -- you can use it to build models for 3D printing, game design, art projects, and more. It uses the popular GLB format for improved compatibility and saves your creations for 28 days so you can go back to your recent creations for later use. These tools show how Microsoft is trying to make its AI tools useful for both work and play. However, at the time of writing it is unclear whether these features will only work with NPU-equipped devices, or if they rely on the internet so any PC (or Mac) can run them.

Copilot will now tell you how to "get good" when you die in Dark Souls for the 47th time

* Microsoft is using Copilot, an AI agent, to assist gamers with achieving objectives in-game. * Copilot will be accessible from the Game Bar and can be used in voice mode to provide tips and guidance. * Copilot can analyze screenshots of gameplay to offer explanations and help gamers progress through challenging levels. You know, back in my day, when you wanted to know how to beat a level or craft a specific item in a game, you had to get on your bike, go to an internet cafe, and print out 30 pages of a GameFAQs page that re-designed the game's logo in ASCII art. Now you turn on Twitch and stream your gameplay, and you have hordes of people telling you exactly what you should do in varying degrees of rage. Well, Microsoft is adding a new way to tell you what you need to do next. It uses an AI agent to take a look at what you're doing in-game, listens to what you want to achieve, and then looks through its database to find your solution. And you're never going to guess what AI agent Microsoft is using to do this. Well...yeah, okay, it's Copilot. You got it. Microsoft adds Copilot-based gaming help to the Game Bar for Insiders In a post on the Xbox Wire, the company explains how Copilot can help you game better. The gist is that Copilot will live inside the Game Bar, and you can invoke it when you're feeling stuck. If you already use the Game Bar to do stuff in your games, it should just slide on in without any issue. Of course, having to bring up the Game Bar so you can quickly type how to kill the huge demon boss making its way toward you is unideal. Fortunately, you don't need a keyboard to use it: Pin the Gaming Copilot widget and use Copilot in Voice Mode while you're actively playing any game on PC. This lets you talk to Copilot and get help if you're stuck or need tips, without breaking away from gameplay. To do this, navigate to the Gaming Copilot widget, select the "Microphone" from the left navigation bar, begin a conversation, pin the widget, and continue the conversation seamlessly and at any time while you return to your gameplay. Microsoft also gives Copilot the ability to look at screenshots of your game. If you're looking at something weird and you want more context, pass an image over to Copilot, ask it what it is, and it'll do its best to explain it. Humorously, the example Microsoft gives for this feature (and the same one used in the image at the very top of the article) involves someone using Copilot to identify one of the Kraken's tentacles during a Sea of Thieves session. Let's hope Copilot can respond quickly enough to save you from being snagged by one. This new feature isn't ready for release just yet, but if you join the Xbox Insiders Program, you should gain access to the beta. And if you like the sound of mixing the power of AI with the bustling gaming scene, be sure to catch up on what Nvidia is doing with G-Assist.

You can now try Microsoft's Gaming Copilot AI assistant on PC

Microsoft revealed that it was specifically for gamers back in March, and now it's here for a test drive. The beta version of Gaming Copilot is available to Xbox Insiders on PC via the Game Bar functionality. It's also available to try on Windows-based handheld gaming devices, but the company says the functionality there is currently limited. A variant of this tool recently launched . This is kind of like an AI version of those old Nintendo phone help lines. The chat box appears as an overlay on the screen and players can use it to ask questions about the game they're playing. Microsoft says this tool has been designed to help players traverse obstacles and that it "knows what you're playing and understands your Xbox activity." The system uses in-game screenshots to make sure it's providing useful advice. With that in mind, Gaming Copilot can also offer suggestions about new games to buy and answer questions about system accounts. It allows for text-based and voice-based queries, assuming a microphone is connected. Microsoft says it'll be adding more features in the future, including "richer game assistance such as proactive coaching." The beta build of Gaming Copilot is only available in English at the moment, though it can be used in various regions including the US, New Zealand, Japan and Singapore.

The Windows Game Bar Is Getting a New Feature in Beta

Microsoft is bringing its Xbox Copilot to the Windows Game Bar, so you can get help with your games without ever having to leave the screen. This is a huge deal for anyone who gets stuck in a game and doesn't want to break their focus by searching for a walkthrough on Google. The new "Gaming Copilot" is a chatbot that is a lot like an assistant, and it's being tested by Xbox Insiders in the PC Gaming Preview right now. Microsoft is also planning to optimize this feature for the upcoming ROG Xbox Ally handhelds. This compatibility is also coming to the mobile app, but Microsoft is also moving it to the PC. It's a good idea to have this in the game bar itself and not just out of convenience. Since Copilot knows what game you're playing and what your Xbox activity is, it can give you an experience that's completely centered around you without looking away from the screen. This means you can ask it for help with the game you're currently playing, and it can also give you access to your achievements and play history. The Gaming Copilot will be a widget inside the Game Bar, and it even has a voice mode so you can talk to the AI assistant without having to leave the game. One of the best features is that the Gaming Copilot can use screenshots of your game to help answer your questions without you having to describe things in detail. This means, theoretically, you should be able to see a quest in the game, ask Copilot about it, and it should give you a detailed walkthrough. Based on Microsoft's images, the same could be done for bosses and enemies. This is the first step in Microsoft's plan to turn its Xbox Copilot into a full-on AI gaming coach. The Gaming Copilot is meant to help you get to the gameplay faster, sharpen your skills, and be there when you need them. In the future, Microsoft wants to add even more features, like proactive coaching and a deeper level of personalization. However, that should be an on/off feature and should keep spoilers in any form optional. For now, you can already use it to get help with your account, play history, and achievements, along with a few other things. You can even pin the widget and use the voice mode to get help while you're in the middle of a game. To use the Game Bar, you just need to make sure you have the Xbox PC app installed on your Windows device and be a part of the insiders program. Then, all you have to do is press the Windows logo key + G to open the Game Bar on your game or desktop. Once it's open, you just need to look for the Gaming Copilot icon and log into your Xbox account to get the full experience. This early preview of the Gaming Copilot is only available in English and is currently limited to the US, Australia, New Zealand, Japan, Singapore, and a few other regions. Microsoft hasn't made it available in the UK or any other European markets just yet, but the company does have plans to release it in more regions at a later date. Source: Xbox, The Verge

Microsoft is testing a Gaming Copilot AI on the Windows Game Bar

Stuck on a boss? Don't know how to "perfect parry?" Microsoft is adding its AI assistant, Copilot, to help you with your games via an addition to the Windows Game Bar application. Right now, what Microsoft is calling "Gaming Copilot (Beta)" will show up only for those enrolled in previews for Xbox Insiders on the PC platform. Microsoft has issued a similar update for handheld PCs, too. Microsoft says that Gaming Copilot understands what game you're playing, which makes sense. A Voice Mode allows you to open an oral conversation via the "microphone" widget and then pin the widget to your screen as you play. You can also snap a screenshot of your screen and then let Gaming Copilot "look" at it for additional help. To use the new Copilot-equipped Game Bar, you'll need to be enrolled in the program, launch Game Bar (the Windows key+G) and then look for the Gaming Copilot logo on the home bar to open the widget, Microsoft says. It's available for Xbox Insiders in the United States, but only if you're 18 or older. Microsoft doesn't explain the last requirement. Oddly, Microsoft already has a technology to "see" your entire screen and help you out: Copilot Vision, which Microsoft says is out of beta and is available for all PCs in the United States. But Copilot Vision's vision was pretty lousy when I tested it on games like Solitaire, let alone quick first-person games. Microsoft may have improved its "vision" capabilities with the new Copilot Game Bar, but I wouldn't expect it to be as helpful as an online guide or just a nearby buddy. But with the Copilot application cropping up everywhere -- within Windows, the Edge browser, even applications like Microsoft Excel -- expect Copilot's march to continue into pretty much every PC application that Microsoft owns.

Microsoft's Gaming Copilot AI is now in Windows 11's Game Bar - and I can't decide whether this is a good or bad thing

Gaming handheld owners may worry about performance, but Microsoft says it's optimizing the feature for these devices As Microsoft promised earlier this year, Gaming Copilot - the AI helper for gamers - is now on PCs (after previously debuting in testing on smartphones in the Xbox mobile app). The AI understands what game you're currently playing, and is designed to help you when you get stuck in a game, and to do so seamlessly - in the Game Bar overlay - without having to switch away to a web browser window to look for tips and so forth. Microsoft also points out that for added convenience, there's a Voice Mode, so - assuming you have a microphone - you can talk to Copilot to ask for help. The AI will also use screenshots of the game that's running to understand what you're asking about, and hopefully provide more relevant info and guidance as a result. As noted, this feature is now available on the Game Bar, but only to those who are in the Xbox Insider program - you'll also need to ensure the Xbox app on your PC is updated to the latest version. Note that Microsoft has only deployed this feature to the US and certain other countries, including Australia, New Zealand, Japan, and Singapore. Notably, the UK and Europe aren't getting Gaming Copilot yet, which is unusual and likely bound up in data regulations. This looks to be a useful feature, despite the cries of 'no, not more AI jammed into Windows 11'. I get that sentiment, but in fairness, if you're not interested - or worried about privacy in some way (with the AI 'watching' your gaming) - just don't use Gaming Copilot. As to how useful this AI assistant will be, I haven't tried it personally, but it does look like a neat time-saver. There is a worry when it comes to Windows 11 gaming handhelds, namely that it's going to hit performance (and battery life). After all, this is an extra task going on in-game (which is the mentioned convenience) - so how might that affect frame rates? Microsoft acknowledges this concern in its blog post and notes that right now, the feature is limited on handhelds. The company advises: "While Gaming Copilot (Beta) is also available now for Xbox Insiders to try on Windows handhelds with limited functionality, further optimizations are underway for handhelds as we approach the launch of the ROG Xbox Ally and ROG Xbox Ally X." Of course, if you have your smartphone handy, you can always use Gaming Copilot on that second screen - via the Xbox mobile app - and swerve any performance concerns (albeit that's hardly a seamless way of working). Perhaps there's another potential worry here, too, and that's related to the convenience of having hints and help on tap in such an easy fashion. Are AI-powered abilities like this going to encourage more laziness in gaming, with people reaching for help after only being stuck for a very short time, rather than trying to figure things out themselves? I guess that's a broader, more philosophical concern about gaming, and the choice to enlist the help of AI seamlessly, at the drop of a hat, could have more of an impact than we might imagine in the future. Although the bigger worry will doubtless be with AI snaking its helpful tentacles into competitive games, and Microsoft is already talking about how Gaming Copilot will offer 'proactive coaching' and 'richer game assistance' in general.

Microsoft's beta Gaming Copilot AI is now available for Xbox Insiders, to 'help you get to gameplay faster, sharpen your skills, and be there when you need it and out of the way when you don't'

I'm going to date myself here, but I do have a nostalgic fondness for ye olde game guides of yore, ASCII art and all. Though I'll never forget those community-based efforts, I am glad guide writing has become a more dedicated job role -- especially as I continue to have my arse handed to me in Elden Ring: Nightreign. Alas, Microsoft reckons that its Gaming Copilot AI can do it better. According to a recent Xbox Wire blog post, Gaming Copilot aims to make "gaming with Xbox more seamless and personalized, helping you get to gameplay faster, sharpening your skills, and being there when you need it and out of the way when you don't." Announced back in March as Copilot for Gaming, it's essentially an AI chatbot attempting to answer all of your gameplay questions, from puzzle solutions to battle strategy. Gee, I wonder where the AI is pulling all of that insight from... A Beta version of Gaming Copilot is now available on Game Bar for Windows PC, but only for a select few. Xbox Insiders enrolled in the PC Gaming Preview based in "the U.S., Australia, New Zealand, Japan, Singapore, and other regions" can noodle with Gaming Copilot in English right now. Notably, the UK is missing from the full list of territories, likely due to local AI regulations. This Beta follows an earlier test that ran via the Xbox mobile app, allowing gamers to "access Copilot on a second screen without distracting you from your core gameplay experience." This latest rollout sees Gaming Copilot now "directly embedded into the PC gaming environment." That means you can either type out a question in a chat window overlaying your gameplay, or use "Voice Mode." Personally, I'd rather continue quietly typing out my gaming fumbles into a search engine, and not have to voice out loud, "How do I stop funnelling myself into dead ends in Blue Prince?" Anyway, apparently, Xbox is quite proud of this voice mode functionality. When you voice a question, Gaming Copilot "can now use screenshots of your gameplay to provide you with more helpful responses." This means that rather than describing your problem in detail, the AI can look for context clues in your gameplay to offer "more relevant and timely support." For me, I'd also rather the AI didn't see just how many times it took me to beat Nightreign's Everdark Caligo (which is still a bit of a work in progress, to be honest, but I'm thankful for Sean's guide pointing me in the right direction). Xbox has also set their sights on bringing Gaming Copilot to a wider range of devices beyond PC and mobile, with work "underway for handhelds as we approach the launch of the ROG Xbox Ally and ROG Xbox Ally X." Personally, I'm less than eager to welcome Gaming Copilot onto any of my devices. For one thing, an AI is simply never going to know a game like a real-life guide writer will. Furthermore, I can't help but wonder whether Gaming Copilot will simply make stuff up just because its LLM foundation dictates that it 'sounds' right. After all, we've already seen ChatGPT say the Soundslice app had a feature that it definitely didn't, so what's stopping Gaming Copilot from leading players on an in-game goose chase for nonexistent characters, items, or even mechanics? As I'm based in the UK, I've yet to interrogate Gaming Copilot for myself, but I find it hard to believe it will ever have all of the answers.

Gaming Copilot is now rolling out to Xbox Insiders on PC, coming soon to ROG Xbox Ally

TL;DR: Microsoft's Gaming Copilot is an AI-powered Xbox Game Bar assistant for Windows PC, offering personalized, real-time gaming support through voice interaction. Currently in beta for Xbox Insiders, it leverages gameplay data to provide tips, coaching, and seamless assistance, enhancing player experience ahead of upcoming handheld launches. Microsoft's Gaming Copilot is described as the "ultimate gaming sidekick," an AI assistant designed to help you out when you get stuck in a game as it "knows what you're playing and understands your Xbox activity." Gaming Copilot (Beta) is now available as part of the Game Bar for Windows PC for Xbox Insiders, with Microsoft confirming that optimizations are on the way for the upcoming launch of the ROG Xbox Ally and ROG Xbox Ally X handhelds. "This companion makes gaming with Xbox more seamless and personalized, helping you get to gameplay faster, sharpening your skills, and being there when you need it and out of the way when you don't," Xbox's Taylor O'Malley writes. Gaming Copilot appears as one of the widgets on the Xbox Game Bar, and it fully supports voice interaction with a dedicated Voice Mode. Apparently, this will let you engage and talk to the Gaming Copilot AI assistant while you're playing a game, so you'll be able to get tips and hints "without breaking away from gameplay." The Gaming Copilot (Beta) AI assistant has access to your full Xbox account details, play history, and achievements. And with real-time access to what you're currently playing, it can even take screenshots of your gameplay to demonstrate something or provide a more "helpful" response. Capture settings can be adjusted in the settings. Still, it's very interesting to learn that Gaming Copilot can now see what you're doing in-game and be able to use that information to answer questions or offer advice. Ahead of today's launch, Gaming Copilot has only been made available in a beta version of the Xbox mobile app with a simple text-based interface that acted more like your traditional AI chatbot - with the added bonus of it knowing what you're currently playing. The Xbox team notes that future Gaming Copilot updates will enhance the personalization of the tool, offer "richer game assistance" that includes coaching for competitive games, and more. "We will continue to explore new ideas, experiment, and bring new features like Gaming Copilot to you early and often," Taylor O'Malley adds. "Feedback from Xbox Insiders will be critical in helping us shape and improve the feature as it expands to more Xbox surfaces."

Copilot for Gaming beta brings AI tips to Windows players

Microsoft has released Copilot Gaming (beta) and Copilot 3D creative features this week, providing AI-powered assistance to Windows PC gamers and global creatives. These tools leverage Microsoft's Copilot AI technology to enhance user experience in gaming and 3D modeling. Copilot for Gaming is now accessible in Beta for Xbox Insiders using Windows PCs. This feature is also being rolled out to handheld devices, albeit with certain limitations. Microsoft initially introduced Copilot for Gaming to the Xbox mobile app earlier this year. Its functionalities are now expanding to the Game Bar, where the AI assistant can identify the game being played and comprehend on-screen events. A new Voice Mode has been integrated, enabling users to request tips and strategies when facing challenges at specific game levels. To access Copilot for Gaming, individuals must be Xbox Insiders enrolled in the PC Gaming Preview. Users are required to be at least 18 years old and reside in the U.S., Australia, New Zealand, Japan, Singapore, or other specified regions. Enrolled participants can activate the feature by pressing Windows + G to open the Game Bar. They then select the Gaming Copilot icon located in the Home Bar to open the widget, prompting a login to their Xbox account. For creatives, Microsoft has introduced Copilot 3D within Copilot Labs. This feature facilitates the conversion of a two-dimensional image into a three-dimensional model, eliminating the need for extensive manual file construction. Copilot 3D is available without charge to any user possessing a Microsoft Account, with global accessibility. This tool supports various applications, including the creation of models for 3D printing, integration into game design, and utilization in art projects. It employs the GLB format, which enhances compatibility, and stores user-generated creations for 28 days, allowing for subsequent access and utilization. The operational requirements for these new AI features remain undefined. It is not currently clear whether these features necessitate devices equipped with Neural Processing Units (NPUs) for local processing or if they rely on internet connectivity, thereby allowing any PC or Mac to run them.