Meta Quest 3 to Receive AI Upgrades, Challenging Apple's Vision Pro

Meta Quest 3 set to receive Meta AI Vision and Advanced Chatbot features

Meta AI on Quest will possess the ability to search for information online. Meta has officially announced that Meta Quest 3 will be merging with their artificial intelligence (AI) chatbot, Meta AI. This integration will make the chatbot the designated AI assistant for the mixed-reality headset, capable of responding to spoken inquiries. Also Read: iPhone 16 models to feature internal upgrades for better heat management Similar to the social media applications developed by the company, Meta AI on Quest will possess the ability to search for information online. Furthermore, it will be equipped with computer vision features to provide answers regarding the user's environment. It is worth mentioning that Meta AI was previously incorporated into the Ray-Ban Meta smart glasses back in April. The company has announced in a blog post that Meta AI will be introduced to Quest 3 next month. Initially, the AI assistant will only be available in English in the US and Canada. This integration will be included as part of the Meta Horizon OS. Following this update, users will have a hands-free interaction with the AI chatbot. Additionally, Meta Quest 3 will be equipped with Meta AI vision capabilities, allowing the mixed-reality headset to utilize passthrough to observe and analyze the user's surroundings, enabling it to respond to related queries. For example, users can request the AI to identify a flower or a famous landmark in their vicinity, reports Gadgets 360. Additionally, users have the option to pose more intricate inquiries, such as presenting a t-shirt in front of the headset and inquiring about what would complement the outfit. Meta AI vision is set to be introduced in an experimental capacity next month, as stated by the company. It is important to note that the AI assistant is limited to recognizing physical objects visible through passthrough and cannot identify virtual objects. Although the company specifically mentioned Quest 3, the functionality may also extend to Quest Pro due to its passthrough capabilities. However, confirmation will only be possible once Meta officially confirms the support. While Quest 2 will not receive the vision feature, the Meta AI assistant will still be accessible on the device. It is worth mentioning that Quest is not the initial wearable device from the company to incorporate the hands-free Meta AI integration. In April, the company introduced Meta AI with vision capabilities to the Ray-Ban Meta smart glasses in specific markets. Also Read: iPhone 16 models to feature internal upgrades for better heat management

Meta Quest 3 beats Apple Vision Pro as first mixed-reality headset to adopt AI

The already formidable Quest 3 mixed-reality headset is set to reach all-new heights. Yesterday, Meta announced plans to bring its 'supercharged' Meta AI assistant to the device in August -- racing ahead of Bloomberg reports that Apple plans to bring Apple Intelligence to Vision Pro headsets in the near future. First unveiled in Sept. 2023, Meta AI has been a core pillar in the company's push to bring AI into its services and hardware over the last ten months. The multimodal large language model (LLM) is similar in scope to OpenAI's ChatGPT, capable of performing natural language interactions with users across text, voice, and more recently visual inputs. In a nutshell, Meta AI can understand and reply to anything it reads, sees, or hears, and has already begun to feature in services like WhatsApp, Instagram, Messenger, Facebook, the Meta.AI web portal, and within the company's Ray-Ban Meta Smart Glasses. In a post to the Meta Quest blog, Meta revealed that its AI assistant's next destination would be Quest 3 headsets -- making use of the platform's mixed-reality abilities to showcase its multimodal capabilities, especially that of translating and understanding real-time video and visuals. Meta AI will begin rolling out to Quest 3 headsets next month, with the assistant available to users in the US and Canada first. To access Meta AI, you'll need to enable it in the Horizon OS Experimental panel, found within the main Settings panel. Initially, users will be able to interact with Meta AI through voice prompts, where you can quiz the AI on any topic you're interested in, or even ask for more contextual information about places to eat or visit nearby, as well as using your visual surroundings as part of prompts. However, Meta states that it will be seeking to improve the experience over time, which could lead to an explosion in use cases for its AI assistant. While not everyone is keen to see Meta's AI pop-up in their messaging and social media platforms, there's a boatload of untapped potential in seeing the assistant interact with the world around you. In one video shared by Meta, the Quest 3's upcoming AI assistant recommends nearby spots to play basketball, another shows it passing out wardrobe advice based on an item of clothing it can see. However, the potential applications of Meta AI within the Quest headset could stretch far beyond these more advanced Siri/Alexa-like interactions -- with the AI gaining an entirely new arsenal of potential tools and features to make use of. Let's look at some of the ways we'd love to see Meta AI use these tools to transform the Quest experience. After seeing the videos accompanying yesterday's announcement, it's hard not to see a vertical slice of Meta's vision of the future when it comes to the merging of wearable tech and AR/AI. The company is dead-set on its VR/AR ambitions, and a recent sighting of a strange pair of smart glasses in a months old Mark Zuckerberg post to Threads has ignited rumors that the company is preparing to showcase its long-rumored 'true' AR glasses later this year. Could this new Quest experience be an early example of what Meta hope to achieve in the smart glasses market over the coming years? It's highly likely. However, while CEO Mark Zuckerberg has confirmed that there'll be news to share about future frames this year, Meta's AR smartglasses are still said to be in the prototype stage.

Meta Quest 3 is getting AI before Apple's Vision Pro does - here's how to enable it

Also: Meta Quest 3 review: The VR headset most people should buy in 2024 Soon, Meta's announcement says, "You'll be able to get answers to a wide range of questions with real-time information thanks to Bing." In addition, Meta AI with Vision will be part of the update. This means you'll be able to ask questions about your surroundings while wearing the headset. With Meta AI, your Quest 3 headset can see people, objects, and more and -- give more information about them, much like Meta's Ray-Ban glasses. You'll be able to look at a sign and have it translated, get fashion tips, find info on a landmark or portrait, and more. The Quest 2 will not be getting the feature because it lacks color passthrough. In a demo video, a Quest 3 user holds up a pair of black-and-white patterned shorts and asks Meta AI for help finding a matching t-shirt. Meta suggests a black, white, gray, or pastel-colored shirt. If you're watching a YouTube video to plan a vacation, for example, you might ask Meta how to dress for a vacation to that area or for nearby restaurant recommendations. If you're listening to music while completing school work on a virtual monitor, Meta says, you might ask Meta for some memorable Shakespeare quotes and an explanation of their significance. Also: Stanford's VR breakthrough could spell the end of clunky headsets - thanks to AI Vision will only work in passthrough mode for the experimental version of Meta AI. Virtual objects will not be recognized. Meta AI on the Quest 3 will start rolling out next month in the US and Canada in experimental mode. When it does roll out, you'll need to enable it. To do this, head to your settings menu, then "Experimental" and toggle it on.

Meta Quest Headsets Are Getting AI Next Month

Meta is beating Apple to AI in VR, although the release will be limited at first. Meta is skipping ahead of Apple at implementing AI in VR. While Bloomberg's Mark Gurman previously reported that Apple Vision Pro will get Apple Intelligence sometime after this year, Meta's Quest headsets will get AI next month. Based on the Meta AI Assistant already baked into Ray-Ban smart glasses, Meta AI with Vision on Meta Quest will allow users to ask Meta questions based on what they can see with the full-color passthrough cameras on their device. That means it will be limited to Quest 3 and Quest Pro devices at first, although Meta did hint the AI might eventually also be able to detect objects seen on the Quest 2's more basic sensors as well. Meta says users could ask the AI to help them complete an outfit just by looking at it, and released a video where the headset was able to identify a dog even as it was running in and out of frame. The AI can also answer more general, non-vision based questions, like suggesting good restaurants nearby or helping bring up the most famous quotes from Hamlet. Meta AI can additionally use Bing to search live data from the internet, so it's not restricted to its training data. Meta AI with Vision relies on multimodal technology, meaning it can accept inputs from multiple mediums and translate content between them. The company recently said it would not be releasing its next multimodal model in the EU, so the Quest's AI features will only be available in Canada and the United States first. They'll also only support English to begin with. Despite the limited initial release, the company is going all in on the AI assistant, saying "Meta AI will replace the current Voice Commands on Quest." This means it will also be able to help you control your headset, although to what extent has yet to be clarified. Meta AI will launch on Quest in experimental mode first. To try it out once it's ready, open the settings menu on your headset and toggle it on under the "Experimental" submenu.

Meta to give its Quest VR headsets AI powers in bid to one-up Apple

Meta AI will arrive on Canadian and American Quest VR headsets in August, Meta has announced. The company said in blog post that the Meta Quest 3 owners in North America will get access to the Meta AI in beta form. Meta claimed that this initiative to add Meta AI in Quest VR can allow you to get access to Meta's 'intelligent scanning' of your physical surroundings. Meta AI coming to Quest VR Meta explained that "These changes will begin rolling out in the US and Canada in experimental mode next month -- and we'll continue to improve the experience over time." It further added that the older Meta Quest 2 will also be receiving this AI update. However, the more advanced Meta AI with Vision capabilities will remain exclusive to newer hardware. It is expected that as part of the Meta Horizon OS, Meta AI will let you go beyond the hands-free experience you're used to. With the update, soon you'll be able to get answers to a wide range of questions with real-time information. All thanks to Bing search engine. Meta explained that the collaboration of Meta AI with Vision will be part of the new update (available in Passthrough to start). The proposed update will allow you to ask questions about your physical surroundings while in-headset. For example, you're watching YouTube videos of some cooking in mixed reality while getting ready for your day out. With the new update apart from watching video you can also ask Meta AI for advice on how to best dress, simultaneously. The way ahead The addition of Meta AI in Meta Quest 3 can be associated with Ray-Ban Meta smart glasses. As artificial intelligence (AI) continues to make its mark on wearable devices, Meta plans to bring more AI in its consumer hardware sector. Meta also highlighted that AI has been a driving force of their consumer hardware since the beginning. In addition to this the recent innovation in generative AI has helped it to accelerate key pieces of their vision for the next great wave of human-oriented computing. Furthermore, the company also shared plans of improving the existing 'voice command' feature of Quest headsets. However, the dates are not disclosed yet. Other big tech companies such as Microsoft are also betting big on Copilot, Google is pushing forward with Gemini, and even Apple recently announced its own brand of AI, named Apple Intelligence.

Meta's Quest 3 headsets to get Llama-powered AI feature

Meta announced that its Quest 3 headset will get the company's Llama-powered Meta AI update. This means eligible users with the Meta Quest 3 headset will be able to carry out activities such as looking at virtual screens within the headset while also getting responses about objects in their offline view. "As part of the Meta Horizon OS, Meta AI will let you go beyond the hands-free experience you're used to. Soon you'll be able to get answers to a wide range of questions with real-time information thanks to Bing. Meta AI with Vision will be part of the update (available in Passthrough to start), so you'll be able to ask questions about your physical surroundings while in-headset," said Meta in a blog post. (Unravel the complexities of our digital world on The Interface podcast, where business leaders and scientists share insights that shape tomorrow's innovation. The Interface is also available on YouTube, Apple Podcasts and Spotify.) AI is learning from what you said on Reddit, Stack Overflow or Facebook. Are you OK with that? The company suggested that the headset could help answer questions, put together outfits, provide weather forecasts, or even serve as a teaching aid. It promised more functions in the future. Meta's Quest 2 is not covered by the update, and virtual objects cannot be recognised by Meta AI yet. The social media company's Meta AI offering is available via Facebook, Instagram, WhatsApp, Messenger, the web, and the Ray-Ban Meta smart glasses. The Meta AI headset update will come to users in the U.S. and Canada in an experimental mode next month and the experience will be improved over time. The feature is only available in English for now. The development also raises questions about when Apple will definitively bring its own Apple Intelligence AI offering to the Vision Pro headset. Copy link Email Facebook Twitter Telegram LinkedIn WhatsApp Reddit READ LATER Remove SEE ALL PRINT

AI just came to VR in a big way | Digital Trends

Meta just announced a huge update to its AI and it's coming soon to the Meta Quest platform, home to some of the best VR headsets you can buy. In the next few weeks, you'll soon have access to a smart voice assistant anytime you wear your Quest 3, Quest Pro, or Quest 2. You can see some examples of how Meta AI works on the Quest 3 in the video below. Note that Meta AI can use the Quest 3's and Quest Pro's mixed reality mode to see real-world objects and answer your questions. The Quest 2's black-and-white passthrough camera isn't supported for visual input. Meta Quest VR headsets in the U.S. and Canada will start getting these new multimodal AI capabilities as early as August. It will be a gradual rollout, so you might not see the update until September. Recommended Videos I asked Meta about generating AI images in Quest headsets and whether the AI would be able to see my Meta avatar and virtual environment. I'll update this article when I hear back. Meta AI has been in Facebook, Messenger, and WhatsApp since September 2023, helping you with customized AIs for cooking, jokes, and games. It can even generate images when you use the @MetaAI /imagine command while chatting with a friend. Meta AI also supercharges Ray-Ban Meta Smart Glasses, giving you cutting-edge AI assistance wherever you go. The upcoming Meta Quest update will bring a version of AI that's similar to what's available in Meta smart glasses. You can ask general questions like what to wear in the typically hot weather of Palm Springs and even hold up a pair of shorts and ask Meta's AI to suggest a matching top. This announcement came at the same time Meta announced its 405B parameter Llama 3.1, the largest AI ever released as open source. Meta also posted AI benchmarks that indicate Llama 3.1 is competitive with the latest OpenAI and Anthropic models. You can try Meta AI 3.1 right now at meta.ai and on WhatsApp in the U.S. Keep your Meta Quest VR headset plugged in and in sleep mode to get the update as soon as its available in your area.

If Meta can get AI working on a Quest 3, then why can't we get Apple Intelligence on Vision Pro?

Apple and Meta are rivals in the space to capture customers in what was once seen as the "next big thing" in tech - virtual reality. But the past year has seen attention turn to AI as the next "next big thing." So what do you do if you're working on both of these things? Why you put them together, of course! Meta is going to start rolling out its Meta AI feature to the Quest 3 headset. Naturally, this gives you access to the ChatGPT-rivalling voice assistant powered by Llama, which lets you ask any number of questions. Plus, this will let the VR headset see what you see so you can translate text, get fashion tips, and more, all completely hands-free. For now, these features have only been shown off in a series of demos released by Meta on its blog, but the features are heading to the Quest 3 soon. At the moment, they're already available on the Meta x Ray-Ban glasses, but these lack any AR or VR elements. And after this announcement, I'm left with one question: why can't we get Apple Intelligence on Vision Pro? There is some good news. In a recent Power On newsletter, Bloomberg's Mark Gurman says unequivocally that "Apple Intelligence is coming to the Vision Pro". But it reportedly won't happen this year. So that's good news! Notification prioritization, the new Siri, and integration with ChatGPT are all particularly relevant for Apple's headset. With Vision Pro being pegged as a productivity device, the writing tools could also come in particularly handy. While vision-based translation features weren't shown off as part of Apple Intelligence, we're sure they're achievable further down the road. As a multimodal set of AI features, Apple Intelligence should be able to match Meta's incoming AI features. If not, Apple could even beat the Quest 3 offering. All the Apple Intelligence features are tied into Apple's ecosystem. So not only will you be able to do more in the headset, you'll be able to bring it back to your other devices as well. Plus, let's not forget Apple Intelligence runs locally, while these Meta AI features run in the cloud. So AI on the Vision Pro would be faster and more secure. It seems like Apple Intelligence is coming to the Vision Pro, thankfully. And when it does arrive, it could be a compelling addition to the mixed reality headset.

Meta Quest to get AI features ahead of Vision Pro - 9to5Mac

The move will see the artificial intelligence features currently available in Ray-Ban smart glasses brought to the Quest too, initially as an experimental feature ... Previous reports have suggested that Vision Pro won't get any Apple Intelligence features this year - and possibly not at all. Earlier this month, John Gruber suggested that AI wasn't coming to Vision Pro at all, because the devices simply don't have enough spare processor capacity to run it. One other surprising revelation is that Apple Intelligence won't be supported by Vision Pro, despite the M2 chip being powerful enough to do so. The reason here, says Gruber, is that the chip is already running at close to capacity, so doesn't have enough spare capacity for Apple Intelligence. "According to well-informed little birdies, Vision Pro is already making significant use of the M2's Neural Engine to supplement the R1 chip for real-time processing purposes  --  occlusion and object detection, things like that." Bloomberg's Mark Gurman posted a conflicting report, saying that AI would be coming to Vision Pro next year. Either way, it appears there will be no AI features in 2024. Ray-Ban Meta smart glasses already include AI features, and the company says they will now be coming to Quest headsets too. Engadget reports that the rollout will initially be limited to North America.