Grok AI Expands Capabilities with Vision and Multilingual Voice Support

Even Grok AI Can 'See' Now

There are a lot of trends in generative AI right now. There are the reasoning models like OpenAI's o3, that "think" through each step of a problem before it answers. There are also "deep research" features that can compile information from across the web to generate reports for you. But perhaps the trend that is most "futuristic" of all is Voice Mode. This is the future 2013's Her promised: a chatbot that you can talk to like any other person. The chatbot doesn't say anything differently than it would if you were chatting over text; however, it responds in a "realistic" and "natural" voice, which could create the illusion that you're talking to a person, not a robot. I've never found the feature to be particularly engaging, even from big names like ChatGPT. The tech is impressive, sure, but it's still painfully obvious to my ear that I'm talking to a bot. AI companies haven't been able to shake these identifying quirks, but that hasn't stopped people from forming "relationships" with chatbots -- even falling in love with them. What's more impressive to me is the feature's "vision" component. Some chatbots can not only talk back to you, but can access your camera to see what you're seeing, and incorporate that information in its replies. Both ChatGPT and Gemini offer these features, and now, so does Grok. Grok is the latest chatbot to gain this ability in its Voice Mode. xAI developer Ebby Amir announced the feature, dubbed "Grok Vision," on X Tuesday, noting that Grok Vision supports multilingual audio as well as realtime search. Those latter features are exclusive to SuperGrok subscribers, however. This Tweet is currently unavailable. It might be loading or has been removed. The feature is already live on my end. You can access it by tapping the existing Voice Mode option. If you haven't used this feature already, you'll need to grant Grok permission to access your device's microphone. Following this, you'll be able to start chatting immediately. However, to access Vision, you'll need to tap the camera icon in the bottom left corner. Here, allow Grok to access your camera. Once the feed is live, you can start asking Grok about what it sees. I'm not super keen on sending my live video feed directly to xAI, so I kept my phone directly on the table, so the video feed was all black. Grok, to its credit, tried earnestly to help me fix the problem, suggesting there might be something wrong with the camera, or that my environment was too dark. When I informed it that I had actually taken my phone up to outer space with me, it "laughed," and concluded that had to be the problem: "Ha, outer space, huh? That black feed makes sense now -- no light out there, and the camera's probably not designed for that environment. You might need a space-grade device to get a proper feed." This is the second big feature drop for Grok this month. Last week, xAI rolled out a memory feature for the bot, which allows it to access past conversations for more relevant responses.

Grok Can Now See Your Surroundings and Speak in Five New Languages

It can identify objects, landmarks, and answer queries about them Grok is getting a couple of new features, the AI firm announced on Wednesday. xAI's artificial intelligence (AI) chatbot is rolling out Grok Vision, a computer vision feature, to its iOS app. Additionally, the company is also shipping support for multilingual audio and real-time web search to the Grok app for iOS and Android smartphones. iPhone users can currently access these features for free, but they are only available to paid subscribers on Android. All of these features are part of the chatbot's Voice Mode. In a post on X (formerly Twitter), Ebby Amir, a member of technical staff at xAI, announced the new Grok features. As mentioned above, all three new features are available to iOS users without a subscription. Android users will need to pay for SuperGrok to access these features. A SuperGrok subscription is priced at Rs. 700 per month and Rs. 6,500 for a year. The biggest addition to the AI chatbot is Grok Vision, which is currently an iOS exclusive feature. Similar to Gemini Live with Video and ChatGPT's Advanced Voice with Video, Grok can now access the device's camera and process the feed in real-time. With this, users can point the device at any object and ask the AI questions about it. Gadgets 360 staff members tested Grok Vision, and the feature seems to have very low latency when connected to a relatively fast Wi-Fi network. In most cases, it was able to correctly identify the object, such as a smartphone or pair of earbuds, or something more abstract like the pattern on a shirt. With multilingual audio support, Grok can now speak in five new languages, alongside English. These include French, Hindi, Japanese, Spanish, and Turkish. While the chatbot previously accepted multilingual text input and generated text in these languages, it can now also understand multilingual verbal prompts and respond in the same language. Additionally, the voice mode is also being upgraded with real-time web search. This means users can ask Grok about current news and other information that requires a web search, and the AI will be able to respond to the queries.

Elon Musk's Grok AI Can See the World and Talk in Real-Time

Grok Vision is rolling out to iOS users, while only multilingual voice chat is coming to Android users. Elon Musk's xAI has added two new features to its Grok AI chatbot. You can now share your camera with Grok to allow the AI chatbot to see the world around you. xAI is calling it 'Grok Vision' which can see the surroundings and interact with you in different languages. However, there is no option to share your screen yet. In addition to Vision, xAI has added multilingual voice support in Grok so you can voice chat in real-time with Grok, in several local and global languages. xAI is putting effort into making Grok a personal AI chatbot. Recently, Grok received 'Memory' support too which can remember crucial parts from your conversation. Grok Vision is currently rolling out on the Grok app for iOS. Meanwhile, the Grok app for Android gets multilingual audio and real-time search support only. However, you will have to subscribe to the SuperGrok plan which costs $30 per month to access the new features on Android. Lately, many AI labs are starting to offer vision and voice capabilities in their AI chatbots. OpenAI added live audio, screen sharing, and camera sharing in ChatGPT last year in December. Then, Google brought Project Astra to Gemini, which allows the AI chatbot to see the screen and the world around it. Thankfully, Google has made the feature free for all Android users and it's rolling out in a phased manner.