Elon Musk's xAI Ventures into 'World Models' for Advanced AI Gaming and Simulations

Elon Musk's xAI joins race to build 'world models' to power video games

Elon Musk's xAI is pushing to build so-called world models, joining rivals such as Meta and Google in the race to develop artificial intelligence systems that can navigate and design physical environments. The San Francisco-based start-up hired specialists from Nvidia over the summer to work on these next-generation AI models, which train on videos and data from robots to understand the real world. World models could push the capabilities of AI beyond that of the large language models, trained on text, that underpin popular AI tools such as ChatGPT and xAI's Grok. Two people familiar with the plans said the company was building world models with a view to applying them in gaming, where they could be used to generate interactive 3D environments. One of the people added that they could be applied to AI systems for robots. xAI has hired Zeeshan Patel and Ethan He, two AI researchers from Nvidia with experience in world models. Nvidia has been a leader in developing this technology with its Omniverse platform, which creates and runs simulations. Some tech groups have vast expectations of world models, which could unlock uses for AI beyond software and computers in physical products such as humanoid robots. Last month, Nvidia told the Financial Times that the potential market for world models could be almost the size of the present global economy. xAI would release a "great AI-generated game before the end of next year", Musk said in a post on X, confirming a target the billionaire set last year. On Tuesday, xAI launched its latest image and video generation model, which it said had "massive upgrades" and is free to use. Current video generation models, such as OpenAI's Sora, generate frames of images for videos by predicting patterns learned from training data. World models would be a big advance as they would have a causal understanding of physics and how objects interact in different environments in real time. The company is advertising for technical staff in both image and video generation to join its "omni team", which "creates magical AI experiences beyond text, enabling understanding and generation of content across various modalities, including image, video and audio". Salaries for these jobs range from $180,000 to $440,000. It also has an open position for a "video games tutor", who will train Grok to produce video games and enable "users to explore AI-assisted game design", for $45 to $100 an hour. Musk follows other leading AI labs, such as Google and Meta, that are also working on these systems. However, world models remain a huge technical challenge. Finding sufficient data to simulate the real world and to train such models has proved difficult and costly. Michael Douse, head of publishing at Larian Studios, which develops the video game Baldur's Gate 3, said on X this week that AI could not solve the "big problem" for the games industry, which is "leadership [and] vision". He added that the industry did not need "more mathematically produced, psychologically trained gameplay loops [but] rather more expressions of worlds that folks are engaged with, or want to engage with". xAI, Patel and He did not respond to requests for comment.

Elon Musk's xAI hires Nvidia experts to develop advanced 'world models' - The Economic Times

Elon Musk's AI startup, xAI, is developing advanced "world models" that help AI understand and interact with physical environments. To do this, xAI hired experts from Nvidia experienced in training AI with videos and robot data. Nvidia leads this field with its Omniverse simulation platform.Elon Musk's AI startup, xAI, is working on developing "world models", according to the Financial Times. These are advanced AI systems that can navigate and design physical environments. These models aim to take AI beyond large language models like Grok and ChatGPT. To compete with companies such as Meta and Google, xAI hired experts from Nvidia over the summer. These specialists bring experience in training AI using videos and robotic data to help machines better comprehend the real world. As per the report, the experts recruited are Zeeshan Patel and Ethan He, both AI researchers from Nvidia who have worked on world models. Nvidia itself has been a leader in this field through its Omniverse platform, which builds and runs simulations. The report said that many in the tech industry have high hopes for world models, as they could enable AI to move beyond software and into physical products like humanoid robots. The company is also hiring technical staff for its "omni team," which focuses on creating AI systems that can understand and generate content across various formats, including photos, videos, and audio. Musk recently reiterated a goal he set last year, saying xAI will release a "great AI-generated game before the end of next year." Last month, xAI launched aits new image and video generation model, Grok Imagine, on social media platform X and offering it free to users. While current video generation models, like OpenAI's Sora, produce video frames by predicting patterns from training data, world models represent a significant leap forward. They would have a real-time, causal understanding of physics and how objects interact in different environments.