Runway launches GWM-1 world model, expanding beyond video generation into robotics and AI agents

With GWM-1 family of "world models," Runway shows ambitions beyond Hollywood

AI company Runway has announced what it calls its first world model, GWM-1. It's a significant step in a new direction for a company that has made its name primarily on video generation, and it's part of a wider gold rush to build a new frontier of models as large language models and image and video generation move into a refinement phase, no longer an untapped frontier. GWM-1 is a blanket term for a trio of autoregression models, each built on top Runway's Gen-4.5 text-to-video generation model and then post-trained with domain-specific data for different kinds of applications. Here's what each does. GWM Worlds offers an interface for digital environment exploration with real-time user input that affects the generation of coming frames, which Runway suggests can remain consistent and coherent "across long sequences of movement." Users can define the nature of the world -- what it contains and how it appears -- as well as rules like physics. They can give it actions or changes that will be reflected in real-time, like camera movements or descriptions of changes to the environment or the objects in it. As the methodology here is basically an advanced form of frame prediction, it might be a stretch to say these are full-on world simulations, but the claim is that they're reliable enough to be usable as such. Potential applications include pre-visualization and early iteration for game design and development, generation of virtual reality environments, or educational explorations of historical spaces. There's also a major use case that takes this outside Runway's usual area of focus: World models like this can be used to train AI agents of various types, including robots. The second model, GWM Robotics, does just that. It can be used "to generate synthetic training data that augments your existing robotics datasets across multiple dimensions, including novel objects, task instructions, and environmental variations."

Runway releases its first world model, adds native audio to latest video model | TechCrunch

The race to release world models is on as AI image and video generation company Runway joins an increasing number of startups and big tech companies by launching its first one. Dubbed GWM-1, the model works through frame-by-frame prediction, creating a simulation with an understanding of physics and how the world actually behaves over time, the company said. A world model is an AI system that learns an internal simulation of how the world works so it can reason, plan, and act without needing to be trained on every scenario possible in real life. Runway, which earlier this month launched its Gen 4.5 video model that surpassed both Google and OpenAI on the Video Arena leaderboard, said its GWM-1 world model is more "general" than Google's Genie-3 and other competitors. The firm is pitching it as a model that can create simulations to train agents in different domains like robotics and life sciences. Runway released specific slants to the new world model called GWM-Worlds, GWM-Robotics, and GWM-Avatars. GWM-Worlds is an app for the model that lets you create an interactive project. Users can set a scene through a prompt, and as you explore the space, the model generates the world with an understanding of geometry, physics, and lighting. Runway said that while Worlds could be useful for gaming, it's also well positioned to teach agents how to navigate and behave in the physical world. With GWM-Robotics, the company aims to use synthetic data enriched with new parameters like changing weather conditions or obstacles. Runway says this method could also reveal when and how robots might violate policies and instructions in different scenarios. Runway is also building realistic avatars under GWM-Avatars to simulate human behavior. Companies like D-ID, Synthesia, Soul Machines, and even Google have worked on creating human avatars that look real and work in areas like communication and training. Besides releasing a new world model, the company is also updating its foundational Gen 4.5 model released earlier in the month. The new update brings native audio and long-form, multi-shot generation capabilities to the model. The company said that with this model, users can generate one-minute videos with character consistency, native dialogue, background audio, and complex shots from various angles. The Gen 4.5 update nudges Runway closer to competitor Kling's all-in-one video suite, which also launched earlier this month, particularly around native audio and multi-shot storytelling. It also signals that video generation models are moving from prototype to production-ready tools. Runway's updated Gen 4.5 model will be available to enterprise customers first and then to all paid plan users in the coming weeks. The company said that it will make GWM-Robotics available through an SDK. It added that it is in active conversation with several robotics firms and enterprises for the use of GWM-Robotics and GWM-Avatars.