Google DeepMind Plans to Merge Gemini and Veo AI Models for Enhanced Real-World Understanding

DeepMind CEO Demis Hassabis says Google will eventually combine its Gemini and Veo AI models | TechCrunch

In a recent appearance on "Possible," a podcast co-hosted by LinkedIn co-founder Reid Hoffman, Google DeepMind CEO Demis Hassabis said Google plans to eventually combine its Gemini AI models with its Veo video-generating models to improve the former's understanding of the physical world. "We've always built Gemini, our foundation model, to be multimodal from the beginning," Hassabis said, "and the reason we did that [is because] we have a vision for this idea of a universal digital assistant, an assistant that [...] actually helps you in the real world." The AI industry is moving gradually toward "omni" models, if you will -- models that can understand and synthesize many forms of media. Google's newest Gemini models can generate audio as well as images and text, while OpenAI's default model in ChatGPT can natively create images -- including, of course, Ghibli-style art. Amazon has also announced plans to launch an "any-to-any" model later this year. These omni models require a lot of training data -- images, videos, audio, text, and so on. Hassabis implied that the video data for Veo is coming mostly from YouTube, a platform that Google owns. "Basically, by watching YouTube videos -- a lot of YouTube videos -- [Veo 2] can figure out, you know, the physics of the world," Hassabis said. Google previously told TechCrunch its models "may be" trained on "some" YouTube content in accordance with its agreement with YouTube creators. Reportedly, Google broadened its terms of service last year in part to allow the company to tap more data to train its AI models.

YouTube is quietly training Google's next AI brain

Google DeepMind CEO Demis Hassabis revealed plans to eventually fuse the company's Gemini AI with its Veo video generator, aiming to teach the AI more about the physical world, during a recent appearance on the Possible podcast. Hassabis explained the strategy aligns with their vision for a "universal digital assistant" capable of aiding users in real-world scenarios. "We've always built Gemini, our foundation model, to be multimodal from the beginning," he stated on the podcast co-hosted by Reid Hoffman. This move reflects a broader industry shift towards versatile "omni" models. Google's latest Gemini versions already handle audio, image, and text generation, while rivals like OpenAI enable image creation in ChatGPT, and Amazon intends to launch an "any-to-any" model. Developing these comprehensive models demands vast datasets spanning video, images, audio, and text. Hassabis hinted that the video data fueling Veo largely originates from YouTube, a Google-owned platform. He elaborated that by processing extensive YouTube content, Veo learns about real-world physics. "[Veo 2] can figure out, you know, the physics of the world," Hassabis commented regarding the model watching "a lot of YouTube videos." Google previously acknowledged to TechCrunch its models "may be" trained on "some" YouTube content, consistent with agreements with creators. Reports suggest Google updated its terms of service last year, potentially expanding access to data for AI training purposes.