Google's Project Genie now simulates real streets using 280 billion Street View images

Google's Genie world model can now simulate real streets with Street View | TechCrunch

We've all pulled up Street View on Google Maps to show a friend what our childhood home looked like, or dropped that little person icon onto the streets of Paris to see if we booked a hotel in a cool neighborhood. Imagine being able to do that, but in a more immersive, interactive way that allows you to really simulate the street and its environs, and even do things like adjust the weather or see what it would look like in a "Day After Tomorrow" scenario. That's one of the goals of Google's latest integration. Starting today, Google DeepMind is connecting Street View to Project Genie, the company's general-purpose world model that can generate diverse, interactive environments. The new feature launched during the Google I/O developer conference. "It's really powerful for both the agent [and robotics] use case and for humans to play with, and that's always been the thesis of Genie," Jack Parker-Holder, a research scientist on DeepMind's open-endedness team, told TechCrunch. He gave the example of a new robot being deployed in London, which rarely sees the sun. Genie could, Parker-Holder says, simulate those scarce occasions when the sun glints off the Victorian housing, so the rays don't shock the robot when it happens. "Simultaneously, you might say, 'I'm going to New York City, but not this time of year,'" he continued. "'It's going to be snowy. I want to see what that block looks like in the snow.'" Google has been collecting Street View data for 20 years via cars with cameras and individuals strapped with "tracker backpacks." The tech giant has collected north of 280 billion images across 110 countries and seven continents. "With Street View, we have imagery from a large quantity of the world," Jack said. "You can imagine how potentially powerful it is to combine this rich source of real-world information and data with an ability to simulate worlds." Google released its latest world model Genie 3 for research preview last August and opened up access to the tool to Google AI Ultra subscribers in the U.S. in January, allowing customers to create interactive game worlds from text prompts or images. The goal is to use Genie for educational experiences, gaming, and robotics training. Genie 3 is already helping to power one of Waymo's simulators to train its self-driving cars on "exceedingly rare events" like tornadoes or casual elephant encounters. Adding Street View data to that could help Waymo prepare to launch in more cities around the globe. Waymo has its own simulator that it relied on to scale to 11 U.S. cities and test its AI driver in several more. The difference with Genie, says Parker-Holder, is that those are all from the car's point of view. Street View allows for not only simulating a world anchored to a real place, but also shifting the point of view to other types of agents, like a human or a robot. Google is launching Street View in Genie to some Ultra users in the United States starting today, with access rolling out at scale over time. Global Ultra users will gain access over the next few weeks, per the company. The researchers' goal is to put this new capability into as many hands as possible, per Diego Rivas, a product manager at DeepMind. He cautioned that Street View in particular and Genie in general is still an experiment, so there's much to improve upon in terms of accuracy. In the samples the Google team showed me -- including an underwater simulation of a neighborhood I used to live in -- the results are impressive and recognizable, but still video game quality rather than photorealistic. The models are also not yet physics-aware, meaning they don't yet understand cause and effect. For example, in a simulation of a woman running through a snowy Joshua Tree, she ran right through cacti and bushes. Compare that to, say, Google's image generator Nano Banana -- which can now generate perfect text in infographics -- or its video generator Veo -- which understands that paper boats drift on water currents, smoke disperses into the air, and fabric drapes over forms. Physics isn't hard-coded into these models; they learn it intuitively over time through passive observation, as a living being would. "I think for this kind of model, it's maybe six to 12 months behind video in terms of the accuracy and quality, so I think it's something we will solve," Parker-Holder said. Jonathan Herbert, director of Google Maps who started on the Street View team as an intern 12 years ago, said that Genie can't yet create a faithful reconstruction of a street. He thinks the real breakthrough is the AI's spatial continuity. If you turn 360 degrees, the AI correctly remembers and simulates the environment behind you. From that point on, the model can build a new environment on top of that. "We have long thought about how we can build out the best and richest model of the world on top of Street View data," Herbert said. "It's definitely been an idea of ours to use Maps Data in new ways and for new kinds of AI research for a pretty long time."

Google's Project Genie: You Can Now Base Imaginary Worlds on Real Places

Macy has been working for CNET for coming on 2 years. Prior to CNET, Macy received a North Carolina College Media Association award in sports writing. Ever wondered what the Golden Gate Bridge would look like underwater? Now you can see it. On stage at the Google I/O annual developer conference, Google announced the launch of a new feature for Project Genie -- an experimental generative AI world model web app first released in January. The new capability integrates Google Maps' dataset of 280 billion Street View images across 110 countries to ground text-to-video simulations in real-world locations. That way, you can explore your favorite real places, but use generative AI to give them an imaginative twist. By connecting these two products, you can take the generative capabilities of Project Genie and combine them with the vast data of Maps' Street View to anchor the AI model in reality. To get started, tap the Maps pin to choose a location in the US, then use the AI to select a style. (For now, this feature is available only for US locations, with plans to expand.) For instance, if you do want to see the Golden Gate Bridge -- but underwater -- you can tap "Ocean World" style, which will add an imaginary scuba diver and schools of fish to an otherwise realistic depiction of the famous landmark. You can also select from styles like "Desert Sands," "Stone Age" or "B&W film" style for a vintage feel to match whatever vibe you're going for. This capability uses Maps Imagery Grounding, the same technology developers use to make AI visuals with Street View. Project Genie (and the new Street View capability) will now also be rolling out globally to Google AI Ultra subscribers.

Project Genie adds Google Street View integration and goes live for global AI Ultra users - Engadget

Ground your snow-globe worlds in real-world locations from Google Maps. Project Genie is rolling out today for all adult Google AI Ultra subscribers across the globe, following its debut in the United States this January. The service is also getting a new Street View capability that can generate interactive landscapes based on real-world locations found on Google Maps, starting with places in the US. Project Genie is Google's AI-powered system for creating explorable snow-globe environments from written prompts, with creations lasting 60 seconds at 720p and 24 fps. Users are able to create contained worlds in whatever style they'd like, complete with a character of their own description, and then move a camera around that space. The fresh Street View functionality allows users to base their AI worlds on location photos pulled from Google Maps, grounding their creations in a snapshot of reality. In a demo video shown at I/O, Google turned the Golden Gate Bridge into an underwater scuba scene, for instance. Google plans to expand the Street View interaction to more real-world locations over time. On the surface and in Google's ads, Project Genie looks like a Cocoon world generator, producing perfect little video game bubbles out of vague written prompts -- but that's not really what's happening here. Project Genie is not a game generator by any means and it's silly to suggest so. Among a suite of technical differences, it ignores the expertise and precision required to create consistent, responsive and bespoke mechanics in persistent and believable worlds, and belies the importance of narrative flow in interactive stories. Project Genie is capable of generating a limited 3D environment with a controllable camera attached to an avatar. This is cool, but it's not game development.

Project Genie now lets you reimagine your favorite real-world spots with a creative twist

Google is also expanding access to Project Genie, including the new Street View capabilities, beyond the US to eligible Google AI Ultra $200 subscribers worldwide who are 18 or older. Earlier this year, Google launched Project Genie, an AI model capable of generating diverse, interactive world environments from user prompts. Users can let their imagination go wild with the prompts, but if you want something a bit more grounded in reality, Google is now bringing together Project Genie's generative powers with real-world imagery from Google Maps' Street View. With Project Genie's Street View upgrade, AI agents and robots now have a virtual environment that they can navigate and interact with the complexities of the real world. Users will also be able to leverage real-world imagery to explore their favorite spots, or even reimagine them with a creative twist. Within Project Genie, users can now tap the Maps pin to choose a place in the US. They can then optionally select a style for their world, like "Desert Sands" or "Stone Age." The next step in the process involves describing the character for exploration. Project Genie will use all of this information to create an imaginative world with its starting location tied to Street View's real-world imagery. Street View imagery in Project Genie is now available for places in the US, with plans to expand to more places over time. Alongside the announcement, Google is also expanding access to Project Genie beyond the US. Starting today, Project Genie (including its new Street View capability) will gradually roll out to all eligible Google AI Ultra $200 subscribers worldwide, provided they are 18 years or older. Google notes that Project Genie is still an experimental research prototype in Google Labs. The company continues working behind the scenes to make the details even sharper and more accurate.

Google's interactive world builder, Project Genie, will drop you in real places using Street View

Google just announced its world-exploring game, Project Genie, is getting a new feature that drops you in real-world locations, like your own street. Project Genie rolled out at the beginning of this year. In essence, it builds small worlds using Nana Banana Pro models that can be explored with video-game-like controls. Google has positioned the tool as an experimental research prototype with a first or third-person point of view. Project Genie launched with the capability to generate fictional places, but the latest updates add a touch of realism. Google's new version of Project Genie will get access to Google Street View imagery to start creating worlds based on real places. Google explains: With this upgrade, you'll be able to leverage real-world imagery to explore your favorite spots, or reimagine them with a creative twist... When creating imaginative worlds in Project Genie, you can now also base them on real places. Just tap the Maps pin to choose a place in the U.S. and optionally select a style for your world, like "Desert Sands" or "Stone Age." Then, describe your character - like your favorite animal, comic book hero or even a claymation monster, and Genie will use this information to create an imaginative world with its starting location tied to Street View's real-world imagery. When you begin generating a real location, you're given a set of environments to choose from. A character can be chosen by entering its description. It looks like locations are limited only to images available in Street View, which leaves the possibilities virtually endless. Project Genie will still be limited to users on the AI Ultra tier. The tool takes up a lot of processing power, considering it's developing hundreds of images referenced on each other to create a scene that can be explored at 20-24 frames per second. The clips are limited to 60-second sessions, but it's enough time to get to see familiar places in the same or completely different states of being, using familiar WASD controls. Google says Project Genie with Street View access will be available for users in the US, with plans to expand "over time." The research tool won't be entirely accurate, either. There are plans to make details sharper and more accurate.