AI Recreates DOOM: A Groundbreaking Moment in Game Development

AI has remade Doom, and it looks like the real thing

(Image credit: Dani Valevski / Yaniv Leviathan / Moab Arar / Shlomi Fruchter) If AI is the future of gaming, perhaps it makes sense to begin with Doom. The classic game has been ported for all kinds of things it wasn't originally meant for. Honouring that tradition is GameNGen, developed by Google and researchers at Tel Aviv University. GameNGen is a neural model-based game engine that allows real-time interaction with complex gaming environments without the use of a traditional game engine. The researchers think could transform how video games are made, and how they're played, and Doom is just the beginning. As demonstrated in the real-time recordings above, GameNGen can simulate the classic Doom at over 20 frames per second with a visual quality that resembles the original. The team trained a reinforcement learning agent to play the game to provide the data for a generative AI model. They repurposed Stable Diffusion v1.4, an open-source generative AI diffusion model, and conditioned it on a sequence of previous actions and observations. The data was used to train the diffusion model to predict the next frame, allowing it to simulate complex state updates like health and ammo, attacks and interactions with the environment. The model employs conditioning augmentations to mitigate sampling divergence and achieve stable auto-regressive generation through long sequences (see the paper). The researchers Dani Valevski, Yaniv Leviathan, Moab Arar and Shlomi Fruchter plan to refine GameNGen's capabilities, increasing its memory and improving its handling of complex environments. While the proof of concept has only been demonstrated on Doom so far, the team thinks the tech could have much broader applications for interactive software as well as other games. The suggestion is that we could see games that are generated by AI instead of being coded manually, allowing design via text description or example images. Generative AI could also be used to create potentially endless expanding worlds via procedural generation and more immersive games that respond dynamically to player actions. Generative AI isn't only being used to create playable games. AI robot boxing is also now a thing and AI live stream software is going viral due to its ability to create very realistic live deepfakes.

GameNGen is capable of creating an AI Doom game

Google researchers have achieved a significant milestone in AI by developing a neural network capable of creating an AI Doom game, named GameNGen. The system, named GameNGen, represents a major advancement in AI, producing playable gameplay at 20 frames per second on a single chip. GameNGen marks a groundbreaking achievement as it's the first AI-powered game engine capable of simulating a complex video game with high-quality graphics and interactivity. Unlike traditional game engines that rely on meticulously coded software to manage game states and render visuals, GameNGen autonomously simulates the entire game environment using a generative diffusion model. The transition from traditional game engines to AI-driven systems like GameNGen has the potential to revolutionize the $200 billion global gaming industry. By eliminating the need for manually programmed game logic, AI-powered engines can significantly reduce development time and costs. Data-driven design: The science behind the most engaging games This technological shift could democratize game creation, enabling smaller studios and even individual creators to produce complex, interactive experiences that were previously unimaginable. AI-driven game engines could open the door to entirely new genres of games, where the environment, narrative, and gameplay mechanics dynamically evolve based on player actions. This innovation could reshape the gaming landscape, moving the industry away from a blockbuster-centric model towards a more diverse and varied ecosystem. The potential applications of GameNGen extend far beyond gaming. Its capabilities suggest transformative possibilities in industries such as virtual reality, autonomous vehicles, and smart cities, where real-time simulations are essential for training, testing, and operational management. While GameNGen represents a significant step forward, it also presents challenges. Although it can run Doom at interactive speeds, more graphically intensive modern games would likely require much greater computational power. Additionally, the current system is tailored to a specific game (i.e., Doom), and developing a more general-purpose AI game engine capable of running multiple titles remains a tough challenge. Nevertheless, GameNGen is a crucial step towards a new era in game engines -- one where games are not just played by AI but also created and powered by it. As AI continues to advance, we may be on the cusp of a future where our favorite games are born not from lines of code, but from the boundless creativity of machines. This development also opens up exciting possibilities for game creation and interaction. Future games could adapt in real-time to player actions, generating new content on the fly. AI-powered game engines might also dramatically reduce development time and costs, potentially democratizing game creation. As we stand on the brink of this new era in gaming, one thing is clear: the lines between human creativity and machine intelligence are blurring, promising a future of digital entertainment we can scarcely imagine. With GameNGen, Google researchers have given us an exciting glimpse of that future -- a world where the only limit to our virtual experiences is the imagination of AI.

Generative AI creates playable version of Doom game with no code

A neural network can recreate the classic computer game Doom despite using none of its code or graphics, hinting that generative AI could be used to create games from scratch in future An AI-generated recreation of the classic computer game Doom can be played normally despite having no computer code or graphics. Researchers behind the project say similar AI models could be used to create games from scratch in the future, just as they create text and images today. The model, called GameNGen, was made by Dani Valevski at Google Research and his colleagues, who declined to speak to New Scientist. According to their paper on the research, the AI can be played for up to 20 seconds while retaining all the features of the original, such as scores, ammunition levels and map layouts. Players can attack enemies, open doors and interact with the environment as usual. After this period, the model begins to run out of memory and the illusion falls apart. The original Doom was released in 1993 and has become a popular subject for computer science projects in the years since, including attempts to get it running on unusual and limited hardware such as toasters, treadmills and espresso machines. But in all those cases, the hardware is simply running the original game's code. What GameNGen does is fundamentally different: a type of AI called a neural network has learned by observation how to recreate the game without seeing any of its code. The researchers first created an AI model that learned to interact with Doom as a human would. That model was then tasked with playing the game over and over again while a second AI model, based on the Stable Diffusion image generator, learned how hundreds of millions of inputs resulted in changes in the game state. That second model essentially then became a copy of the game, with all of the knowledge, rules and instructions from the original code encoded in the mysterious network of artificial neurons in its own architecture. In tests, human players were only slightly better than random chance at distinguishing short clips of the game from clips of the AI simulation. GameNGen's creators claim in their paper that it is a proof-of-concept for games being created by a neural network rather than lines of code. They suggest that games could be generated from text descriptions or concept art, which would make production less costly than using human programmers. Andrew Rogoyski at the University of Surrey, UK, says the idea of getting a neural network to hallucinate a game environment, and the interactions a human has with it, is an interesting step forward, but not one that will replace human game designers. "I don't think it's the end of those game studios. I think what the game studios have is the imagination, the skills, to actually create these worlds, to understand gameplay, to understand engagement, understand how to draw us into a story. It's not just the nuts and bolts, the bits and bytes," he says. "There's something very human about creating engaging experiences that we as human beings enjoy that, at the moment, and for the foreseeable future, will largely come from other human beings."

AI DOOM first fully AI generated video game from Google Research

Artificial intelligence (AI) is transforming industries at an unprecedented pace, and the world of video game development is no exception. In a groundbreaking development, Google Research has recently unveiled a remarkable breakthrough: fully AI-generated video games. These games represent a paradigm shift in the gaming industry, where every pixel is created in real-time by AI, marking a significant departure from traditional game development methods and paving the way for a new era of AI-driven content creation. This innovation offers a tantalizing glimpse into the future of interactive entertainment, where the boundaries between the virtual and the real become increasingly blurred. The promise of AI-generated content is nothing short of innovative. Imagine immersing yourself in a video game where every element - from the sprawling landscapes to the intricate details of characters and objects - is generated by AI as you interact with it. Unlike traditional games, which are pre-rendered and static, AI-generated games are inherently dynamic and responsive. The AI acts as a real-time game engine, creating the game environment, characters, and actions on the fly, providing a unique and personalized experience for each player. This level of adaptability and customization opens up a world of possibilities for game designers and players alike. You can read the complete Google Research paper here. To fully appreciate the magnitude of this advancement, it is essential to consider the historical context of video game development. The classic game "Doom," released in 1993, was a trailblazer in the gaming industry. It set the standard for first-person shooters and influenced countless games that followed in its wake. The significance of "Doom" in shaping gaming culture cannot be overstated, making it a fitting subject for AI experimentation and showcasing the potential of AI-generated content. The evolution from procedural generation to real-time AI generation represents a significant leap forward in game development. Procedural generation, as seen in games like "Diablo" and "No Man's Sky," has been a stepping stone towards AI-generated content. These games use algorithms to create vast, varied environments, but they still rely on predefined rules and assets. The transition to AI-driven content creation enables the development of more complex and unpredictable game worlds, offering players an unparalleled level of immersion and replayability. The groundbreaking paper "Diffusion Models are Realtime Game Engines," authored by researchers from Google Research, Tel Aviv University, and Google DeepMind, lays the technical foundation for this innovation. The study provide more insights into how diffusion models, a type of neural network, can generate game frames in real-time. This research is a testament to the immense potential of AI in transforming game development and pushing the boundaries of what is possible in interactive entertainment. At the heart of this technology lies neural network prediction. The AI model predicts and generates each frame of the game as the player interacts with it, ensuring a seamless and immersive experience. This real-time generation is made possible by advanced machine learning techniques, which enable the AI to understand and anticipate player actions, adapting the game world accordingly. The implications of AI-generated content extend far beyond the realm of video games. AI has the potential to transform content creation across various media, including TV shows, movies, and even virtual reality experiences. By tailoring content to individual preferences and behaviors, AI can enhance user engagement and satisfaction, delivering personalized entertainment that resonates with each viewer. As AI capabilities continue to expand, the role of human developers may undergo a significant shift. AI has the potential to automate many aspects of programming, from writing code to debugging and optimization. This shift could lead to more efficient and innovative software development processes, reducing the need for human intervention and allowing developers to focus on higher-level creative tasks. Moreover, AI-generated content raises intriguing possibilities for simulation theory and real-world applications. By modeling the real world using AI, we could create highly realistic simulations for training, research, and entertainment purposes. This technology could transform fields such as education, healthcare, and urban planning, allowing for safe and cost-effective experimentation and decision-making. Looking ahead, the future of AI-generated content is brimming with possibilities. We envision a world where AI can create custom video games and other media on demand, tailored to individual preferences and desires. This technology could eventually replace traditional operating systems and application layers, offering a more personalized and immersive digital experience that blurs the line between the virtual and the real. The advent of fully AI-generated video games represents a groundbreaking development in the gaming industry and a harbinger of the future of interactive entertainment. By using advanced AI techniques, we can create dynamic, responsive, and personalized content that redefines the way we engage with digital media. As we continue to explore the vast potential of AI, the future of game development and content creation looks brighter and more exciting than ever before. The possibilities are limited only by our imagination, and the journey ahead promises to be a thrilling ride into uncharted territories of creativity and innovation.

A neural network is 'playing' Doom by generating frames one by one and it's pretty trippy, man

"We note that nothing in our technique is Doom specific except for the reward function for the RL-agent". "But can it run Doom?" is an adage that has managed to travel through almost every piece of tech on the market. From controlling the game with a toaster to running Doom on a pregnancy test (please wash your hands after), it has almost become a benchmark of geeky creativity. A joint effort from researchers at Google Research, Google DeepMind, and Tel Aviv University has managed to get the classic shooter running on nothing but a neural network (via Futurism). This essentially generates a frame, based on a model that is trained on the real game. You can check out a video of it running in real-time right here but there are natural limitations to it. For the unaware, a neural network is an AI structure modelled after the human brain that uses machine learning to process commands and prompts. They are notably used in predictive models, due to their ability to grasp concepts more broadly than traditional AI. A large part of making them better is called "training", where it iterates on a small level, using wide sets of data. When a network is "trained" on something, that is to say it is pulling in the data from it and using it in some way. In the example of generative AI, trained models will largely be quite similar to their source material until those data sets are wide enough. Though it's obviously very impressive to run something this complex through a neural model, it's worth noting that the game is played at a slow pace in the video, with plenty of cuts -- clearly grabbing the most fluid and realistic moments of the game. This isn't to diminish the work but to place it in context. You can't go out and just play through Doom right now with the support of a neural network. It is a test and not much more than that right now. Without touching on any ethical discussions of (specifically generative) AI use in games, the accompanying paper acknowledges the experiment's limitations and makes an argument for the future of the tech. Due to the neural network's limited memory, it can only store 3 seconds from the game itself. It does seem to hold onto HUD effects but memory positioning and more gets lost while playing. It also fails to fully predict following frames, as you can see in the video above, with visual glitches and a lack of clarity in some areas. Following on from this, the paper says "We note that nothing in our technique is Doom specific except for the reward function for the RL-agent". It then says that the same basic network could be used to attempt to emulate other games. Furthering this, it then makes the argument that this same engine could be used to replace or accompany programmers working on actual games. As the network appears to be trained on specific games with specific functions, no argument is made for how this would translate to creating entirely new games. The paper then says that an engine like this could be used to "include strong guarantees on frame rates and memory footprints", following this up by saying "We have not experimented with these directions yet and much more work is required here, but we are excited to try!" It is unclear how well this network will function outside of what we've seen thus far, and I'd encourage not making too many assumptions about the future of the tech just yet, but it is still rather impressive in itself -- even if the goals of the future of the paper seem a bit lofty.