Oasis: AI-Generated Minecraft Clone Showcases Future of Interactive Gaming

Oasis is an interactive AI experience that turns Minecraft into a nightmarish hallucination

Editor's take: In Ernest Cline's 2015 novel Ready Player One, "Oasis" was a virtual interactive dystopia with life-like graphics and too many Delorean replicas fighting to uncover the game's hidden Easter eggs. In Etched's AI-fueled dystopia, Oasis is an interactive model where we can play a hallucinated clone of the worst Minecraft world ever imagined. Cupertino startup Etched recently unveiled Oasis, a new AI experiment described as the first model capable of generating open-world games. Built with the help of Decart, Oasis generates an unlikely Minecraft experience frame-by-frame. Each new frame is derived from the previous and by player input. Oasis resembles a low-definition Minecraft with dementia because of the garbled approximations of the original pixelated graphics. Etched is "excited" to bring this AI-fueled nightmare to the world, stating that future interactive experiences will be much better and capable of running in 4K graphics thanks to new specialized hardware for AI acceleration. The developers trained Oasis on a massive video dataset of recorded Minecraft gameplay. The final AI product has no game engine, no logic, and no code at all. The optimized interactive model runs on Sohu, an AI ASIC technology Etched is developing. The current version of Oasis can render its Minecraft approximation in 360p and 20 frames per second, using Nvidia's H100 GPUs. With Sohu, it can scale to 4K and accommodate 10 users per H100. The developer is working on AI-generated interactive video content because an estimated 70 percent of internet traffic is video. Video models are now learning to generate entire physical worlds and games, ultimately bringing new products and business opportunities to professional and amateur developers. "Whether gaming, generative content, or education, we believe that large, low-latency, interactive video models will be central to the next wave of AI products," Etched said. Today's models are too slow and expensive to run in production environments. However, things should go much smoother with specialized chips like Sohu. Etched is simultaneously building its new AI video models and Sohu chip design to surf this new wave of "exciting" AI products. When asked why someone should waste their time playing a copyright-infringing abomination resembling Minecraft on acid instead of the original game, Decart AI said that the interactive experience is not designed to be better than the original. Oasis simply showcases what real-time AI content generation looks like in its infancy. The model doesn't infringe copyright, as Minecraft-inspired open-source projects such as Minetest have thrived without DMCA takedowns from Mojang.

I've seen the future of gaming -- AI model creates playable Minecraft from a picture

A new AI model called Oasis has made waves by generating a playable version of Minecraft-like gameplay using only images. Developed by Decart in collaboration with Etched, this innovation offers an exciting look at the future of video game creation and interactive experiences. Oasis is referred to as "the world's first real-time AI world model." Unlike traditional game engines, Oasis creates gameplay based on user input and visual data. It processes keyboard and mouse inputs, generating each gameplay frame on the fly at about 20 frames per second. The system simulates physics, game rules, and graphics internally, allowing for a dynamic gaming experience that adapts to how the player interacts with it. Netflix is one company actively exploring the potential for generative AI in gameplay, including shutting its own game studio. The goal is to "accelerate development and create novel gaming experiences," which won't be popular among the employees laid off from the gaming studio. Oasis is built on a new AI architecture that combines Vision Transformer technology with a diffusion model for image processing. This setup enables it to create gameplay frames in real time based on user input. The system operates at 20 frames per second with "zero latency" on a single NVIDIA H100 GPU, generating each frame in about 0.04 seconds. This is much faster than other AI video generators like OpenAI's Sora or Runway Gen-3, which usually take 10-20 seconds to produce one second of video. While this is very impressive, it still has some limitations. The gameplay resembles a low-resolution and somewhat glitchy version of Minecraft, with issues around object permanence. Moreover, the environment can change unexpectedly when the player turns, while the frame rate and visual quality fall short of traditional games. Despite these challenges, the technology shows great promise for the future. The rise of AI-generated gaming experiences like Oasis could significantly impact the industry. Game developers might quickly create and test new gameplay ideas, while models could adjust game worlds and stories in real time based on what players prefer. Additionally, automating parts of game creation could help reduce production costs. However, the use of AI to create games based on existing titles raises important questions about copyright and creative ownership. It's not clear yet if using gameplay footage for training AI models violates game developers' intellectual property rights. As artificial intelligence plays a bigger role in game development, issues regarding authorship and creative control will need to be addressed. Although it's still in its early stages, Oasis marks a notable advancement in AI-driven game development. As the technology evolves, we can expect higher resolutions, better frame rates, and more complex game worlds with improved object permanence. There's also potential for AI generation to be integrated into traditional game development processes. This is an exciting milestone at the crossroads of artificial intelligence and gaming. While challenges remain, this technology offers a glimpse into a future where artificial intelligence will play a key role in crafting personalized gaming experiences.

A New Era in Gaming: 'Minecraft' Clone Is Generated by AI in Real-Time - Decrypt

Silicon Valley computer hardware startup Etched released a demo of its new AI-generated game called Oasis, which lets players create and modify virtual environments, effectively building a complex game from scratch in real time. The Minecraft clone, named after the virtual universe created by Ernest Cline for the iconic sci-fi novel and movie "Ready Player One," was created in partnership with San Francisco-based AI developer Decart. It's being billed as the first playable AI-generated game. Etched showcased the Oasis demo in a Twitter (aka X) post on Thursday. In the demo, players could navigate a block-filled environment, moving, building, and destroying terrain and obstacles as they go. "This first version of Oasis is for research and proof-of-concept, showing that AI models can even do this," Decart CEO and co-founder Dean Leitersdorf told Decrypt. "There's no monetization there -- it's all free. It's all just to show that this can be done from a research perspective." Oasis is the brainchild of Harvard graduate Robert Wachen, co-founder and COO of Etched, and Israel Institute of Technology alum Leitersdorf. As Wachen explained, the duo met in 2022 and were inspired by OpenAI's GPT-3, and Wachen and Leitersdorf launched Etched and Decart in 2022 and 2023, respectively. "Simply put, Etched operates like Nvidia, building next-generation hardware, while we're like OpenAI, developing AI models designed to run on that hardware," Leitersdorf explained. "Etched is laying the groundwork, and Decart is building AI-driven experiences on top." Etched and Decart chose to base the Oasis demo on Minecraft because its open-world environment supports dynamic content, a technical challenge that they wanted AI to overcome. Wachen said that, unlike traditional graphics engines, Oasis relies on artificial intelligence to produce each pixel in real time without the use of a game engine. "There are two key components here: the weights, which represent the intelligence of the model, and the descriptor that runs the weights, known as the inference engine," Wachen explained. "This setup allows the AI to create elements in real-time and efficiently handle large numbers of users." According to Leitersdorf, in the past 24 hours, since Decart and Etched came out of stealth and posted the demo on Twitter, interest in Oasis caused a surge in traffic that nearly took down the site. "We were insanely shocked by the scale of the people trying to use the demo," he said. "Hundreds of thousands of people came in, and we had to turn most away." While the Oasis demo was created using Nvidia's H100 Tensor Core GPUs, Etched is developing its own chipset, called Sohu, that the company said would be several times faster than Nvidia's GPU. Getting to this point hasn't been easy, and there was doubt at the beginning that Etched would be successful, Wachen said. "The reality is this did not make sense as a business when we started," Wachen said. "Building a $100 million chip project for a $10 million model, you had to get lucky and see that it was going to blow up -- and after ChatGPT, it started to become more obvious that it would." Etched and Decart are the latest developers looking to bring "Ready Player One" to reality using AI, including the book' author himself. Just last month, Cline's Readyverse Studios launched Promptopia, a tool that allows players to create in-game items through text prompts. Wachen isn't concerned about the competition, and said the code and weights for Oasis are available on GitHub for developers to examine. "We want to share this with the world. We want to show people what's possible, and now we've given them the tools to build their own things," Wachen said. "The hope is in a month or two, someone else will release something even cooler."

Oasis AI and a single NVIDIA H100 GPU has created a playable 'AI Minecraft' at 720p 20 FPS

Oasis AI, from Decart, is described as "the world's first real-time AI world model" for gaming. It offers a real-time playable AI-generated version of Minecraft running on a single NVIDIA H100 GPU to demonstrate its capabilities. It takes user input, from movement to using a tool, to generate the next frame. It's an impressive little tech demo because what you see isn't rendered in the traditional sense that we associate with gaming; however, in its current state, playing Minecraft at a heavily compressed 720p at 20 FPS isn't exactly immersive or engaging. It's more proof of concept - generating entire frames, physics, lighting, and mechanics after being trained on watching hours and hours of Minecraft gameplay. After playing the demo, which you can do via the Oasis AI site, you will see one pretty big flaw with how it works as a game - there is no real object permanence. When you turn around or backtrack, the Oasis AI version of Minecraft often rearranges objects or "renders" a different version of the environment you just saw and interacted with - which can be disorienting. So, it's more of an approximation of Minecraft than the actual game. Of course, Minecraft has been playable on tablets, phones, consoles, and PCs for several years, so it's not meant to be a replacement but a look at the potential of AI-generated open-world environments. And for experiences that aren't limited to being games. "Oasis marks our initial foray into more complex interactive worlds and offers a glimpse of what we call and hope to coin a 'Generative Interactive Experience.'" Decart explains. "This new technology could change many types of experiences by creating interactive videos that users can control in real time. Imagine a world where AI models make entertainment more personalized." What differentiates Oasis AI from similar AI-generated game examples that we've seen is that it can generate each frame with low latency and is optimized to run on a single GPU or multiple GPUs. "This breakthrough stems from innovations across our entire technology stack," Decart adds. "We've developed a model architecture that enables low action-to-frame latency, coupled with proprietary system optimizations targeting specific GPU and server hardware architectures."

AI-generated Minecraft unveiled, showcasing AI's unlimited ability to copy things but worse

AI has been up to some funny things in the last few years. People have declared the supposed merits of generative AI for creating textures, they have managed to get Doom running on a neural network, and, 'any day now', we're supposed to get user prompt-inspired games to play. As you might expect, today is not that day. Decart, an AI company that seemingly popped up overnight, has unveiled Oasis, "the world's first real-time AI world model". Oasis works via frame prediction. An AI model is given huge masses of data, which can then be used to predict the next frame. This is why the tool seems to invent blocks and doesn't seem to have object permanence. It doesn't store data from your world and loses track of things you've done after some time. "Oasis takes in user keyboard and mouse input and generates real-time gameplay, internally simulating physics, game rules, and graphics," Decart says. You aren't playing the game, per se, you are instead playing an approximation of the game. In the announcement for this new model, Decart published where this tech can go, saying: "Simply imagine a world where this integration is so tight that foundation models may even augment modern entertainment platforms by generating content on the fly according to user preferences. Or perhaps a gaming experience that provides new possibilities for the user interaction such as textual and audio prompts guiding the gameplay." That last part suggests that users can generate their own experiences, by heightening stakes on the fly or manipulating the game as they go. However, the current model is much more limited than that. Decart is seemingly generating a version of Minecraft, and a poor pixelated one at that. The end of the trailer for this new engine says "Imagine what AI experiences could look like if everyone had the power to create them" and that's exactly what I have to do as this game was created by Mojang almost two decades ago. If everything is as Decart says here, the idea of AI-generating Minecraft could be quite impressive, but it's important to note this is not even close to the future envisioned. The work required to mimic a game that is widely accessible is different from the ability to effectively imagine and create bespoke experiences based on user prompts. Decart says it wants to reach a point where a user can say "Imagine that there is a pink elephant chasing me down" and one appears, but right now it's struggling to make a look-a-like of a very old game using tons of training data from said game. Millions of hours of Minecraft gameplay, according to MIT Technology review. This is before mentioning whether or not users even want this, the copyright implications of where that data is scraped from, and if the ability to frame all your art around yourself is particularly good for your media diet. This tool was created with Nvidia H100 cards, and if you thought Minecraft was a generally easy game to run, this isn't. It runs at 360p at 20 fps. Though the future of this hardware is proposed to be made with Etched ASICs, Decart's partner here. These will supposedly allow it to run this sort of tech at 4K. Of course, not all uses of this kind of AI in games are intended to just copy other games, and some of it is, in fact, very good for the average gamer. AI-powered Frame Generation can give much better performance in games with the same basic gear, and in theory, also relies on generating future frames based on the previous information presented to it. I thought I would try to give Oasis a go, by accessing the site, and after getting through the waiting time, the whole thing wouldn't load. This isn't to say you can't access it but that I couldn't, and, at this rate, I'd rather just boot up the real thing.

Worlds First Fully AI Generated Game : Oasis

Oasis marks a significant achievement in the gaming industry, offering the first fully AI-generated game experience that adapts in real-time. Unlike traditional game engines, Oasis creates a fluid, interactive world that evolves with each player action. As you explore this innovative game, you experience the capabilities of real-time AI generation, where every frame and interaction is crafted instantly, responding to your inputs with precision. Imagine a game where every move you make, every choice you take, and every place you explore is crafted in real-time by an intelligent system that learns and adapts uniquely for you. This isn't a far-off vision or sci-fi concept; it's the reality brought to life by Oasis, the first fully AI-generated game. For those familiar with pixelated or static game environments, the idea of a dynamic, ever-evolving universe is nothing short of innovative. Oasis doesn't just offer a game; it redefines the gaming experience by making the environment as responsive as a living, breathing entity. At the core of this innovation is a sophisticated mix of advanced technologies working together to deliver an immersive experience like no other. Oasis uses state-of-the-art AI models, including Transformer technology and diffusion models, to generate a world that instantly reacts to your actions. Whether you're breaking blocks, jumping across chasms, or simply exploring, the game world morphs and adapts in real-time, providing a level of interactivity and engagement unmatched by traditional game engines. While the journey to refine this technology presents challenges, its potential to transform the gaming landscape is immense. If you're ready to enter a world where your imagination is the only limit, Oasis is your gateway to the future of gaming. In Oasis, the game world materializes in real-time, eschewing pre-rendered environments. This means every movement, action, and environmental change occurs instantaneously as you engage with the game. Advanced AI models drive this immediate creation, resulting in a gaming experience that is both fluid and responsive. The environment transforms based on your actions, fostering a dynamic and immersive experience that keeps you engaged. The seamless gaming experience in Oasis is powered by a sophisticated combination of Transformer technology and diffusion models. These neural networks and probabilistic models work in concert to generate complex interactive worlds. Transformers efficiently process vast datasets, anticipating player actions, while diffusion models craft realistic and diverse environments. This synergy ensures an engaging and unpredictable game world, offering an unprecedented level of interactivity. The AI model underpinning Oasis learns by observing gameplay, allowing it to simulate intricate and interactive worlds. This machine learning process involves training the model on extensive gameplay data, allowing it to comprehend and replicate complex game mechanics. As a result, the AI generates environments that are not only visually appealing but also rich in interactive possibilities, enhancing the overall gaming experience. Discover other guides from our vast content that could be of interest on Real-Time AI Generation: Real-time processing. Supporting the substantial computational demands of real-time AI generation requires specialized hardware. The Soo chip, developed by Etched DoAI, rises to this challenge. Optimized for Transformer models, the Soo chip delivers superior performance and efficiency compared to traditional GPUs. This specialized hardware ensures Oasis runs smoothly, even with the high processing power demands, making it a critical component of the game's success. In Oasis, AI-powered mechanics allow you to interact with the environment in unprecedented ways. Whether you're moving, jumping, or manipulating objects, every action is driven by AI, creating a responsive and engaging experience. The AI's adaptability ensures the game remains challenging and exciting, offering endless possibilities for exploration and creativity. Despite its new nature, Oasis faces certain challenges, including lower resolution and slower response times compared to traditional games. These limitations stem from the immense computational demands of real-time generation, which can affect visual quality and responsiveness. However, ongoing advancements in AI technology and hardware development are expected to address these issues, paving the way for more sophisticated AI-generated games in the future. The introduction of Decart's Oasis signals a potential paradigm shift in the gaming industry, where traditional game engines could be supplanted by AI-driven systems. This transition promises a more fluid and dynamic gaming experience, with game worlds adapting to your actions in real-time. As AI technology evolves, the possibilities for creating immersive and interactive gaming environments are boundless. Try out Oasis here. Oasis is not alone in exploring the frontier of AI-generated gaming. Projects like Google's Game Eng demonstrate AI's potential in creating interactive gaming worlds. These initiatives highlight the growing interest in AI-driven game development and the potential for AI to transform the gaming industry by offering new and exciting ways to engage with digital environments. As specialized hardware becomes more prevalent, high-quality AI-generated games will become increasingly feasible, offering players unprecedented levels of immersion and interactivity. The continued advancement of AI in gaming technology is set to redefine the boundaries of what is possible, ushering in a new era of dynamic and adaptive game environments. As AI technology continues to evolve, we can expect to see more sophisticated AI-generated games that push the boundaries of creativity and player engagement. Oasis represents just the beginning of this exciting new frontier in gaming, paving the way for a future where games are limited only by the imagination of AI and the players who interact with them.

Decart's AI simulates a real-time, playable version of Minecraft

Decart, an Israeli AI company that emerged from stealth today with $21 million in funding from Sequoia and Oren Zeev, has released what it's claiming is the first playable "open-world" AI model. Called Oasis, the model, which is available for download, powers a demo on Decart's site: a Minecraft-like game that's generated on the fly, end-to-end. Trained on videos of Minecraft gameplay, Oasis takes in keyboard and mouse movements and generates frames in real time, simulating physics, rules, and graphics. Oasis is a part of an emerging category of generative AI models called "world models." Many of these models can simulate games -- but few at frame rates as high as Oasis. I tried the demo out of curiosity, and I'd say it has a ways to go before it's a genuinely fun experience. The resolution is quite low, and Oasis tends to quickly "forget" the level layout -- I'd turn my character around only to see a rearranged landscape. Decart believes, however, that future versions of Oasis, which was optimized to run on Etched's upcoming AI accelerator chips (the demo currently runs on Nvidia H100 GPUs), could generate up to 4K gameplay. "[These] models may even augment modern entertainment platforms by generating content on the fly according to the user preferences," Decart writes in a blog post. "Or perhaps a gaming experience that provides new possibilities for the user interaction such as textual and audio prompts guiding the gameplay."

AI-powered Minecraft runs without a game engine -- game rendered in real-time at a continuous 20 FPS

DecartAI's Oasis AI world model can run a rough approximation of Minecraft with "no game engine, no logic, no code" at 20 FPS and 360p. (Image credit: DecartAI) In the "apparently" post-copyright age of AI PCs, yet another AI company has introduced a shameless playable ripoff of an actual, copyrighted video game -- now it's DecartAI's Oasis world model and Minecraft, with resolution and framerate more characteristic of Nintendo 64 games like Ocarina of Time (20 FPS, 360p) than any modern port of Minecraft. Even fan ports to platforms like the GameCube and Dreamcast run better than this! Add numerous AI hallucinations that render truly complex gameplay unfeasible, including a complete lack of object permanence to the point that even digging a hole drops you back above ground, and NO environmental fixtures are permanent, and one wonders why anyone would want to play a survival-building RPG in these conditions. Oasis, developed in collaboration with Etched, isn't necessarily commercial. In fact, according to comments on Etched Twitter, it's apparently due to be open-sourced. How exactly they're being allowed to open source and distribute code so blatantly ripping off (and training with) an existing video game is anybody's guess. This project is also based on the open-source Minecraft training dataset from OpenAI, Minecraft Video PreTraining (VPT). VPT was trained on 70 thousand hours of IDM-labeled online video. While AI zealots cheer advancements like AI Counter-Strike: GO and Doom as advancements for the gaming space, it seems clear to anybody paying attention that the most affordable, performant, and sensible option for years, if not decades, to come will be actual game engines running on actual hardware. Proudly touting a "game" without a game engine, game logic, or code is perhaps missing the point of what makes games fun -- or even coherent, particularly considering the fact that these projects simply don't exist without real video games to rip off. In other words, no -- with how this technology works fundamentally, requiring training off existing content, you aren't able to simply AI prompt the creation of a truly complex, original game. The closest thing to that is a shameless ripoff of Angry Birds assembled from AI-generated assets. That is still just Angry Birds, though it is a remarkably more coherent experience than these real-time 3D AI world models, which are so prone to disorienting hallucination as to make them feel like hallucinations to play.

This AI-generated Minecraft may represent the future of real-time video generation

The companies acknowledge that their version of Minecraft is a little wonky. The resolution is quite low, you can only play for minutes at a time, and it's prone to hallucinations like the one described above. But they believe that with innovations in chip design and further improvements, there's no reason they can't develop a high-fidelity version of Minecraft, or really any game. "What if you could say 'Hey, add a flying unicorn here'? Literally, talk to the model. Or 'Turn everything here into medieval ages,' and then, boom, it's all medieval ages. Or 'Turn this into Star Wars,' and it's all Star Wars," says Leitersdorf. A major limitation right now is hardware. They relied on Nvidia cards for their current demo, but in the future, they plan to use Sohu, a new card that Etched has in development, which the firm claims will improve performance by a factor of 10. This gain would significantly cut down on the cost and energy needed to produce real-time interactive video. It would allow Decart and Etched to make a better version of their current demo, allowing the game to run longer, with fewer hallucinations, and at higher resolution. They say the new chip would also make it possible for more players to use the model at once. "Custom chips for AI hold the potential to unlock significant performance gains and energy efficiency gains," says Siddharth Garg, a professor of electrical and computer engineering at NYU Tandon, who is not associated with Etched or Decart. Etched says that its gains come from designing their cards specifically for AI development. For example, the chip uses a single core, which it says makes it possible to handle complicated mathematical operations with more efficiency. The chip also focuses on inference (where an AI makes predictions) over training (where an AI learns from data). "We are building something much more specialized than all of the chips out on the market today," says Robert Wachen, cofounder and COO of Etched. They plan to run projects on the new card next year. Until the chip is deployed or its capabilities are verified, Etched's claims are yet to be substantiated. And given the extent of AI specialization already in the top GPUs on the market, Garg is "very skeptical about a 10x improvement just from smarter or more specialized design." But the two companies have big ambitions. If the efficiency gains are close to what Etched claims, they believe, they will be able to generate real-time virtual doctors or tutors. "All of that is coming down the pipe, and it comes from having a better architecture and better hardware to power it. So that's what we're really trying to get people to realize with the proof of concept here," says Wachen. For the time being, you can try out the demo of their version of Minecraft here.