Roblox Unveils Groundbreaking AI-Powered 3D Creation Tools

Roblox Builds Open-Source 3D AI Model, Adds Tech for Faster Game Loading

Roblox is developing a new open-source generative AI tool, dubbed the 3D Foundational Model, the game platform announced at the Roblox Developer Conference on Friday. The model will understand text, image, video, or 3D asset prompts, meaning users can drop in an existing 3D game asset and ask the AI to make something similar, change it, or expand upon it, for example. The model is being trained on 3D data provided by Roblox creators who opted in, the company tells PCMag. Roblox is also using open-source datasets and licensing some datasets to refine the model, which can generate 3D objects or entire scenes. Roblox previously launched an AI assistant tool that can add textures and pull from preexisting assets to assemble scenes for players, while the upcoming 3D model generator will be able to make content from scratch. Over time, Roblox plans to add this model to the assistant tool. Roblox will make its upcoming 3D Foundational Model open source -- and wants it to see wider adoption across the gaming industry. This means third-party game developers could change the model as needed, tuning it for specific purposes. While generative AI tools could help speed up the game development pipeline and make more ambitious projects possible for indie creators, it's also been controversial because of its potential impact on game industry jobs for workers like environment artists, animators, and character designers. There's also been some debate around the value of open-source tech, and whether some open-source AI models from companies like Meta and Google even quality as such. Ultimately, Roblox is trying to build what it calls "4D" generative AI, meaning that the AI can also create interactions between the 3D game elements. This means that animations between objects would be built-in, and the 4D tool could create dynamic environments with interactive objects that can move through game spaces. Roblox has already been developing and implementing numerous AI tools across its game, seeing it as a way for players to more easily make content. The company has already launched over 250 different AI models, from texture and avatar tools to chat translation, to name a few. While these tools can help gamers generate more content, the AI features take some time to create objects and textures, meaning gamers may be waiting for anywhere from half a minute to many minutes for the tool to finish its creation. GenAI tools can also require a lot of computing power, and therefore can suck up shocking amounts of electricity. Roblox also announced some technical improvements to its overall game experience on Friday. Its Harmony game engine tool is being developed on the backend to optimize interactions between a user's CPU, GPU, and RAM on their device, and make changes in response to a player's internet speeds. This means that Roblox game environments may be loading a lot faster, especially on devices like smartphones, in the near future. It's already made changes to better Roblox performance on Android devices, improving average frames per second by 24%, making the gaming experience look and feel smoother overall. "Optimizing for performance is something that permeates everything we do at Roblox," says Tian Lim, Roblox VP of creator and engine product. "Superior performance also empowers creators to experiment with bigger, bolder ideas that push the boundaries of what exists on the platform, from larger maps to servers with more players." In the future, Roblox says it wants gamers to be able to join a 100-player open-world battle royale game -- and have it run seamlessly on a device with just 2GB of RAM. If this actually happens, gamers with older iPhones or budget phones could have some fun.

Roblox shows new 3D AI tool gunning for 'realtime creation integrated with gameplay'

The Roblox Developers Conference is underway, and at the keynote address CEO and co-founder David Baszucki came out all guns blazing. We're gonna have a billion daily users! 10% of all global gaming revenue! We already have your children! OK not the last one, though he might as well have, and you can read all about Roblox's rude health and even ruder goals here. There was an element of pie-in-the-sky about some of the keynote. But one of the genuinely impressive touches came when Baszucki gave a flourish in the direction of AI, which Roblox has been gradually integrating into its creator development tools for years. It has a tool that helps automate creating textures, for example, but apparently we ain't seen nothing yet. Roblox is developing an AI "3D foundational model" that will be open source and create 3D assets through text, video and 3D prompts. Baszucki describes this as an "incubation project" with around 40 dedicated employees, and says they're only three months down the road on it. He then shows an overhead view of a map being created through simple text prompts, then becoming a desert, then prompted into becoming a forest scene (with both a text and image prompt). This is a pre-recorded element of a stage presentation, of course, so take the above evidence of this AI model's capabilities with a pinch of salt. "We've been building generative AI into Roblox for years with tools like Assistant, Texture Generator, and Avatar Auto Setup. But these are just beginning to scratch the surface of what we have planned," said Baszucki. "We see a powerful future where Roblox experiences have extensive generative AI capabilities to power real-time creation integrated with gameplay -- in a resource-efficient way so we can make this available for everyone on the platform." [Unfortunately I can only currently embed the livestream: when the full video is available, I'll replace the below with a timestamped link] The most intriguing promise is that idea of "realtime creation integrated with gameplay." These tools look impressive already (albeit in a stage demo), and Baszucki didn't put a timeframe on it, but if they can get there this is a leap forward in AI functionality for games that has enormous implications. Imagine a virtual D&D game where the world is coalescing into existence as the dungeon master describes it. OK, with Roblox graphics, but still. As if this wasn't enough for the optimists among us, you could even tie-in the AI promises to a more sedate part of Baszucki's presentation. Improvements to the platform's backend and tech stack are not exactly headline news, even if things like improving app launch time, framerate and crash reduction make a big difference to players every day. But Baszucki says the next stage in these "platform efforts" is making it possible for a developer "to host a high-performance, 100-player open world, sports or battle royale-type game on Roblox and have it run on 2 GB RAM devices anywhere in the world at good frame rates. We won't stop until this is possible." I'm still waiting for my flying car, but in the meantime I suppose taking part in 100-player experiences being created on-the-fly sounds totally awesome. It's the kind of feature that, if it wasn't in Roblox, would have most of us fulminating. Whether Roblox can deliver what it says it can, however, remains to be seen.

Roblox is launching a generative AI that builds 3D environments in a snap

Developers are already excited. "Instead of sitting and doing it by hand, now you can test different approaches," says Marcus Holmström, CEO of The Gang, a company that builds some of the top games on Roblox. "For example, if you're going to build a mountain, you can do different types of mountains, and on the fly, you can change it. Then we would tweak it and fix it manually so it fits. It's going to save a lot of time." Roblox's new tool works by "tokenizing" the 3D blocks that make up its millions of in-game worlds, or treating them as units that can be assigned a numerical value on the basis of how likely they are to come next in a sequence. This is similar to the way in which a large language model handles words or fractions of words. If you put "The capital of France is ..." into a large language model like GPT-4, for example, it assesses what the next token is most likely to be. In this case, it would be "Paris." Roblox's system handles 3D blocks in much the same way to create the environment, block by most likely next block. Finding a way to do this has been difficult, for a couple of reasons. One, there's far less data for 3D environments than there is for text. To train its models, Roblox has had to rely on user-generated data from creators as well as external data sets. "Finding high-quality 3D information is difficult," says Anupam Singh, vice president of AI and growth engineering at Roblox. "Even if you get all the data sets that you would think of, being able to predict the next cube requires it to have literally three dimensions, X, Y, and Z." The lack of 3D data can create weird situations, where objects appear in unusual places -- a tree in the middle of your racetrack, for example. To get around this issue, Roblox will use a second AI model that has been trained on more plentiful 2D data, pulled from open-source and licensed data sets, to check the work of the first one. Basically, while one AI is making a 3D environment, the 2D model will convert the new environment to 2D and assess whether or not the image is logically consistent. If the images don't make sense and you have, say, a cat with 12 arms driving a racecar, the 3D AI generates a new block again and again until the 2D AI "approves." Roblox game designers will still need to be involved in crafting fun game environments for the platform's millions of players, says Chris Totten, an associate professor in the animation game design program at Kent State University. "A lot of level generators will produce something that's plain and flat. You need a human guiding hand," he says. "It's kind of like people trying to do an essay with ChatGPT for a class. It is also going to open up a conversation about what does it mean to do good, player-responsive level design?"