AI-Powered NPCs: The Dawn of Unscripted Gaming Conversations

Tech 24 - Meta and Fortnite push AI video game characters into the mainstream

We're used to video game characters repeating the same lines of scripted dialogue. But artificial intelligence is now allowing players to have unique, unscripted conversations with the characters they meet. In Tech 24, we look at this experimental tech and its path to the mainstream. Hobbyists have turned classic games like The Elder Scrolls: Skyrim and Fallout 4 into AI-powered playgrounds. Thanks to the downloadable mod Mantella, you can talk to non-playable characters (NPCs) with a microphone, and they'll respond in character. The technology is powered by the groundbreaking capabilities of AI models released by companies like Google and Meta, and opens up whole new ways to interact in games. Some players find it so immersive that they develop deep connections. One forum user, for instance, shared his experience with a Skyrim character called Uthgerd, with whom they claimed to have conceived a child. "We role played her water breaking and her giving birth which was wild ... SkyrimVR is definitely therapy ... when I play it I am instantly put in a good mood." Testing the experience on Tech 24, we found that it required a lot of investment and patience to get it running smoothly. Characters would often misunderstand or respond with a significant delay. But for those willing to invest time and money, it's easy to see how it can become an engrossing experience. Paying for better AI models can make the characters quicker and more interactive, even recognising things they see around them with computer vision. Downloading better voice models can make them sound more natural. And there's the option of putting on a virtual reality (VR) headset to get fully immersed. Under the alias 'Brainfrog' on YouTube, Eddie publishes videos of his adventures talking to AI NPCs in VR. "You're creating your own world with your own relationships and I found myself building genuine relationships with these characters," said Eddie, "There are a substantial amount of issues and times where you run into this stuff and you're just like, oh, I'm speaking to a dumb computer, but there are some moments that everything aligns and it's really, really breathtaking." The tech might be expanding beyond the realm of the modding community and hitting mainstream games sooner than you think. Just this week, developers for Meta's Horizon Worlds VR experience were let loose on tools letting them design characters who can respond with AI. The hugely popular battle royale game Fortnite introduced an AI-voice Darth Vader earlier this year which proved wildly popular, and developer Epic Games is promising custom AI characters by the end of this year. The next boundary that AI labs are pushing is virtual characters that don't just react verbally, but interact physically with the space around them. The Financial Times newspaper reported this Sunday that Elon Musk's xAI is joining this race against the likes of Meta and Google, poaching specialists from AI behemoth Nvidia. But while it seems the tech giants are going full steam ahead on supercharging games with AI, there are many dissenting voices within the industry. Michael Douse, head of publishing at Larian Studios which developed Baldur's Gate 3, said this week that AI could not solve the biggest problem of the games industry, "leadership and vision." "What this industry needs is not more mathematically produced, psychologically trained gameplay loops," he posted on X, "Rather more expressions of worlds that folks are engaged with, or want to engage with."

The era of unscripted AI game characters has officially begun

Hobbyists are using artificial intelligence to enable unscripted conversations with characters in video games. The technology, powered by models from Google and Meta, is now being integrated into mainstream platforms, moving beyond niche modifications created by players. In the enthusiast community, players have been utilizing a downloadable modification, or mod, known as Mantella to transform classic games such as The Elder Scrolls: Skyrim and Fallout 4. This tool allows players to use a microphone to speak directly with non-playable characters (NPCs), which then generate unique, unscripted responses that remain consistent with their established in-game personalities. The functionality relies on advanced AI models released by technology companies, including Google and Meta, which process the player's speech and formulate new lines of dialogue in real-time. The level of immersion provided by this technology has led some players to report developing deep personal connections with the AI-enhanced characters. One user on a community forum detailed an experience with Uthgerd, an NPC from Skyrim. The player described an extended role-playing session with the character. "We role played her water breaking and her giving birth which was wild ... SkyrimVR is definitely therapy ... when I play it I am instantly put in a good mood," the user wrote, illustrating the profound effect the interactive experience had. A test conducted by Tech 24 confirmed the viability of the technology, producing several conversations with NPCs where the AI-generated dialogue was difficult to distinguish from the game's original, pre-written script. The test also revealed that achieving a smooth experience required a significant commitment of both time and patience from the user. Characters frequently misunderstood player queries or exhibited considerable delays before responding. Despite these technical hurdles, the potential for an engrossing experience was evident for those willing to make the necessary effort. Users can improve the performance and realism of these interactions through several upgrades. Paying for access to higher-quality AI models can make characters respond more quickly and interactively. Some advanced systems even incorporate computer vision, allowing NPCs to recognize and comment on objects within their virtual environment. Further enhancements include downloading superior voice models to make the characters sound more natural, and using a virtual reality (VR) headset to achieve a state of full immersion within the game world. A YouTuber named Eddie, who operates under the alias "Brainfrog," documents his experiences talking to AI-powered NPCs in VR. He described the process as one of co-creation with the technology. "You're creating your own world with your own relationships and I found myself building genuine relationships with these characters," Eddie stated. He acknowledged the system's limitations, noting that technical glitches can break the illusion. "There are a substantial amount of issues and times where you run into this stuff and you're just like, oh, I'm speaking to a dumb computer, but there are some moments that everything aligns and it's really, really breathtaking." This technology is beginning to transition from the modding community into mainstream gaming. Developers for Meta's Horizon Worlds, a VR social platform, were recently given access to tools that let them design characters capable of AI-driven responses. In another example, the battle royale game Fortnite introduced an AI-voiced version of the character Darth Vader earlier this year, which proved to be highly popular among its player base. The game's developer, Epic Games, announced plans to offer custom AI characters to players by the end of this year. AI research labs are also working to advance the capabilities of virtual characters beyond verbal communication. The next objective is to create characters that can physically interact with the virtual spaces they inhabit. A report from the Financial Times indicated that Elon Musk's company, xAI, is entering this field, competing with firms like Meta and Google. To support this effort, xAI has reportedly been hiring specialists from the AI technology company Nvidia. While technology companies are advancing AI integration in games, some figures within the video game industry have expressed dissent. Michael Douse, the head of publishing at Larian Studios, the developer of Baldur's Gate 3, stated this week that AI cannot resolve what he sees as the industry's most significant issue: a need for "leadership and vision." In a post on the social media platform X, Douse elaborated on his position. "What this industry needs is not more mathematically produced, psychologically trained gameplay loops," he wrote. "Rather more expressions of worlds that folks are engaged with, or want to engage with."