Study claims 95% of players enjoy AI NPCs, but limitations spark debate over generative AI

Company that makes generative AI-powered NPCs reports that 95% of players enjoy their generative AI-powered NPCs

Participants ate up Dead Meat and other gen AI powered detective games. Hands up if you've ever thought the following: "Sure, Elden Ring is pretty great, but what it really needs is the mournful, silent creature I'm fighting to say something FromSoft didn't write like 'come at me, ye tarnished dickhead'" during a tense encounter. If you do have your hand raised, I imagine you likely already work for Meaning Machine, the company who still proudly boast their 'Battle Banter' tool on their website, a "Game Conscious™" system that "gives NPCs the ability to deliver reactive dialogue in response to real-time gameplay." You spin. A gargoyle with no voice box calls you out for spinning. That sort of thing. Meaning Machine is also the developer behind Dead Meat, a noirish murder mystery detective game which, from the limited footage I've seen, doesn't immediately strike me as the sort of slop you might expect. It's broadly stylish in a classic cartoon sort of way. The idea is you'll interrogate NPCs by asking your own questions using either microphone or keyboard, and they'll give custom responses in slightly offputting AI voices. You know the type. They sound adjacent to humans, but there are uncanny wrinkles in the delivery that just make it all seem off. Still, it would seem I'm an outlier here, at least compared to the 68 people who recently played Dead Meat at a study Meaning Machine conducted with the University Of Bristol, via Gamesindustry.biz. 95% of participants "found the experience enjoyable", 97% found it rewarding, and 75% felt the games let them "express themselves or make meaningful choices". "This research helps to ground what is otherwise quite an emotionally charged debate about AI in games," said lead researcher Dr Richard Cole. "It does this by putting the player at the heart of the debate -- asking what they feel about AI-powered experiences in practice, not just in theory." "Players kick back at AI that is taking away from creativity," said Meaning Machine's co-founder Thomas Keane. "But when AI is used to power totally new types of interactive experience, then it's a very different story." I suppose it is technically possible that someone could say those words in that order and not come across as so smug I want to throw myself into a lake, but not someone who's got a horse in the race. While the full paper will be out at the end of the year, the available study (which frequently uses the same branding font as Dead Meat itself, like all good academic research) says that the 68 participants played Dead Meat for 20 minutes each, and acknowledges that's a limiting factor: "We don't yet know how players will respond to a longer play session, although we do know from our sample that they want to do this". I can imagine that, in that timeframe, you could approach the games in the study like interactive novelty exhibitions, and they wouldn't hold up too badly. That's not especially convincing when it comes to these NPCs providing a comparable experience to one crafted by actual writers in the long term, though. It's also not clear whether the participants knew exactly what the study was going to entail before starting. It looks like players knew about the gen AI NPC's before reporting they enjoyed their time. They were presumably fine with gen AI games on principle, in other words. That's a far cry from assuming that at least some players won't just reject these games outright. I quite enjoy taking absurdly long showers, but I know it's probably not great for the environment, so I try to avoid it. And my shower never convinced a studio to lay off a bunch of staff so it could take all their money.

Study Conducted by a Studio Making a Game With AI-Powered NPCs Claims 96% of Players Enjoy AI-Powered NPCs

Dead Meat is, for the moment, a very new kind of video game that could potentially be commonplace, depending on how Generative AI (GenAI) continues to be welcomed (or unwelcomed) into the video game industry. It rides that thin line because Dead Meat is a game where you speak to AI-powered NPCs as you investigate a murder mystery. It's developed by Meaning Machine, a studio that says "AI characters and quests have the potential to revolutionize interactive storytelling" on their website, adding that it can only work "if human authors retain creative control of those stories. Because without the human author, there is only slop." Meaning Machine goes on to discuss its "radical new model for wrangling AI - to deliver meaningful stories that balance human authorship with emergent properties." That is seemingly what Dead Meat is. A game with a story that has a human author, where GenAI-powered NPCs give custom responses to players as players try to work through the game's mystery. It's an interesting concept, to say the least, and the studio, through the University of Bristol, recently put Dead Meat to the test in a new research study. The university conducted the study (the first part of which is available now, with a longer paper to come later) by asking players to play Dead Meat and Blood Will Out, two games in development at Meaning Machine, for 20-minute play sessions followed by semi-structured interviews. The first part of the study focused on Dead Meat, and had a sample size of only 68 players, 31 who identified as male, 31 who identified as female and seven who identified as non-binary, and asked them to share how they felt about the NPCs and their thoughts on Dead Meat after their short time with the game through interviews and giving different criteria ratings between 1-10. It should be noted, as it is in the headline of this article, that this is a company with a vested interest in people liking GenAI-powered NPCs in video games. Yes, the study was publicly funded and not funded by Meaning Machine, but it was still done in collaboration with the studio. The small sample size and the limited time each person had with the game should also be heavily noted, the latter of which the study does call out as a limiting factor. With all of that said, according to the University of Bristol's research, 96% of players found the experience of playing Dead Meat to be enjoyable, rating it "6 or above." "Virtually every player didn't just enjoy the game, they actively liked it. 90% of players rated Creative Freedom at 5 or above, 56% at 6 or above. 87% of players rated Engrossment at 5 or above," the study reads. "I did find it really rewarding just like making my own questions up for once," said one participant. "The freedom can be very exhilarating at first, and it feels immersive, and it feels cool. And it can also feel overwhelming," said another, who also began to get at another point the study makes, which is that participants also wondered what about their conversations with these NPCs was actually meaningful. Could a lot of these results be more about the shortened playtime and the novelty of it? Sure, that's one possibility, and to the research's credit, it does acknowledge that as a limitation. That also doesn't mean that these responses should be dismissed offhand. It is a potentially interesting use of GenAI and AI in video games. Does a technology like this actually have the potential that Meaning Machine claims it does? Of course, we can't know for sure, but the majority of 68 people in Bristol who, for all we know, could have already been in support of GenAI tech and not outright against it, as we've seen many are, certainly does not sound like proof of anything revolutionary.