Sony's AI Prototype of Horizon's Aloy Sparks Controversy and Concerns in Gaming Industry

Sony is testing AI for Horizon Forbidden West and fans hate it - Softonic

If they were expecting a different kind of reaction, they are completely disconnected from what is happening in the world A leaked video has revealed an artificial intelligence version of the character Aloy from Horizon Forbidden West, having a conversation with Sharwin Raghoebardajal, director of software engineering at Sony Interactive Entertainment. This material, now removed due to copyright, showed Aloy interacting in an apparently natural way, thanks to a system that uses OpenAI's Whisper for voice conversion, OpenAI and Llama for interaction, and facial animations developed in-house by Sony. The system, which currently works on PC and consoles, has been described as a "rapid prototyping project" by Raghoebardajal. Despite its advancements, the interaction showed a still poor quality, being rated by many on YouTube as "creepy" and "stale". The criticisms focused on the video production and an expressed desire for Sony to redirect its efforts towards other projects, such as a remaster of Bloodborne. In addition to the negative reaction to the video, comments have also focused their concerns on the long-term impact of AI on the video game industry, especially regarding the relevance of voice actors in future productions. Although Sony is not the only company working on similar technologies, others like Ubisoft and Nvidia are also exploring AI applications in their developments, a trend that presents both promises and challenges. As AI begins to take a more prominent role in video game development, there are fears about the decline in the quality of human performances, which could lead to a more cost-effective approach at the expense of a less enriching gaming experience. Despite this, many argue that there is a place for the improvement of artificial intelligence in video games, suggesting that the conversation about its use should be careful and thoughtful.

Sony's made an AI-generated Aloy. Does it not see the irony?

Video game companies seem much more keen than gamers to add more generative AI to games. Now it turns out that Sony is experimenting with fully fledged AI-generated characters. For its first test - or at least the first we've seen - Sony picked perhaps the most inappropriate subject: Ahoy from the Horizon series. The leaked video makes me wonder a) if Sony realises how bad the results look, and b) how did it fail to see the irony in using a character from a game franchise that's all about AI taking over and destroying humanity using predatory machines? The video of Sony software engineering director Sharwin Raghoebardajal experimenting with a AI-driven version of Aloy has been circulating online after it was leaked to the Verge. The video was removed from YouTube after Muso, a copyright enforcement company that counts Sony as a client made a complaint, but it's already been replicated across other platforms. Sharwin asks AI Aloy questions about her life before he starts to play Horizon Forbidden West with the AI-generated character. In the game, she describes the surroundings, including whether she sees enemies and how to deal with them - allusing AI-powered synthesised voice and facial movements. The tech used for the demo apparently included OpenAI's Whisper for speech-to-text, GPT-4 and Llama 3 for decision-making alongside Sony's own Emotional Voice Synthesis (EVS) for speech generation and Mockingbird for facial animation. The test shown was done on PC, but Sony is reported to have also experimented with running at least parts of this system on PS5 hardware. Sharwin clarifies that the character was only a prototype created with Guerilla Games to test the tech, but fans are already dismayed. Predictably, the AI Aloy responds in a monotone voice that sounds like text-to-speech software, and her reactions are stiff and robotic. "Creepy," "ghastly," "rancid," and "cursed" were some of the words that came up in the commends on YouTube before the video was removed. One user pleaded with Sony to "please work on literally anything else." Like maybe the Bloodborne remaster? Nvidia has also teased tech that could create AI-generated NPCs that players would be able to talk to in games. It demonstrated Ace at Gamescom 2024, and created the playable tech demo Covert Protocol with Inworld AI. Microsoft has been working with the same company with the aim of bringing AI characters to Xbox and also has its Muse AI model for gameplay generation. Characters that can talk completely freely could transform video games and open up new possibilities, but the Sony's video shows that a lot of work is needed to make the quality acceptable. There's also the issue of the threat to voice actors. "Eventually I have a feeling the suits see this as a replacement for programmers and artists and motion capture. Essentially the human factor eliminated," one person wrote about the video on X. That's the kind of dystopian future that the Horizon series envisioned.

Future PlayStation games could have AI-powered characters, if this leaked prototype of Aloy is anything to go by

This could be an hint at the future use of AI in PlayStation games Future PlayStation games could have generative AI-powered characters if this leak is anything to go by. A video posted by X (formerly Twitter) user Oliver Darko shows what appears to be leaked footage of a presentation by Sharwin Raghoebardajal, a director of software engineering at Sony Interactive Entertainment, discussing work on an AI prototype as part of Sony's PlayStation Studios Advanced Technology Group. The footage (sadly, it's not available to embed) showed Raghoebardajal interacting with an AI-powered Aloy from the Horizon series; it would appear this prototype has been created with the help of developer Guerrilla Games. Raghoebardajal asks this AI Alyo "How are you?", and after a pause, she responds, complete with accurate facial moments, that she's ok but is "dealing with a sore throat". The interaction feels a tad stifled and a little 'uncanny valley' but demonstrates generative AI in action. The conversation then evolves into Raghoebardajal asking about the premise of the Horizon series and - *spoilers for Horizon Zero Dawn* - how Aloy feels about being a clone. She responds it's a unique feeling. The Verge got access to the full video and repeated that the prototype uses OpenAI's Whisper speech-to-text tech, Sony's own Emotional Voice Synthesis to generate a somewhat realistic voice, and large-language models GPT-4 and Llama 3 to power AI Aloy's decision-making. While this is very much a prototype and has some clear obstacles to overcome, like faster responses and less stilted speech, it's an interesting demonstration of how AI tech could be used in games. This is just a glimpse of what is possible," said Raghoebardajal, The Verge reported. The natural reaction to this tech would be a concern that generative AI could be used to potentially replace voice actors and writers in the gaming industry. While I think that's a legitimate concern worthy of deeper discussion and scrutiny, from this demo it doesn't seem like generative AI is anywhere near up to the task of replacing the quality of speech and narrative delivered by human development teams, certainly not for AAA games. However, much like Horizon Call of the Mountain was used as a high-end showcase for all the features of PSVR 2, there's scope that the same approach could be used to create a spin-out game or virtual experience using AI and the Horizon setting. I could imagine a short-form experience where you visit a handful of small hubs across both Horizon games and interact with their inhabitants, who use generative AI tech to answer questions and serve up observations. Equally, given the extensive lore in the Horizon universe, I could see an AI-powered Aloy being used as some form of in-game or app-based companion to serve up information about certain parts of the world or how to tackle some of the robotic enemies. I know from my experience taking on some of the larger dinosaur-like machines in Horizon Forbidden West I could've done with someone or something directing me on what to target instead of me speedily scanning for vulnerable parts while desperately avoiding unnervingly accurate, health bar-deleting attacks. Of course, Sony appears to be very much at the experimental stage here, and I doubt we'll see a deluge of AI in PS5 games anytime soon. But I'd also be surprised if AI wasn't used in PlayStation games further down the line, say when the PS6 arrives.

Sony hides evidence of AI being used to generate iconic PlayStation characters

TL;DR: A recent leak reveals Sony is using AI to develop iconic PlayStation characters, resulting in a noticeable decline in quality. Sony appears to be experimenting with feeding artificial intelligence systems assets from its staple franchises and asking it to recreate iconic characters from that franchise. A new leak has revealed that Sony is, at the very least, looking into AI-generated characters from its staple franchises, with the leak being shared by The Verge that shows Horizon Forbidden West's Aloy being recreated with some kind of AI tool. The video, which has since been removed from YouTube due to copyright infringement, has been captured as the above GIF showed the iconic character responding to someone asking her questions, and the quality is clearly indicative of AI-generated content. As you can probably imagine, the quality of the video is far below that of traditionally developed content, but according to reports, Sony celebrated what the AI produced in a behind-the-scenes presentation that has since gone public. Sharwin Raghoebardajal, the director of software engineering at Sony Interactive Entertainment, said the footage of Aloy is merely a prototype that has been developed alongside Guerrilla Games. Aloy responding to questions While this is just a prototype, Sony working on AI-generated processes raises a lot of eyebrows by principle, especially considering the current climate of AI-generated content making its way into video games. The topic is already contentious, but it appears there will be no putting this genie back in the bottle, it's just a matter of how and when it's used. "This is just a glimpse of what is possible," says Raghoebardajal Notably, the YouTube video is no longer available due to a copyright claim lodged by Muso, which lists Sony Interactive Entertainment (PlayStation) as a client.

Sony is experimenting with AI characters

It appears that Sony is taking a big step forwards with its AI investment, as it looks to experiment with AI characters. A prototype of Aloy from the Horizon games has been shown in a video doing the rounds on the internet. The video was given to The Verge by an anonymous sender, before it was taken down from YouTube thanks to a copyright claim from Muso, a company that advertises Sony as one of its clients. The video has made its way to Twitter/X however, and more people are seeing it. The irony of a character whose world was ruined by AI-infused robots being the prototype for a new AI video game character doesn't appear to be lost on the internet. Still, as this remains nothing more than a prototype, we don't know how much Sony is pushing for this technology, or if it was just an idea shown to garner interest. Either way, as AI remains one of t he most controversial topics within gaming today, we doubt that this video is going to be pass by without meeting its fair share of backlash.

Sony demos an AI-powered PlayStation character

A since-deleted video shows Aloy from Horizon Forbidden West. Sony has used AI to imbue a PlayStation character with the ability to converse with a player, The Verge reports. A source shared a since-deleted video with the publication featuring an AI-powered version of Aloy from Horizon Forbidden West. Engadget viewed the video before it was pulled. In the demo, Aloy can hold a conversation with voice prompts during gameplay, and respond to questions with a synthesized voice and facial movements. It starts with a delay from Aloy, after being asked how they are, followed by the character stating they are "managing alright, just dealing with a sore throat," a weird aside. The technology appears to work both within a controlled demo as well as the entire Horizon Forbidden West game. During gameplay, Aloy answers queries about where they are and what the time is, though Aloy states it's afternoon and hot, rather than a specific time. After the game, Aloy returns to interview mode and gives an overview of what just happened. The creepy bit, at least to me, came when Aloy is prompted to say goodbye to the audience and thank everyone. The character does, but the robotic voice creates an eerie vibe. Sony used a few different models to create this prototype, including GPT-4 and Llama 3 for powering decision making and conversation, along with OpenAI's Whisper allows for speech-to-text. In the demo, Sony reportedly said it is using its own Emotional Voice Synthesis (EVS) system and Mockingbird technology for speech generation and audio to face animation, two technologies that the company hasn't talked about much yet publicly. NVIDIA and Microsoft have been working on similar technology. The former has used ACE, its suite of technology for enlivening game characters with AI, to create AI-powered conversations for NPCs. Engadget's senior editor, Devindra Hardawar, was far from impressed, stating in January, that "seeing several NVIDIA ACE demos back-to-back made me genuinely sick to my stomach."

Sony is experimenting with AI-powered PlayStation characters, ironically chooses Horizon's Aloy to demonstrate

A new leak has revealed PlayStation maker Sony is working on AI-driven characters, including Aloy from the Horizon series, which seems like a somewhat ironic choice given she is a character growing up in a post-apocalyptic world overrun by rogue AI-controlled machines. The leaked footage shows Sharwin Raghoebardajal - who serves as a director of software engineering at Sony Interactive Entertainment, working on video game technology, AI, computer vision, and face technology for Sony's PlayStation Studios Advanced Technology Group - interacting with the AI Aloy. AI-oy? In the footage, Raghoebardajal asks Aloy a number of questions, such as "how are you", to which the character replies with AI-powered voice and facial movements that she is ok, but "dealing with a sore throat". When questioned further, AI-oy then explains some Horizon lore. The demo shown in the leaked footage is a prototype which has been developed alongside Horizon developer Guerrilla Games, Raghoebardajal says. "This is just a glimpse of what is possible," adds the Sony employee. As for how it has been created, The Verge reports this demo uses OpenAI's Whisper for speech-to-text, as well as GPT-4 and Llama 3 for decision-making, and Sony's proprietary Emotional Voice Synthesis for speech generation. Aloy's facial animations, meanwhile, are powered by Sony's Mockingbird technology. Eurogamer has reached out to Sony for further comment. AI remains a hot topic in the industry and several developers have flirted with the technology. Last month, Microsoft trumpeted the announcement of Muse, a new "generative AI breakthrough" designed to aid "gameplay ideation". Also in February, Take-Two Interactive boss Strauss Zelnick opened up about his views on AI, calling the concept of artificial intelligence "an oxymoron". Meanwhile, the Microsoft-owned Activision recently came under fire for its use of generative AI, admitting some in-game content in Call of Duty has been created using the technology. For more on the subject be sure to check out Ed's feature: How AI is changing video game development forever.

'Creepy,' 'ghastly,' 'rancid': Viewers react to leaked video of Sony's AI-powered Aloy

The video of Aloy hooked up to OpenAI and various other systems was not meant for public consumption, and you can see why. The Verge has shared a leaked Sony PlayStation video demonstrating an AI-powered version of Aloy from Horizon Forbidden West taking part in a conversation with Sharwin Raghoebardajal, a director of software engineering at SIE. The video, which has since been taken down under a copyright claim, featured Aloy responding to questions from Raghoebardajal and showing off a segment of gameplay, after which the character described everything that happened in the clip. OpenAI's Whisper is used for the voice-to-text, OpenAI and Llama handle the conversation and decision-making, and the facial animations come from Sony's own internal Mockingbird system. The system seen in the video is running on PC, but Raghoebardajal says they've been able to get Whisper and Mockingbird running on consoles (presumably PS5) "with relatively little overhead." "Nothing is scripted," Raghoebardajal says in the video. "You can ask her anything, and she'll answer [as] Aloy." It's not great, to be blunt. The interaction is stilted and awkward, and it's all very uncanny valley: It's very clearly a machine spouting off text-to-voice search results, and there's nothing organic or "natural" about it. Which is probably to be expected: Raghoebardajal says the initial demo was "just a quick, fun prototyping project," while the version seen in this video was bolstered by a few weeks of extra work for closed-door demonstrations at the Sony Technology Exchange Fair. The presentation is "just a glimpse of what is possible," Raghoebardajal says, and nowhere near a final product, or even meant for public consumption. Despite those caveats, the reactions to the video on YouTube are not hugely positive. "Creepy," "ghastly," "rancid," and "cursed" are amongst the adjectives used to describe the video; one user begged Sony to "please work on literally anything else," while a couple others want to know -- you can probably see this coming -- why Sony is horsing around with this instead of making the Bloodborne remaster. Not all of the complaints are entirely serious, of course, although the person noting the grim irony of doing an AI demo in a game in which AI was responsible for the annihilation of humanity makes a fair point. But there are also concerns expressed about the long-term impact of this sort of work on game development, particularly with regard to the role that voice actors will play in future productions. Sony is far from unique in working on this sort of thing. Ubisoft and Inworld AI showed off their own AI-powered NPC creations at GDC in 2024, for instance, and Nvidia is working on "Co-Playable Character" technology in partnership with other developers including Krafton -- not exactly the same thing, but broadly similar in the sense that it's AI stuff being shoehorned into videogames. None of it has added up to much at this point, and as anyone who futzed with config files to improve the godawful NPC pathfinding in the original Baldur's Gate can tell you, there's definitely a place for better AI in videogames. But the rise of AI in game development has a potentially dark underbelly and there's a growing feeling of inevitability to it: I don't think AI will ever replicate the genuine human performances we've enjoyed in games like Baldur's Gate 3 or Hellblade: Senua's Sacrifice, but it's also presumably a whole lot cheaper (you don't have to pay a machine and they never need time off) and sooner or later that's going to start figuring into the conversation. I've reached out to Sony for comment on the video and will update if I receive a reply.

PlayStation is testing AI-driven characters, and I'm not a fan

If you've been dreaming about having a more in-depth conversation with one of your favorite gaming characters, it seems that Sony is working on making that happen. A leaked video shows Aloy, the main character from Horizon Forbidden West, having a conversation with the player and explaining her background. I've seen the video, and while I'm impressed with the advancement of AI, I'm not sure how I feel about the result so far. A leaker sent a video to The Verge, showcasing a PlayStation prototype where Sharwin Raghoebardajal, Sony's director of software engineering, speaks to Aloy. It's an odd thing, given that you play as Aloy in Horizon Forbidden West, but the overall impression from the video is even weirder. Recommended Videos Raghoebardajal asks Aloy to tell him more about the fact that she's been looking for her mother. The dialogue flows naturally even though the responses are powered by AI, but Aloy's voice and facial expressions are a give away for the fact that this is all generated and not made by developers. It dips into uncanny valley territory. Please enable Javascript to view this content A bit of the video follows below, but be careful -- it does spoil some Horizon Forbidden West storylines. Leaked footage reveals Sony is testing AI-driven characters in Horizon Forbidden West Upon the initial leak, it was unclear whether the video was legitimate or not, although it certainly seemed to be that way. The Verge said that the video was nothing more than a prototype that the company developed with Guerrilla Games; the idea was to show off the technology at Sony, so this wasn't meant to be seen by outsiders. The demo features OpenAI's whisper model to power speech-to-text; meanwhile, GPT-4 and Llama 3 are said to have been used for the conversations. There's now been an update that gives it all more weight, though. The original leaked video has since been taken down by Muso, which, as The Verge explains, is a copyrights enforcement company for Sony Interactive Entertainment. That covers PlayStation. It's unclear when, or if, Sony will start using AI-driven characters in actual games. The company isn't the first to experiment with using AI to power character interactions. Nvidia has shared multiple demos of Ace, its suite of AI-generated NPC tools, and Ubisoft and Tencent are starting to use it.

Sony is experimenting with AI-powered PlayStation characters

Tom Warren is a senior editor and author of Notepad, who has been covering all things Microsoft, PC, and tech for over 20 years. Sony is working on a prototype AI-powered version of at least one its PlayStation game characters. An anonymous tipster has shared an internal video from Sony's PlayStation group with The Verge that demonstrates an AI-powered version of Aloy from Horizon Forbidden West. The video is narrated by Sharwin Raghoebardajal, a director of software engineering at Sony Interactive Entertainment who works on video game technology, AI, computer vision, and face technology for Sony's PlayStation Studios Advanced Technology Group. Raghoebardajal demonstrates an AI-powered version of Aloy that can hold a conversation with a player through voice prompts during gameplay. Aloy can be seen responding to queries with an AI-powered synthesized voice and facial movements, both in a demo setting and within the full Horizon Forbidden West game. Raghoebardajal makes it clear this is just a prototype that has been developed alongside Guerrilla Games to demonstrate the technology internally at Sony. The technology demo uses OpenAI's whisper for speech-to-text, and both GPT-4 and Llama 3 for conversations and decision making. Sony's has its own internal Emotional Voice Synthesis (EVS) system that it uses for speech generation, according to Raghoebardajal, and audio to face animation is powered by Sony's own Mockingbird technology. While the demo is seen running on PC, Sony has also experimented with running parts of this technology directly on PS5 consoles with "little overhead," according to Raghoebardajal. Sony first demonstrated this internally a year ago, before showing a more advanced version behind closed doors at its Sony Technology Exchange Fair (STEF) in Tokyo in November. "This is just a glimpse of what is possible," says Raghoebardajal. This tech demo doesn't answer the more obvious questions over whether it even makes sense to talk to Aloy when you're supposed to be playing as her in Horizon Forbidden West, or the impact such a technology could have on voice actors and game developers. Nvidia has been working on its own similar technology for AI-powered NPCs in games, where you speak freely to video game characters. Ace, as Nvidia calls it, has been demonstrated several times throughout 2024, and the GPU maker has even worked with Inworld AI to create Covert Protocol, a playable tech demo of Nvidia's AI game technology. Microsoft has also been partnering closely with Inworld AI to eventually bring AI characters to Xbox, allowing game developers to use generative AI characters, storylines, and more. Microsoft has also created its own Muse AI model that generates gameplay, designed initially for game developers to create a game environment for prototyping and ideating. There is already a lot of fear among game developers and studios about how AI could impact the creative process of making video games, particularly when so many layoffs are impacting the gaming industry. Almost half (49 percent) of 3,000 respondents to a Game Developers Conference (GDC) 2024 survey said that generative AI tools are currently being used in their workplace, with 31 percent saying they personally use them. Sony's experiments with AI-powered PlayStation characters are bound to generate more conversation around the role of AI in video game creation, just ahead of GDC next week in San Francisco. We reached out multiple times to Sony for a comment on this leaked internal video, but the company didn't respond in time for publication.

Leaked Internal Sony Video Uses Horizon's Aloy to Show AI-Powered PlayStation Character Prototype - IGN

A leaked internal video has revealed Sony is experimenting with AI-powered PlayStation characters. The Verge reported on the emergence of an internal video allegedly created by PlayStation Studios' advanced technology group that uses Aloy from the Horizon games to demonstrate AI-powered game characters. The video was subsequently pulled from YouTube following a copyright claim from Muso, an internet enforcement company The Verge said lists Sony Interactive Entertainment as a client. This suggests the video is legitimate. IGN has asked Sony for comment. In the video, as reported by The Verge, Sony Interactive Entertainment director of software engineering, Sharwin Raghoebardajal, has a conversation with an AI-powered Aloy via voice prompts and AI-generated speech and facial animations. The speech recognition is via OpenAI's Whisper for speech-to-text, the conversational AI via GPT-4 and Llama 3 for dialogue and decision-making, the speech synthesis via Sony's Emotional Voice Synthesis (EVS) system, and the facial animation via Sony's Mockingbird technology. In the tech demo, Raghoebardajal asks Aloy how she's doing, to which she responds: "Hello, I'm managing alright. Just dealing with a sore throat. How have you been?" The voice heard coming from Aloy's mouth is not that of voice actor Ashly Burch, who famously plays the character in the video games. Rather, it is a robotic voice similar to the kind you get from any of the thousands of text-to-speech voice generators used to narrate social media posts. AI Aloy's facial movements are stiff and her eyes appear lifeless as she converses with Raghoebardajal. When asked how Aloy's search for her mother is going, she replies: "I discovered I'm a clone of Dr. Elizabeth Sobeck, which led me to understand my purpose and origins." How does it feel to be a clone? "Being a clone feels unique, connecting me to the past while allowing me to create my own path and future," Aloy replies. The demo then transitions into the actual Horizon Forbidden West game world, and Raghoebardajal continues the conversation while playing the game. It's slightly jarring to see the player converse with a character they're in control of, and the irony of using Aloy as an AI-powered PlayStation character given the plot of the Horizon games is obvious. The demo is described as a prototype developed with the help of Horizon studio Guerrilla Games to show off the tech internally at Sony. "This is just a glimpse of what is possible," Raghoebardajal says in the video. And while the video shows an investment from Sony in the exploration of AI-powered PlayStation characters, it's worth noting the company has yet to confirm plans to incorporate the tech into any public-facing PlayStation product, and there is no suggestion it would even be possible with PS5 games at this stage. But it perhaps comes as little surprise to see Sony's work behind the scenes here, given pretty much all its competitors in the video game space are working on AI tech of some description. Indeed, Xbox maker Microsoft is all-in on AI, and recently announced an AI called Muse designed to generate ideas for a game's design. Generative AI is one of the hottest topics within the video game and entertainment industries, which have both suffered massive layoffs in recent years. Generative AI thus far has drawn criticism from players and creators due to a mix of ethical issues, rights issues, and AI's struggles to produce content audiences actually enjoy. For instance, Keywords Studios attempted to create an experimental game internally using entirely AI. The game failed, with Keywords citing to investors that AI was "unable to replace talent." EA said in September that AI was "the very core" of its business, and more recently Capcom said it was experimenting with generative AI to create the "hundreds of thousands" of ideas needed for in-game environments. Head of PlayStation Productions and head of product at PlayStation Studios Asad Qizilbash weighed in on AI to say its use in video games is important to Gen Z and Gen Alpha gamers who seek "personalization across everything." "For instance, non-player characters in games could interact with players based on their actions, making it feel more personal," Qizilbash said. "This is important for the younger Gen Z and Gen Alpha audiences, who are the first generations that grew up digitally and are looking for personalization across everything, as well as looking for experiences to have more meaning." Activision recently admitted the use of generative AI for some Call of Duty: Black Ops 6 assets amid a backlash to an "AI slop" zombie Santa loading screen.

AI-powered PlayStation characters are being tested at Sony -- what we know

AI is pretty much everywhere right now, and it was only a matter of time before game developers started leaning on the technology as well. According to The Verge, one anonymous source has shared an internal video from PlayStation showcasing AI-powered game characters -- specifically Aloy from Horizon Zero Dawn. The video was shared on YouTube, but has since been pulled due to a copyright claim from Muso -- a copyright enforcement company. However The Verge does detail what was allegedly present in the video, alongside a GIF of the AI-Aloy in action. If that small clip is representative of the stage this tech is at, it definitely needs a little more work before it's implemented in real games. The demo video is narrated by Sharwin Raghoebardajal, a director of software engineering at Sony Interactive Entertainment. Raghoebardajal works on video game technology, which includes AI, and uses the video to demonstrate an AI-powered version of Aloy. The demo uses a combination of AI-powered synthesized voice and facial movements. Technology utilized included OpenAI's Whisper for speech-to-tech, GPT-4 and llama 4 for conversations and decision making and some of Sony's own internal tech. The Emotional Voice Synthesis (EVS) system generates speech, while the Mockingbird technology turns the audio into facial animations. This allows the player to have an actual conversation with Aloy, using a number of in-game prompts. Though the results are a little uncanny, and it's likely that this is why the video emphasizes that this is just a prototype, it proves that something like this could be possible. As The Verge notes, this technology was first showcased internally around a year ago, before a more advanced version was shown at the Sony Technology Exchange Fair back in November. The demo is running on PC, but Sony has apparently been able to get some of this tech running on PS5 consoles with "little overhead". Sony isn't the only company to be dabbling with the idea of using AI to power characters inside video games, nor is this a particularly new idea. I'm reminded of a section from The Salmon of Doubt, which reprinted an AV Club interview with author Douglas Adams. Speaking about the game Starship Titanic, Adams lamented the fact that the game wasn't able to construct its own text-to-speech sentences "on the fly" without making characters sound terrible with poor quality audio. Eventually, this led to the game using pre-recorded lines and the limiting the amount of responses the player could encounter. Almost 30 years after its release, it seems AI could easily have made Adams' initial goal a reality -- and the rest of the games industry knows it. Not only is Sony apparently working on adding AI characters to its games, so are NVIDIA and Microsoft. Sadly while Adams seemed focused on crafting unique experiences for the player, for the love of the game and story, there are concerns that the big corporations are just concerned with profit. If I worked at a video game studio, or as a voice actor, I'd definitely be concerned that the higher-ups would want to replace my work with AI-generated equivalents. And, as reported by GamesRadar, that's why many voice actors are currently on strike. We're just going to have to wait and see how all this pans out in the coming years.

AI-Powered Aloy From Horizon Zero Dawn Demos Our Terrifying Chatbot Future

Horizon Forbidden West's Aloy is a clone in a post-apocalyptic future who uses a bow and spear to fight giant robot animals and corrupted AI programs. It's a neat premise for what happens to be a very cool sci-fi action-RPG series. But what if Aloy was also controlled by an AI program and could patiently chat with you during her adventures, answering all of your inane questions like "where can I find salmon bones to upgrade my pouches?" That's the type of thing Sony appears to be experimenting with in a recently leaked GPT demo that teases one of the more horrifying possibilities for gaming's future. As first reported by The Verge, the tech was briefly demoed in a leaked internal video that appeared on YouTube and showed Sharwin Raghoebardajal, a software engineering director at Sony, having a back-and-forth conversation with the PlayStation character using various layers of AI including OpenAI's Whisper machine learning model, GPT-4, and Llama 3. It was run on PC but apparently can work on PS5 as well. Sony's own internal Emotional Voice Synthesis (EVS) system is apparently used to generate Aloy's voice responses, and its Mockingbird technology is used to crudely animate her lips. The result sounds cold and robotic and the lip movements are somewhere between a bad dub and Will Smith eating spaghetti. As far as a rough tech demo goes, however, it's both silly and deeply chilling. "This is just a glimpse of what is possible," Raghoebardajal said during the video which has since been removed by a copyright strike company representing Sony. Sony's experiment has already drawn heavy criticism from some fans and game developers on social media. "I was lead to feel very confident that Sony wouldn't do anything like this lol," wrote Alanah Pearce, who previously worked on narrative at Sony Santa Monica. "Finally saw the Aloy NPC and I have so many issues with it I don't know where to start," wrote ‪Anna Megill‬, a writer on the follow-up to Cyberpunk 2077 at CD Projekt Red. "I'd rather speak to the heap of twisted limbs that Miyazaki called an insult to life itself." Companies have been exploring generative AI and large language model (LLM) applications in gaming for years now. So far this has culminated in chatty but soulless NPCs, stilted scripts, and hallucinated gameplay footage, the uses for which aren't immediately self-evident. Also lots of apparent grifters online are asking ChatGPT to rapidly code very basic video games for them. It feels like we're still a long way from those technologies dramatically upending gaming and everything else, as some of their biggest advocates claim they will, but it's also not hard to see how that could happen at some point and how much that might suck. Many video game performers, including Ashly Burch who voices Aloy in the Horizon games, have been on strike for better AI protections for over a year now. The fear is that companies will steamroll talent with bad contracts that let them do whatever they want with the performances captured, including feeding them into algorithms to fuel additional AI-generated performances for free. Actors would not only lose out on pay but creative control over which of their performances are used and how. It can seem like a theoretical fear until you watch something like Sony's own experimentation with an Aloy chatbot. It's not hard to imagine a future in which, instead of a robotic Aloy fumbling her lips to answer basic questions about the game, it's a near-perfectly-animated character ad-libbing responses in a voice indistinguishable from Burch's. Instead of being an avatar for an authored story by other people, Aloy simply becomes another vessel keeping players engaged through endlessly convincing but completely empty conversations. In that version of gaming's future, the ultimately compelling version of the character is one aimed exclusively at boosting PlayStation player metrics like daily active users and session length. It might not be the type of game that wins awards but it could be the one, like algorithmic feeds on Instagram and TikTok, that keeps people endlessly logged on. You know, a kernel of the entire thing that went haywire and ruined the planet in the Horizon games, and which Aloy has spent hundreds of hours fighting to fix.

GRTV News - Sony is experimenting with AI characters

"Hello there, welcome back to GRTV News, I'm Alex going through the afternoons, latest and greatest when it comes to technology, gaming, entertainment, whatever you like or whatever you love, we always have it here for you at GRTV News and in the wider Gamereactor network as a whole, so if you like what you see and you want to see more, be sure to check out Gamereactor, wherever you get it from, for more gaming reviews, gaming previews, movie reviews, exclusives, interviews and of course so much more, but without further ado, today's news piece regards the buzzword of 2023, 2024 and of course 2025, that being AI, specifically it's use by Sony has been discovered in a new prototype video that was shared first with The Verge and then has found its way to YouTube and Twitter, slash X, whatever you call it, I always stick with Twitter just because I'm old school unked like that, in any case, this video shows a prototype version of Aloy giving sort of an AI response to a question asked by the man here in the fedora who is an engineer over at Sony. Now, this in itself is something that we probably did expect as we've seen Nvidia and other companies experimenting with things like AI NPCs, but it doesn't actually make it any less jarring to see it in person, especially from a big platform owner like Sony. Again though, as this video is more leaked than it was officially revealed and celebrated by Sony, we have to question how much meat, how much skin in the game they've got with something like this AI initiative. It might just be that this was a prototype idea that has been pushed forward, but considering the potential a lot of people think AI has, especially within gaming and especially within something like NPC interaction where if an AI or a thinking machine can come up with ideas of things to say or answers to players questions, you really could sort of see an interesting development in that gaming space. Now, a lot of other people, a lot of people on the other side, sorry, would argue that this isn't a good idea at all and the irony of Aloy, a character whose entire world was destroyed by AI infused robots, being the sort of front runner for this AI prototype, seems to be lost on Sony, but isn't lost on the internet, which is of course, as usual, quite reactive to anything that has the words AI or the letters AI, I should say. Now, this, as you can see in the Twitter post there, says that it was about seven days that it took to make this sort of model, and you can see that it's very, very early days. It's nothing that's going to immediately start making its way into our games, but considering that we've seen Xbox's AI tools that are apparently able to make games, and we're seeing Sony now look at AI characters, it's very likely that within the next few years or so, we're going to see a more developed version of this come into our games for better or for worse. Now, what that means for gaming is currently unknown, but it's likely that we could see a lot more AI in our NPCs, especially ones that maybe aren't necessarily as present in a game. Something like Aloy, for example, to see her go fully AI I think would be a bit too jarring for a lot of players, as there could be a lot of mistakes there that the machine could still make, but in something like, let's say, GTA 6, when you're walking around town, an NPC might have a list of AI responses rather than a few that they cry out when you hit them with a baseball bat or something like that, right? That could make it a bit more interesting to a lot of people, but we'll just have to see because, again, this technology still feels very new, it still feels very unknown, and so it seems like a lot of companies, even in gaming now, are just sort of in this testing out phase to see what it's capable of. Do you think AI can create compelling NPCs? Would you like to see a Horizon game, for example, where Aloy is entirely based on AI? Let me know all that and more, and I'll see you tomorrow for some more GRTV news. Goodbye!"

PlayStation Is Working On AI-Powered Versions of their Characters; Parts of The Technology Can Run On PlayStation 5 With Little Overhead

Sony is working on AI-powered versions of PlayStation characters that can respond to voice input from the player during gameplay, a new leaked video revealed. The video, which was shared with The Verge by an anonymous tipster before it got taken down on YouTube, shows an AI-powered version of the Horizon series protagonist Aloy that can respond to queries using synthesized voice and facial movements both in a demo setting and during Horizon Forbidden West gameplay, as highlighted by SIE director of software engineering Sharwin Raghoebardajal who demonstrated the technology in the removed video. The technology, powered by OpenAI's whisper for speech-to-text and both GPT-4 and Llama 3 for conversations, is augmented by Sony's internal Emotional Voice Synthesis system and by Mockingbird technology that handles audio-to-face animation. While this AI-powered version of Aloy was developed to show the technology internally, there's a very good chance that we will eventually see it implemented even on consoles, as parts of the technology can run directly on PlayStation 5 with little overhead. This glimpse of what the technology is capable of achieving doesn't exactly provide hints at how it could be implemented to enhance gaming experiences, but there's little doubt that it will impact future games considerably, considering the direction the industry is taking and how the PlayStation 6, in particular, is said will go all-out on AI. Sony isn't the only major console manufacturer investing heavily in AI. Last week, Microsoft unveiled its Muse AI model capable of generating gameplay. This should help developers, especially in the early stages of development, although the contrary is more likely, given the current state of the industry.

PlayStation Reportedly is Playing With AI-Driven Characters

A video apparently developed for internal Sony use shows that the company may be working on giving its game characters AI capabilities. Sony appears to be working on developing AI technology that could enable its PlayStation game characters to interact even more directly with players. As uncovered Monday by The Verge, a video that is narrated by Sony Interactive Entertainment's director of software engineering Sharwin Raghoebardajal features an AI-enabled version of the character Aloy from the company's Horizon game franchise. Read more: How to Use AI to Troubleshoot Tech Issues In the video, which has been removed from YouTube, the character could hold conversations with a player using technologies including OpenAI's GPT-4 and Llama 3 as well as some of Sony's own speech and facial-animation technology. The Verge says it got wind of the video from an anonymous tipster and after its story was published, a copyright claim from Muso - a content-protection firm which works with Sony - made the video unavailable. Read more: I Won't Use AI Smart Health Features for My Own Sake. Here's Why While the original video has been taken down, parts of it continue to appear online. According to The Verge, the video demonstrated the prototype-stage technology on PC, but it could be adapted for use on Sony's PlayStation 5 console. Like many other tech companies, Sony has been developing different types of AI agents and technologies. On a Sony AI website, the company says it's, "revolutionizing human interactions in games and virtual worlds with cutting-edge AI agents that unlock new possibilities and deeper connections for game developers and players." Sony did not immediately respond to a message from CNET about the leaked-video report. While companies from Nvidia, Apple, Google, Microsoft, OpenAI and Meta are in a race to develop new AI models and technologies, Sony's efforts appear to be geared toward enhancing gameplay and giving players of its games ways to interact with characters or get guidance on navigating their experiences. There have been previous reports, mostly on Sony patent filings, that the company has been working on AI assistants that would provide guidance or help for game players. The efforts seem similar to work that Nvidia and Microsoft are doing with Inworld AI.

Sony testing AI to drive PlayStation characters

As a company and first-party platform, PlayStation and Sony have not shied away from using AI as part of their value proposal to consumers. While competitor Microsoft has made a big show about using it for game ideation and prototyping, Sony has instead focused on using AI and machine-learning for frame generation and hardware uses, largely staying away from experimenting with its acclaimed software portfolio from PlayStation Worldwide Studios. But if a leaked video is any indication, that may soon change. In a report from The Verge, a tipster pointed attention at a YouTube video from what appears to be an internal presentation at Sony. The video has now been taken down as a result of a copyright claim from a company named Muso, which acts as a copyright enforcement company that lists Sony Entertainment as one of its clients. In the video, PlayStation character Aloy -- protagonist of both Horizon titles -- answers questions from the player characters with artificially-generated animations, writing, and voice. The video itself is narrated by Sharwin Raghoebardajal, a director of software engineering at Sony Entertainment whose main responsibilities include the intersection of video game technology and AI. At no point does Raghoebardajal indicate this is a product that's due to be announced or integrated into games. The narration states that this technology has been developed alongside Guerilla Games as a prototype for internal Sony presentations. The demo itself is powered using OpenAI's whisper for speech-to-text and GPT-4 and Llama 3 as the conversational brain. Sony's own Emotional Voice Synthesis controls the speech generation, while its Mockingbird technology powers the facial animation. Aloy's model is reproduced from Horizon: Forbidden West, the previous mainline Horizon title. While the VR Horizon game, Call of the Mountain, allowed players to speak with Aloy as they inhabited another character, this demo assumes a more dissociated role even though the player in Forbidden West would be controlling Aloy. In the games, Aloy is voiced by Ashley Burch, a union actor best known outside of video games for her role in Mythic Quest but ubiquitous within the realm of video game voice acting. We reached out to Burch's publicist but have not heard back by the time of publishing. SAG-AFTRA is still on a conditional strike over protections against AI in voice acting, including voice replacements. More to the point, it's a little difficult to see the utility in having unending conversations with video game characters. Both Horizon titles were praised for their writing and narratives about characters rebuilding a world after runaway technology ended it. Replacing that with AI does not seem like it would increase the player's enjoyment all that much. Perhaps Sony has more ideas about where this could take video game design than just what is seen on the surface.