AMD's FSR 4.0: AI-Powered Upscaling for Enhanced Gaming Performance and Battery Life

It's happening: AMD is going to use AI in FSR 4 to drive much better battery life for PC gaming handhelds

Tackling one of the biggest challenges for handhelds - and much more besides, no doubt AMD is going to power up the next-generation of its FSR tech by using AI, in what Team Red admits is a turnaround regarding its stance on using artificial intelligence for the purpose of boosting frame rates in PC gaming. This came to light in an interview that Tom's Hardware conducted with Jack Huynh, who is Senior VP and GM of the Computing and Graphics Group at AMD. When exploring the topic of battery life in gaming handhelds like the Asus ROG Ally X, Huynh lamented the lack of longevity when playing more demanding games (giving Black Myth: Wukong as an example). What's the cure for that? Well, that's an extremely thorny problem to deal with, naturally. However, Huynh feels that FSR 4 taking the strain off the hardware will be a sizeable part of getting more mileage out of demanding games when playing on the move - and that frame generation tech (meaning artificially inserting extra frames to up the frames per second count) therein will use AI. Huynh told Tom's: "So now we're going AI-based frame generation, frame interpolation, and the idea is increased efficiency to maximize battery life. And then we could lock the frames per second, maybe it's 30 frames per second, or 35. My number one goal right now is to maximize battery life." Huynh then admits this is a major switch to go with artificial intelligence, but said he told his team he wanted something with a very fast time to market with FSR 4, and that the future was AI in that respect - and AMD 'completely pivoted' about a year ago to go AI-based with next-generation FSR. In many ways, this feels inevitable - the switch to AI, that is, and not just because artificial intelligence is such a huge presence across the entire tech industry these days. While AMD initially poured scorn on the need for AI with upscaling tech like FSR, the mood at Team Red has clearly changed in more recent times. Indeed, other AMD executives like CTO Mark Papermaster have already dropped hefty hints that FSR will get infused with AI at some stage, and so it seems that's exactly what's happening with FSR 4. The framing here of FSR 4 boosting battery life in handhelds may seem odd, but in context, this was part of the conversation Tom's had with Huynh at the recent IFA 2024 show where the exec was talking specifically about handhelds. So, don't read too much into this - it's not like FSR 4 will be designed primarily for gaming handhelds, or at least we can't imagine that's remotely the case. However, the thought AMD is putting into the angle of FSR 4 working to beef up the weakest area of Windows 11 handhelds (and the Steam Deck too) is notable, certainly - and perhaps a reflection of the importance Team Red places on the gaming portable world right now. In short, FSR 4 won't be all about increased efficiency - that'll just be one part of a bigger puzzle (and with handhelds, the tech may well leverage the powerful NPUs on new mobile CPUs which considerably accelerate AI workloads). Hopefully, AI will also be in the mix with improving other major considerations such as the quality of the upscaled image with FSR 4, and smoothing over glitches (something FSR 3.1 has been criticized over failing to tackle). With this interview, though, we're only getting a narrow and fleeting glimpse of AMD's plans, and we don't know the full picture by any means. Still, what we do know is that AMD is bringing in AI to power up FSR 4 in some way, and maybe the next-gen tech isn't that far off (if development has been underway for the best part of a year already).

AMD's FSR 4 will be AI-powered like NVIDIA DLSS, and optimized for handhelds

In a recent interview with Tom's Hardware, AMD's Jack Huynh, the company's Computing and Graphics Business Group leader, confirmed that the next version of AMD's FidelityFX Super Resolution or FSR 4 will embrace and use AI. This applies to FSR 4's Frame Generation and Super Resolution components, and the reasoning isn't so much to follow NVIDIA's lead but to improve battery life on handhelds. AMD's new AI-baed FSR 4 is FSR for PC gaming handhelds. Citing devices like the ASUS ROG Ally and Lenovo Legion GO, Jack Huynh states that battery life is the number one priority for AMD's graphics teams and Radeon group. "I need to play a Black Myth: Wukong for three hours, not 60 minutes," Huynh explains. "This is where frame generation and interpolation [come in], so this is the FSR4 that we're adding." AMD's FidelityFX Super Resolution, or FSR, is the company's response to NVIDIA DLSS. However, unlike Team Green's approach, its algorithms and technology are currently not AI-based. The result is lower image quality and stability than DLSS, with the trade-off being that FSR can run on any GPU. Switching to an AI-based approach to upscaling and frame generation for FSR 4 would improve visual fidelity and overall image stability. However, in talking about FSR 4, AMD's Jack Huynh's response focuses on how FSR 4's AI will dramatically improve efficiency on PC gaming handhelds. He also didn't clarify whether or not this will require a Ryzen processor with a dedicated NPU or use Radeon hardware. It's still unclear what FSR 4's new AI approach will look like on the upcoming next-generation of RDNA 4 graphics cards - specifically, the rumored Radeon RX 8800 XT. AMD could implement a system similar to Intel's XeSS solution, where the AI component is limited to specific Radeon and Ryzen chips and other systems and graphics cards (including older Radeon GPUs) using the non-AI version of FSR 4. Whatever the case, according to Jack Huynh, FSR 4 has been in development for "about 9-12 months," so we should expect a full reveal soon.

AMD is set to finally join Nvidia with FSR 4, using AI to power frame generation, though no word on upscaling yet

FidelityFX Super Resolution, AMD's answer to Nvidia's DLSS, has been around for a little over three years now and it's been a real boon to all kinds of GPUs, including Nvidia's GeForce models. The latest version, FSR 3.1, offers a shader-based upscaler and frame generation system but the tech world hyped on AI, AMD has decided it can't ignore it any longer and has announced that the next iteration will be AI-based, for frame generation at least. In an interview with Tom's Hardware, AMD's senior vice president Jack Huynh was talking about some of the frustrations about playing on handheld gaming PCs and that enjoying a high frame rate greatly limited how long you could game for. "On the handheld side, my number one priority is battery life. If you look at the Asus ROG Ally or the Lenovo Legion Go, it's just that the battery life is not there. I need multiple hours. I need to play [Black Myth] Wukong for three hours, not 60 minutes." He's not wrong. Even with the huge battery in the ROG Ally X, I rarely get more than an hour of gaming in performance mode and while you don't need to have the chip inside running at full speed for every game, you certainly do for anything new with high-end graphics. The solution, Huynh reckons, will be in the form of the next version of FideltyFX Super Resolution. "This is where frame generation and interpolation [come in], so this is the FSR 4 that we're adding. [W]e completely pivoted the team about 9-12 months ago to go AI-based. So now we're going AI-based frame generation, frame interpolation, and the idea is increased efficiency to maximize battery life. And then we could lock the frames per second, maybe it's 30 frames per second, or 35." That's all that Huynh had to say about it all so we're left with a whole bunch of things to ponder and speculate on. Let's start with what seems to be the primary motive for making FSR 4 AI-based: improving battery life. Admirable but I rather hope that the use of AI also helps to solve FSR's visual weaknesses (watch the video below). Anyway, no current handheld gaming PC or Radeon-equipped laptop have any specific hardware in the GPU for accelerating neural network calculations, unlike Intel Arc and Nvidia RTX chips. But the new Ryzen AI 300 series does have an NPU (neural processing unit) that can do such workloads, all while using relatively little power. So does that mean FSR 4 will require access to one or will it remain entirely shader-based, just like the current version? One doesn't need matrix/tensor cores to handle AI algorithms, but offloading those tasks onto them frees up the standard ALUs for shader routines. If FSR 4 does use an NPU or AMD's next generation of GPUs, using the forthcoming RDNA 4 architecture, has a modicum of matrix units inside them, does this mean that FSR 4 will not be accessible to anyone with a different system? FSR's strongest feature is the fact that it can be run on all kinds of GPUs: it just needs to support a certain level of compute shader support in Direct3D or Vulkan. I'd be very surprised if AMD plans to abandon that approach which leads me to suspect that the AI nature of FSR 4 won't be as reliant on hardware support as XeSS and DLSS are. That said, Intel does have two modes for its upscaler, one that's open to all kinds of GPUs like FSR and another that's exclusive to Arc GPUs, utilising the matrix units in the GPU. So here's what I think: FSR 4 will still be shader-based for upscaling and will employ an AI-based system for frame generation that can potentially leverage all kinds of different hardware to accelerate the process (GPU, NPU, perhaps even a CPU). It might be similar to its driver-based Fluid Motion Frames system that works with any RDNA 2 or newer GPU. In fact, it could even just be AFMF that then uses a neural network to clean up the interpolated frame, just like DLSS does for upscaling. AMD has previously said that it plans to get AI upscaling rolled out on all its gaming devices at some point, so perhaps every aspect of FSR 4 will be 100% AI-powered. Right now, your guess is as good as mine. We probably won't hear any more about the next version of FSR until AMD is ready to talk about RDNA 4. So until then, let's all just have fun guessing what it will actually be. Anyone fancy a bet on it being a combination of ChatGPT and Stable Diffusion? "Create an image like these two frames but one frame in the future."

AMD FSR 4.0 to Incorporate Full AI-Based Upscaling

Thus far AMD has not implemented FSR based on artificial intelligence (AI) into its upscaling techniques. The latest iteration, FSR 3.0, introduced frame generation capabilities to enhance gaming performance without significantly impacting image quality. That seems to change. FSR is an open-source spatial and temporal upscaling technology designed to boost frame rates by rendering games at a lower resolution and then upscaling the images to higher resolutions using advanced algorithms. This approach aims to increase performance without requiring additional computational power from the GPU. Jack Huynh, an executive at AMD, announced during an interview that the upcoming AMD FidelityFX Super Resolution (FSR) 4.0 will leverage AI technology for its upscaling techniques. According to Huynh, this shift marks a departure from the previous versions of FSR, which primarily relied on analytical and algorithmic methods for image enhancement. In the interview conducted at IFA 2024 by Tom's Hardware, Huynh, who leads AMD's Computing and Graphics division, highlighted that FSR 2 and FSR 3 used a combination of analytical upscaling and algorithmic filters to enhance image quality. However, the team has shifted focus over the past year to prioritize AI as the core of their upscaling methods. While AMD's FSR has traditionally relied on analytical and algorithmic methods for image enhancement, a shift to AI-based upscaling in a future FSR 4.0 could represent a significant change in strategy. AI-driven techniques have the potential to provide superior image quality and performance by leveraging machine learning models trained on high-resolution imagery. Incorporating AI could also address common concerns such as battery life in gaming handhelds. By optimizing resource utilization, AI-based upscaling might improve energy efficiency, allowing for longer gaming sessions on portable devices. NVIDIA's Deep Learning Super Sampling (DLSS) is an example of AI-based upscaling that utilizes dedicated hardware (Tensor Cores) for machine learning operations. DLSS has been praised for its ability to deliver high-quality images with improved performance. If AMD were to adopt a similar approach, it could enhance the competitiveness of FSR against NVIDIA's offerings. Source: tomshardware