AMD Researches Neural Supersampling and Denoising for Real-Time Path Tracing

AMD aims to catch up to Nvidia's DLSS with "neural supersampling"

Serving tech enthusiasts for over 25 years. TechSpot means tech analysis and advice you can trust. Something to look forward to: Thanks to its DLSS tech, which has become something of a gold standard, Nvidia has always led the charge when it comes to upscaling and denoising techniques for boosting performance in demanding ray-traced game scenes. But AMD hasn't been sitting idle - Team Red has steadily improved its own FSR upscaling tech, and according to a new official blog post, the company is taking things to the next level. The latest FSR 3.1 release this year brought significantly improved stability over previous versions. However, there's still room for improvement with denoising path-traced frames. Path tracing, which uses raytracing for all lighting calculations, is hugely computationally intensive. To achieve real-time path tracing performance, the number of light ray samples calculated per pixel has to be drastically lowered. There's just one shortcoming: it introduces noise in the form of incomplete illumination data that needs to be cleaned up. We've already seen how well Nvidia's DLSS handles games like Cyberpunk 2077 and Alan Wake II with path tracing enabled, thanks to techniques like Ray Reconstruction to remove sampling noise. Meanwhile, AMD seems to rely more on game-integrated denoisers which do a satisfactory job but can't match Nvidia's AI-powered prowess. That's because, unlike DLSS, FSR does not rely on AI or machine learning. Instead, it uses a combination of spatial and temporal upscaling. That may change soon based on a recent AMD post. It says the company's current research efforts are centered around enabling real-time path tracing on its RDNA GPU architecture using AI neural networks. Specifically, the innovation AMD is working on is a single neural network model that combines upscaling and denoising into one step - simultaneously reconstructing complete scene details while also removing noise. "We are actively researching neural techniques for Monte Carlo denoising with the goal of moving towards real-time path tracing on RDNA GPUs," notes the company. The goal is to enable "a neural supersampling and denoising technique which generates high quality denoised and supersampled images at higher display resolution than render resolution for real-time path tracing with a single neural network." There's no indication of when we can expect this technology to come out. It's also unclear if it will work across all RDNA generations or just on future RDNA 4 and newer GPUs. The most likely scenario is that this neural denoising magic may be part of an FSR 4 release, perhaps even in a limited form. Also unknown is exactly how AMD plans on implementing it. So far, FSR's biggest selling point has been its universal compatibility, but this could change if the feature requires specific RDNA hardware. Nonetheless, if it all pans out, a future version of FSR could help AMD finally catch up to Nvidia's DLSS.

AMD Begins Working On Its Own Neural Supersampling & Denoising Techniques For RDNA-Based GPUs, Competitor To DLSS Ray Reconstruction

AMD's upcoming FSR package could solve the noise issue on path-traced frames with AMD's proprietary neural supersampling and denoising techniques. AMD will reportedly follow NVIDIA (DLSS Ray Reconstruction) in its course for developing AI-based denoising techniques & neural supersampling NVIDIA has come far with its upscaling techniques, which have solved the performance issues in demanding scenes for modern titles when ray tracing is turned on. As soon as NVIDIA started implementing various ray tracing and upscaling techniques, AMD and Intel also followed with their own FSR and XeSS upscaling methods. While DLSS has matured quite decently in the past years, AMD and Intel are still catching up by releasing newer and better upscalers. AMD has released its FSR 3.1 this year, which is much more stable than the previous versions, but there are still issues when we talk about denoising the visuals in the path traced images. NVIDIA is way ahead of its rivals as it introduced Ray Reconstruction to improve the visuals by reducing the noise, but AMD depends on the denoisers present in modern games. While the in-game denoisers do a satisfactory job, the AI-based neural network i.e., NVIDIA's Ray Reconstruction is much more powerful, which denoises the images more accurately. To do this job, NVIDIA uses its own Tensor Cores but AMD relies on the WMMA(Wave Matrix Multiply Accumulate). In a recent blog post, AMD announced that ...actively researching neural techniques for Monte Carlo denoising with the goal of moving towards real-time path tracing on RDNA GPUs. This means that AMD is now on its journey to develop a technique that will use AI to deliver better visuals and denoising that will work on RDNA GPUs. It's not clear whether these techniques will work on the existing RDNA-based GPUs from previous and current generations or only on the upcoming RDNA 4 or future GPUs. AMD's FSR is known for its universal compatibility, which gives it an edge over DLSS, but the upcoming techniques may work only on RDNA GPUs. AMD also added, Our technique can replace multiple denoisers used for different lighting effects in rendering engine by denoising all noise in a single pass as well as at low resolution. It's also possible that the FSR 4.0 will have some limited compatibility with NVIDIA and Intel GPUs but will work at full potential for RDNA GPUs outside the box. It's no surprise that AMD might consider using AI for developing its neural network because this year, AMD's CTO, Mark Papermaster, announced that they are enabling AI-based upscaling on their gaming devices.

AMD research suggests plans to catch up to Nvidia using neural supersampling and denoising for real-time path tracing

Nvidia currently dominates the GPU market, thanks to a combination of performance, features, and brand recognition. Its advanced AI (Artificial Intelligence) and machine learning-based technologies have proven particularly potent, and AMD hasn't really caught up, especially in the consumer market. But the company hopes to change that very soon. According to a post on GPUOpen, AMD research is currently focused on achieving real-time path tracing on RDNA GPUs via neural network solutions. Nvidia uses its own DLSS for image upscaling with AI, but DLSS has come to mean a lot more than "Deep Learning Super Sampling" -- there's DLSS 2 upscaling, DLSS 3 frame generation, and DLSS 3.5 ray reconstruction. AMD's latest research centers on neural denoising to clear up noisy images caused by using a limited number of ray samples in real-time path tracing -- basically ray reconstruction, as far as we can tell. Path tracing normally uses thousands or even tens of thousands of ray calculations per pixel. It's the gold standard and what movies typically, often requiring hours per rendered frame. In effect, a scene gets rendered using calculated ray bounces where even a slight shift in the path taken can result in a different pixel color. Do that a lot and accumulate all of the resulting samples for each pixel, and eventually the quality of the result improves to an acceptable level. To do path tracing in real-time, the number of samples per pixel needs to be drastically reduced. This results in more noise, as light rays frequently fail to hit certain pixels, leading to incomplete illumination that requires denoising. (Movies use custom denoising algorithms as well, incidentally, as even tens of thousands of samples doesn't guarantee a perfect output.) AMD aims to address this with a neural network that performs denoising while reconstructing scene details. Nvidia's solution has been praised for preserving details that traditional rendering takes much longer to achieve. AMD hopes for similar gains by reconstructing path-traced details with a few samples per pixel. The innovation here is that AMD combines upscaling and denoising within a single neural network. In AMD's own words, their approach "generates high-quality denoised and supersampled images at a higher display resolution than render resolution for real-time path tracing." This unifies the process, allowing AMD's method to replace several denoisers used in rendering engines plus doing upscaling in a single pass. This research could potentially lead to a new version of AMD's FSR (FidelityFX Super Resolution) that might match Nvidia's performance and image quality standards. Nvidia's DLSS technologies require dedicated AI hardware on RTX GPUs, along with an Optical Flow Accelerator for frame generation on RTX 40-series (and later) GPUs. AMD's current GPUs generally lack AI acceleration features, or in the case of RDNA 3, there are AI accelerators that share execution resources with the GPU shaders, but in a more optimized way for AI workloads. What's not clear is whether AMD can run a neural network for denoising and upscaling on existing GPUs, or if it will require new processing clusters (i.e. tensor units). Achieving this on existing hardware would potentially allow a future FSR iteration to work across all GPUs, but it might also limit quality and other aspects of the algorithm. We'll need to wait and see what AMD ultimately delivers. A refined approach to neural path tracing and upscaling could bring accessible, high-fidelity graphics to a broader range of hardware, but given the demands of path tracing in games (see: Alan Wake 2, Black Myth Wukong, and Cyberpunk 2077 RT Overdrive), we suspect AMD will need much faster hardware than existing products to reach higher levels of image fidelity.

AMD could swipe some of the best features of Nvidia GPUs | Digital Trends

Nvidia overwhelmingly dominates the list of the best graphics cards, and that largely comes down to its feature set that's been enabled through DLSS. AMD isn't sitting idly by, however. The company is researching new ways to leverage neural networks to enable real-time path tracing on AMD graphics cards -- something that, up to this point, has only really been possible on Nvidia GPUs. AMD addressed the research in a blog post on GPUOpen, saying that the goal is "moving towards real-time path tracing on RDNA GPUs." Nvidia already uses AI accelerators on RTX graphics cards to upscale an image via DLSS, but AMD is focused on a slightly different angle of performance gains -- denoising. When enabling path tracing in a game like Alan Wake 2 or Cyberpunk 2077, you're only getting a small fraction of the rays cast into the scene. In a real-time context, only a handful of samples per pixel are cast into the scene, and they bounce around, but rarely go back to a light source within the scene. That leads to a noisy image -- see the top left of the image above -- that needs to be cleaned up with denoising. AMD is applying a neural network to the denoising process. Recommended Videos Nvidia has this technique covered already with Ray Reconstruction, which is a DLSS feature that's sorely underrated. It makes a massive difference in image quality, preserving details in path tracing that would normally take minutes or hours to render for a single frame offline. AMD is looking at something similar: taking a small number of samples per pixel and reconstructing the fine details of path tracing using a neural network. Get your weekly teardown of the tech behind PC gaming ReSpec Subscribe Check your inbox! Privacy Policy The technique AMD is researching, however, combines upscaling and denoising into a single neural network. "We research a Neural Supersampling and Denoising technique which generates high-quality denoised and supersampled images at higher display resolution than render resolution for real-time path tracing with a single neural network," the blog post reads. "Our technique can replace multiple denoisers used for different lighting effects in rendering engine by denoising all noise in a single pass, as well as at low resolution." This looks like some foundational research for the next version of AMD's FSR, which could finally match Nvidia on performance and image quality. The lingering question is if these techniques require any bespoke hardware. Nvidia claims that dedicated accelerators on its RTX graphics cards are necessary for AI-assisted upscaling and denoising with DLSS, so AMD may need dedicated hardware on its GPUs, too. However, there is a world where AMD could open up FSR 4 -- or whatever the next version is called -- to all graphics cards while still leveraging a neural network. RTX GPUs already have the hardware, and we've seen with features like Intel's XeSS that it's possible to run AI models on GPUs through separate instructions, though usually with a hit to image quality and performance.

AMD drops a possible hint about how AI could be used in its next-gen upscaler package, FSR 4

Does this mean RDNA 4 GPUs will have dedicated hardware for this stuff? Possibly. In a post on GPUOpen, a site for game and graphics developers, AMD may well have let slip that it plans to take a leaf from Nvidia's book of rendering tools by including a ray tracing denoiser system in its next generation of FSR. And just as important, it will use an AI neural network to do it all. Unless you've been firmly sticking with an old graphics card and consciously ignoring every GPU development in the past six years, you'll know that AMD, Intel, and Nvidia have all been furiously busy implementing techniques to improve ray tracing performance and visual quality. The latter is greatly affected by the number of rays that are used to calculate the lighting, shadows, reflections, and so on. Unfortunately, even on monstrous graphics cards like AMD's RX 7900 XTX and Nvidia's RTX 4090, ray tracing is extremely demanding so games only use a relatively small number of rays. That results in a very 'noisy' image -- grainy in appearance and often full of white spots -- so games have to carry out a process called denoising to clean it up. While the likes of Cyberpunk 2077, Black Myth: Wukong, and Alan Wake 2 employ their own denoiser system, Nvidia has an AI-powered one called Ray Reconstruction (RR). Ray reconstruction is all about making ray-traced images look much better and more accurate, rather than improving performance, and in Cyberpunk 2077, it's noticeably better than the game's own denoiser. But the GPUOpen post makes it clear that Nvidia won't be the only GPU vendor offering such a feature in the near future. "We are actively researching neural techniques for Monte Carlo denoising with the goal of moving towards real-time path tracing on RDNA GPUs." AMD's RDNA 2, 3, and 3.5 GPUs can all do denoising right now but only those provided by the game in question and the shader cores handle it all. The fact that the research is specifically about using a neural network to do it means that AMD is very much on board with Nvidia in using AI to boost ray tracing results. But does this mean that future RDNA GPUs will have dedicated hardware for doing the AI calculations? While Nvidia RTX chips have discrete tensor cores for this job, AMD doesn't and instead uses specific instructions (referred to as WMMA) and the standard shader cores. That might change in RDNA 4, for two reasons. One is the fact that Sony's PlayStation 5 Pro has a dedicated chip for accelerating the AI routines for its new PSSR upscaler, and AMD will certainly be aware of the benefit discrete hardware brings to such tasks. The second is one of the goals listed in AMD's denoiser research: "Highly optimized performance for real-time path tracing at 4K resolution." To me, that alone points to AMD having specific hardware for doing the neural networks, because at 4K, general-purpose shader cores just aren't going to be good enough, unless one has a small mountain of them. RNDA GPUs are the only ray tracing chips in the desktop market that don't have dedicated tensor/matrix units, so it's inevitable that AMD will follow suit at some point. Coupled with the fact that AMD has previously stated that it plans to have all its gaming devices use AI for upscaling too, I'd say there's a very good chance that RDNA 4 chips will have matrix cores that get used to do FSR 4 AI-powered upscaling, frame generation, and denoising. That said, AMD has always been of the mind that its FSR package should run on as many GPUs as possible -- not just Radeon cards, but those from Intel and Nvidia too, as long as they have the right level of shader support. If the new tech was exclusive to one generation of RDNA hardware, it could well backfire on AMD, given that its discrete GPU market share is pretty small. It's possible that AMD could offer a two-tier FSR 4 system, as Intel does with XeSS, where the full AI-powered package only works on RDNA 4 chips, but a slower and less impressive version is available for everyone. Until we know more, it's all just guesswork of course, but Radeon fans should take comfort in the fact that AMD is working hard on making its GPUs as modern as possible.

AMD's RDNA team says its working toward AI-powered 4K Path Tracing for Radeon GPUs

AI-Assisted TLDR: NVIDIA's DLSS 3.5 Ray Reconstruction, introduced in 2023, significantly enhances PC game visuals by improving ray-traced effects with AI-powered denoising. This technology sharpens reflections and enhances lighting realism, making games look better on GeForce RTX systems.* Generated from the content by Kosta Andreadis below. NVIDIA's DLSS 3.5 Ray Reconstruction, which debuted in 2023, has been a game changer for PC games with multiple ray-tracing effects - particularly those with ray-traced reflections. It's an AI-powered denoiser, a part of the rendering process that cleans up an image. Alan Wake 2's Path Tracing delivers stunning visuals, but you'll need a GeForce RTX 40 Series GPU to witness it. AMD is looking to change that. NVIDIA's Ray Reconstruction AI model was trained on over 6X the data used for DLSS Super Resolution and Frame Generation. It's a game-changer that 'fills in the gaps' in a way that dramatically improves the image quality of ray-traced effects. Reflections become sharper and more detailed, and the lighting looks more realistic and cinematic. The difference is so big that games with ray tracing look better on GeForce RTX rigs - plain and simple. The good news is that AMD is working on a similar-sounding AI denoiser that could be part of RDNA 4's new "AI capabilities." Also, its FSR super-sampling or upscaling is going AI. AMD has published a new paper, 'Neural Supersampling and Denoising for Real-time Path Tracing,' which describes how its new "neural supersampling and denoising work together to push the boundary for real-time path tracing." The article mentions that AMD and the RDNA team are "moving towards real-time path tracing on RDNA GPUs" and "highly optimized performance for real-time path tracing at 4K resolution." Path tracing is exponentially more hardware intensive than a game featuring a single ray-tracing effect like shadows or reflections. Path tracing uses ray tracing for all lighting in a game. It is only possible (translation: playable) thanks to NVIDIA's DLSS suite of RTX AI technologies: Super Resolution, Ray Reconstruction, and Frame Generation. As seen in Cyberpunk 2077, Alan Wake II, Portal with RTX, Black Myth Wukong, and more - it's a glimpse into the future of game visuals. AMD confirms that it's focusing on real-time path tracing on RDNA GPUs, which is excellent news. Hopefully, we will see the first version of AI-powered RDNA 4 path tracing in early 2025.