NVIDIA's DLSS 4: Revolutionizing Real-Time Graphics with AI and Transformer Models

Understanding NVIDIA's DLSS 4: Transformer Models and Real-Time Graphics Performance

NVIDIA's research on DLSS 4 dives into new methods for boosting real-time graphics using AI. The paper explains how the shift from old CNN approaches to transformer models helps capture spatial and temporal details better. This change means that DLSS 4 can improve the quality of ray-traced effects by around 30 to 50 percent. The system uses a clever Multi-Frame Generation feature that creates three extra frames for every native one. This allows for smoother visuals, with each AI-generated frame taking about 1 millisecond on RTX 5090 GPUs, compared to the 3.25 milliseconds seen in the earlier DLSS 3. Such improvements make a noticeable difference for competitive gaming where every millisecond counts. The new DLSS 4 pipeline also takes full advantage of NVIDIA's Blackwell architecture. By using FP8 tensor cores and fused CUDA kernels, the system manages its resources efficiently despite the larger size of transformer models. Vertical layer fusion and smart memory optimizations help keep the extra computational load in check. Because of these improvements, game studios can rely on a unified AI pipeline that reduces the amount of manual tuning needed for different graphical effects. This setup makes it easier for studios to add advanced path tracing to their games across various hardware setups. Plus, with features like hardware flip metering and an integrated display engine, the frame pacing stays on point, ensuring smooth visuals even during intense action scenes. For quality control, NVIDIA has put a dedicated supercomputer to work over more than six years. This supercomputer monitors for issues like ghosting, flickering, or blurriness in a wide range of games. When it spots a problem, it adds new examples of both ideal and challenging scenarios to its training sets. This ongoing process helps the neural networks learn to produce frames that look as realistic as those generated by traditional game engines, but without unwanted artifacts. The whole approach is laid out clearly in the research paper, showing how DLSS 4 is built to handle real-time rendering challenges efficiently. Source: NVIDIA Research Share this content Twitter Facebook Reddit WhatsApp Email Print Samsung Foundry 1.4 nm Node Might See cancelation: What This Means for Semiconductor Manufacturing BC1 Pi Open Benchtable for Raspberry Pi 5

DLSS 4 research paper offers a detailed look at the new transformer model and Multi Frame Gen

TL;DR: NVIDIA's DLSS 4, launched with the GeForce RTX 50 Series, enhances image quality and performance with its new transformer-based models. It also introduces Multi Frame Generation, generating up to three additional frames for smoother gameplay. The new architecture reduces latency and artifacts, improving detail preservation in dynamic scenes. NVIDIA launched DLSS 4 alongside its new GeForce RTX 50 Series GPUs earlier this year, and it has been one of the most talked about aspects of the latest GPU generation. The shift to a transformer-based Super Resolution and Ray Reconstruction model, available to all GeForce RTX gamers, has been a game changer for image quality. Gaming in 4K using DLSS 4's 'Performance' preset now delivers better image quality than DLSS 3 'Quality' and native rendering with TAA. DLSS 4 is now available in over 100 games and apps. This is the next step in the shift toward a future where hardware and software innovations driven by AI will boost performance and image quality, and usher in a new era of high-quality ray-tracing with RTX Neural Shaders. DLSS 4 also introduced the brand-new Multi Frame Generation, exclusive to the GeForce RTX 50 Series. An evolution of the Frame Generation that launched with the GeForce RTX 40 Series, Multi Frame Generation leverages powerful AI hardware and RTX Blackwell's architecture to generate up to three additional frames to deliver a smoother gaming experience that is still responsive. DLSS 4 is one of the most impressive gaming technology releases in 2025, and NVIDIA has just released a new research paper that details how it works and the advancements it brings to neural rendering. NVIDIA on Multi Frame Generation, the new architecture, and how frame times have been dramatically reduced. The new DLSS Multi Frame Generation architecture splits the neural component of DLSS 3's Frame Generation in half. One half of the network runs once for every input frame pair and its output is then able to be re-used. The other (much smaller) half runs once for every generated output frame. This split architecture allowed us to understand and optimize these networks in parallel, helping to bring the final algorithmic latency within the target. In DLSS 3's Frame Generation, a single 4K frame could be generated (thus doubling the frame rate) in around 3.25ms on a GeForce RTX 4090. In contrast DLSS 4's Multi Frame Generation, each of the three new frames (a quadrupling of the frame rate) can be generated in around 1ms on average on a GeForce RTX 5090 at the time of launch. It is a remarkable new upper bound for what can be expected from a real-time interpolation product for gaming, and there is only room for improvement going forward. NVIDIA on the new transformer model significantly reducing artefacts like ghosting. Ghosting artifacts occur when a denoiser struggles to handle fast-moving objects or dynamic lighting changes across frames. The transformer model demonstrates better handling of these scenarios by leveraging its attention mechanism to track spatial-temporal relationships more effectively. As a result, fast-moving objects retain their clarity, and the visual output remains sharp even in highly dynamic scenes. Relatedly, disocclusion areas, where previously hidden parts of the scene become visible due to object or camera movement, are particularly challenging for denoisers. These regions often have very few temporally accumulated samples, leading to visible noise or artifacts. The transformer model shows marked improvements in handling disocclusions, producing smoother and more accurate results by better generalizing from available spatial context and efficiently filling in missing information. Finally, here's NVIDIA on how the new transformer model significantly improves object and environment detail compared to the previous CNN model. This is the one thing you immediately notice when switching to DLSS 4. One of the most significant improvements observed with the transformer-based approach is its superior ability to preserve and reconstruct fine surface details. Traditional convolutional networks often struggle with maintaining the intricate textures and patterns present in complex surfaces, leading to loss of detail or over-smoothing. The transformer model's attention mechanism allows it to better understand the contextual relationships between different texture elements, resulting in more accurate preservation of surface characteristics. This improvement is particularly noticeable in areas with complex materials such as fabric, foliage, and architectural details, where the model maintains crisp, defined textures even at higher super resolution factors. The full research paper is technical, informative, and detailed, so be sure to check it out if you're interested in peeking behind the curtain and learning how DLSS 4 works.