Nvidia's Blackwell Ultra GB300 Dominates MLPerf Benchmarks with Significant Performance Gains

Nvidia Leads the Pack in Latest MLPerf Benchmarks

Nvidia has once again demonstrated its dominance in the AI hardware space with its new Blackwell Ultra GPU, packaged in the GB300 rack-scale design. The latest MLPerf inference benchmarks, often referred to as the "Olympics of AI," have showcased Nvidia's impressive performance gains, particularly in large language model (LLM) inference tasks 1

1

2

New Benchmarks Reflect Rapid AI Advancements

The MLPerf Inference competition has introduced three new benchmark tests, reflecting the rapid evolution of machine learning technologies. These include:

1. The largest LLM benchmark yet, based on the Deepseek R1 671B model
2. The smallest LLM benchmark, using Llama3.1-8B
3. A new voice-to-text model based on Whisper-large-v3

These additions bring the total count of LLM-based benchmarks to four, signaling the growing importance and diversity of language models in the AI landscape 1

1

Nvidia's Blackwell Ultra: A Leap in Performance

Nvidia's Blackwell Ultra GPU has demonstrated significant performance improvements over its predecessors:

• 45% increase in inference performance over the Blackwell-based GB200 platform in DeepSeek R1 tests
• Up to five times the performance of older Hopper GPUs (based on unverified third-party results)
• Top performance across all new benchmark tests, including DeepSeek R1, Llama 3.1 405B, Llama 3.1 8B, and Whisper models 2

  2

Related Stories

NVIDIA Blackwell Dominates MLPerf Inference Benchmarks, AMD's MI325X Challenges Hopper

NVIDIA's Blackwell B200 GPU Shatters AI Performance Records in MLPerf Inference Benchmark

NVIDIA Blackwell Ultra Dominates MLPerf Training v5.1, Sets 10-Minute Record for Llama 3.1 405B

NVIDIA Blackwell dominates MLPerf Training 6.0 with fastest times and 8,192-GPU scale

Hardware and Software Optimizations Drive Success

Nvidia's impressive results can be attributed to both hardware improvements and software optimizations:

1. Hardware enhancements:

   • 2X attention-layer acceleration
   • 1.5X more AI compute FLOPS
   • Increased memory capacity
   • Faster memory and connectivity

2. Software optimizations:

   • Use of Nvidia's proprietary 4-bit floating point format (NVFP4)
   • Disaggregated serving, separating prefill and generation/decoding stages
   • Model sharding across multiple GPUs for larger models like Llama 3.1 405B 1

     1

     2

     2

Implications for AI Development and Deployment

The performance gains demonstrated by Nvidia's Blackwell Ultra GB300 have significant implications for the development and deployment of AI systems:

1. Improved efficiency in running large language models
2. Potential for more cost-effective AI infrastructure
3. Enhanced capabilities for complex reasoning tasks and edge applications

As shipments of GB300 are set to start this month, these benchmark results position Nvidia as a leader in the rapidly evolving AI hardware market, potentially disrupting the economics of "AI factory" development 2

2