Nvidia Dominates New AI Benchmarks, Showcasing Industry Shift Towards Generative AI

MLCommons Introduces New AI Benchmarks

MLCommons, an industry consortium, has unveiled two new benchmarks designed to measure the speed and efficiency of hardware and software in running AI applications. These benchmarks come in response to the growing demand for efficient AI infrastructure following the launch of ChatGPT over two years ago 2

2

3

Focus on Generative AI

The new benchmarks specifically target generative AI applications, reflecting the industry's shift towards large language models (LLMs) and chatbots. One benchmark is based on Meta's Llama 3.1 405-billion-parameter AI model, testing systems on general question answering, math, and code generation. The other simulates consumer AI applications like ChatGPT, emphasizing quick response times 2

2

3

Nvidia's Dominance in Benchmark Results

Nvidia's GPUs have once again demonstrated their superiority in AI performance:

1. Nvidia's systems, including those built by SuperMicro, Hewlett Packard Enterprise, and Lenovo, took top honors in most MLPerf benchmark tests 1

   1

   .
2. The new Grace Blackwell servers, featuring 72 Nvidia GPUs, showed 2.8 to 3.4 times faster performance compared to the previous generation, even when using only eight GPUs for a direct comparison 2

   2

   .

Competition and Notable Performances

While Nvidia dominated, other companies also made notable contributions:

1. AMD's MI300X GPU, in a system built by startup MangoBoost, outperformed Nvidia in two Llama 2 70b tests, producing 103,182 tokens per second 1

   1

   .
2. Google submitted a system showcasing its Trillium chip (TPU v6), though it trailed behind Nvidia's Blackwell in the Stable Diffusion image-generation test 1

   1

   .
3. Intel's Xeon microprocessors powered seven of the top 11 systems in the datacenter closed division, showing improvement over previous years 1

   1

   .

Related Stories

Nvidia's Blackwell GPUs Dominate Latest MLPerf AI Training Benchmarks

Nvidia's Blackwell Ultra GB300 Dominates MLPerf Benchmarks with Significant Performance Gains

NVIDIA Blackwell Dominates MLPerf Inference Benchmarks, AMD's MI325X Challenges Hopper

NVIDIA's Blackwell B200 GPU Shatters AI Performance Records in MLPerf Inference Benchmark

Implications for the AI Industry

These benchmarks highlight several important trends:

1. The industry's focus on developing hardware specifically optimized for generative AI workloads 2

   2

   .
2. The importance of efficient chip connections in AI servers, as demonstrated by Nvidia's efforts to speed up inter-chip communication 2

   2

   3

   3

   .
3. A potential shift in the competitive landscape, with new entrants like MangoBoost making an impact 1

   1

   .

Absence of Key Players

Notably, some major players were absent from this round of benchmarks:

1. Intel's Habana unit did not submit any entries, unlike in previous years 1

   1

   .
2. Qualcomm, a mobile chip giant, also did not participate in this round 1

   1

   .

As the AI industry continues to evolve rapidly, these benchmarks provide valuable insights into the current state of AI hardware and software capabilities, guiding future development and investment in the field.