NVIDIA Unveils GB200 NVL4: A Powerhouse AI Accelerator with Quad Blackwell GPUs and Dual Grace CPUs

NVIDIA Preps GB200 NVL4 Server with 4xBlackwell GPUs and 2xGrace CPUs

NVIDIA introduced the GB200 NVL4, a high-performance AI accelerator designed as a single-server solution that expands the company's Blackwell GPU lineup. This platform integrates four Blackwell GPUs and two Grace CPUs on a single motherboard, offering robust performance for artificial intelligence tasks. The GB200 NVL4 system includes 768 GB of HBM3E memory distributed across its four Blackwell GPUs, achieving a combined memory bandwidth of 32 TB/s. Additionally, the two Grace CPUs are equipped with 960 GB of LPDDR5X memory, enhancing the system's capability to manage demanding AI workloads. A key component of the NVL4 architecture is the NVLink interconnect technology, which enables efficient communication between all processors on the board. This interconnect is essential for maintaining optimal performance across multiple processing units, especially during large-scale training operations or when handling multi-trillion parameter models. Performance evaluations compared to previous generations show significant improvements, with NVIDIA stating that the GB200 GPUs deliver 2.2 times the overall performance and 1.8 times faster training speeds compared to the GH200 NVL4 predecessor. The system's power consumption is 5,400 watts, which is double the 2,700-watt requirement of the GB200 NVL2 model that features two GPUs instead of four. NVIDIA is collaborating closely with original equipment manufacturer (OEM) partners to bring various Blackwell-based solutions to market, including the DGX B200, GB200 Grace Blackwell Superchip, GB200 Grace Blackwell NVL2, GB200 Grace Blackwell NVL4, and GB200 Grace Blackwell NVL72. Incorporating a thermal design power (TDP) of 5,400 watts in a single server necessitates the use of liquid cooling to ensure optimal performance. The GB200 NVL4 is expected to be deployed in server racks for hyperscale customers, who typically utilize custom liquid cooling systems within their data centers. Sources: HardwareLuxx, via VideoCardz

NVIDIA unveils GB200 NVL4: four Blackwell GPUs, dual Grace GPUs, 1.7TB memory, 5400W of power

NVIDIA has announced its new GB200 NVL4, a new module that is a bigger extension to the original GB200 Grace Blackwell Superchip AI solution, doubling the CPU and GPU power while also increasing memory. The new NVIDIA GB200 NVL4 features 2 x Blackwell GB200 GPUs configured onto a larger board with 2 x Grace CPUs, with the new module designed as a single-server solution with 4-way NVLink domain controlling a larger 1.3TB pool of coherent memory. NVIDIA promises performance improvements over the previous-gen Hopper GH200 NVL4 with the new GB200 NVL4 module of 2.2x in simulation, and a 1.8x increase in Training and Inference performance. We know that the GB200 Grace Blackwell Superchip uses around 2700W of power, with the larger GB200 NVL4 solution to use around 6000W of power. The new GB200 NVL4 will share 1.3TB of coherent memory that is shared across all four GB200 GPUs over NVLink, with NVIDIA saying its new fifth generation NVLink chip-to-chip interconnect enables high-speed communication between teh CPUs and GPUs of up to 1.8TB/sec of bidirectional throughput per GPU. NVIDIA also unveiled an extension to its Hopper stack of products, with the introduction of the new H200 NVL which is the company's new PCIe-based Hopper cards that can connect with up to 4 GPUs through an NVLink domain, offering 7x the bandwidth than a standard PCIe solution. The company says that its new H200 NVL solution can fit into any data center, offering a range of flexible server configurations that can be optimized for hybrid HPC and AI workloads. NVIDIA's new H200 NVL solution has 1.5x more HBM memory, 1.7x the LLM inference performance, and 1.3x the HPC performance. There are 114 SMs in total with 14,592 CUDA cores, 456 Tensor Cores, and up to 3 TFLOPs of FP8 (FP16 accumulated) performance. There's 80GB of HBM2e memory on a 5120-bit memory interface and up to 350W TDP.

NVIDIA Announces GB200 NVL4 With Quad Blackwell GPUs & Dual Grace CPUs, H200 NVL Now Generally Available

NVIDIA has unveiled its latest Blackwell GB200 NVL4 solution with four GPUs & two GPUs packed into one powerful HPC & AI solution. NVIDIA Accelerates AI Forward With Strong Hardware Solutions: Blackwell GB200 NVL4 & Hopper H200 NVL As part of its SC24 announcements, NVIDIA is unveiling two brand-new hardware platforms, one based on its existing Hopper stack and the other powered by its latest Blackwell stack. These two solutions are designed for enterprise servers, powering accelerated HPC and AI workloads. Starting with the NVIDIA H200 NVL, the company is now confirming the general availability of these PCIe-based Hopper cards which can connect up to 4 GPUs through an NVLINK domain, offering seven times faster bandwidth than a standard PCIe solution. The company states that the H200 NVL solutions can fit into any data center and offer a range of flexible server configurations that are optimized for hybrid HPC and AI workloads. In terms of specifications, the Hopper H200 NVL solution offers 1.5x more HBM memory, 1.7x LLM inference performance, and 1.3x HPC performance. You are getting 114 SMs with a total of 14,592 CUDA cores, 456 tensor cores, and up to 3 TFLOPs of FP8 (FP16 accumulated) performance. The GPU features 80 Gb HBM2e memory configured across a 5120-bit interface and has a TDP of 350 Watts. Moving over to Blackwell, we have our first showcase of the NVIDIA GB200 NVL4, which is a brand-new module that is sort of a bigger extension of the original GB200 Grace Blackwell Superchip AI solution. The GB200 NVL4 module doubles the CPU and GPU capabilities and also adds increased memory. You are getting two Blackwell GB200 GPUs configured on a larger board with two Grace CPUs. The module is designed as a single-server solution with a 4-GPU NVLINK domain and 1.3T of Coherent memory. In terms of performance, the module will offer a 2.2x improvement in simulation, and a 1.8x uplift in Training and Inference performance. NVIDIA's growing list of partners will offer the NVL4 solution in the coming months. As for TDP, since the Superchip module is about 2700W, we can expect the bigger GB200 NVL4 solution to sip in close to 6KW of power. One can easily see the efforts being made by NVIDIA to accelerate the AI computing segment. The company recently published world records in MLPerf v4.1, both in training and inference, and it's not only that Blackwell is amazing, but Hopper is getting even better with continued optimization. The company is also accelerating its AI roadmap to a one-year cadence with several new infrastructures planned in the coming years, such as Blackwell Ultra and Rubin.

Nvidia Reveals 4-GPU GB200 NVL4 Superchip, Releases H200 NVL Module

At Supercomputing 2024, the AI computing giant shows off what is likely its biggest AI 'chip' yet -- the four-GPU Grace Blackwell GB200 NVL4 Superchip -- while it announces the general availability of its H200 NVL PCIe module for enterprise servers running AI workloads. Nvidia is revealing what is likely its biggest AI "chip" yet -- the four-GPU Grace Blackwell GB200 NVL4 Superchip -- in another sign of how the company is stretching the traditional definition of semiconductor chips to fuel its AI computing ambitions. Announced at the Supercomputing 2024 event on Monday, the product is a step up from Nvidia's recently launched Grace Blackwell GB200 Superchip that was revealed in March as the company's new flagship AI computing product. The AI computing giant also announced the general availability of its H200 NVL PCIe module, which will make the H200 GPU launched earlier this year more accessible for standard server platforms. [Related: 8 Recent Nvidia Updates: Nutanix AI Deal, AI Data Center Guidelines And More] The GB200 NVL4 Superchip is designed for "single server Blackwell solutions" running a mix of high-performance computing and AI workloads, said Dion Harris, director of accelerated computing at Nvidia, in a briefing last week with journalists. These server solutions include Hewlett Packard Enterprise's Cray Supercomputing EX154n Accelerator Blade, which was announced last week and packs up to 224 B200 GPUs. The Cray blade server is expected to become available by the end of 2025, per HPE. Whereas the GB200 Superchip resembles a slick black motherboard that connects one Arm-based Grace GPU with two B200 GPUs based on Nvidia's new Blackwell architecture, the NVL4 product seems to roughly double the Superchip's surface area with two Grace CPUs and four B200 GPUs on a larger board, according to an image shared by Nvidia. Like the standard GB200 Superchip, the GB200 NVL4 uses the fifth generation of Nvidia's NVLink chip-to-chip interconnect to enable high-speed communication between the CPUs and GPUs. The company has previously said that this generation of NVLink enables bidirectional throughput per GPU to reach 1.8 TB/s. Nvidia said the GB200 NVL4 Superchip features 1.3 TB of coherent memory that is shared across all four B200 GPUs using NVLink. To demonstrate the computing horsepower of the GB200 NVL4, the company compared it to the previously released GH200 NVL4 Superchip, which was originally introduced as the Quad GH200 a year ago and consists of four Grace Hopper GH200 Superchips. The GH200 Superchip contains one Grace CPU and one Hopper H200 GPU. Compared to the GH200 NVL4, the GB200 NVL4 is 2.2 times faster for a simulation workload using MILC code, 80 percent faster for training the 37-million-parameter GraphCast weather forecasting AI model and 80 percent faster for inference on the 7-billion-parameter Llama 2 model using 16-bit floating-point precision. The company did not provide any further specifications or performance claims. In the briefing with journalists, Harris said Nvidia's partners are expected to introduce new Blackwell-based solutions at Supercomputing 2024 this week. "The rollout of Blackwell is progressing smoothly thanks to the reference architecture, enabling partners to quickly bring products to market while adding their own customizations," he said. In addition to revealing the GB200 NVL4 Superchip, Nvidia announced that its previously announced H200 NVL PCIe card will become available in partner systems next month. The NVL4 module contains Nvidia's H200 GPU that launched earlier this year in the SXM form factor for Nvidia's DGX system as well as HGX systems from server vendors. The H200 is the successor to the company's H100 that uses the same Hopper architecture and helped make Nvidia into the dominant provider of AI chips for generative AI workloads. What makes it different from a standard PCIe design is that the H200 NVL consists of either two or four PCIe cards that are connected together using Nvidia's NVLink interconnect bridge, which enables a bidirectional throughput per GPU of 900 GB/s. The product's predecessor, the H100 NVL, only connected two cards via NVLink. It's also air-cooled in contrast to the H200 SXM coming with options for liquid cooling. The dual-slot PCIe form factor makes the H200 NVL "ideal for data centers with lower power, air-cooled enterprise rack designs with flexible configurations to deliver acceleration for every AI and HPC workload, regardless of size," according to Harris. "Companies can use their existing racks and select the number of GPUs that best fit their needs, from one, two, four or even eight GPUs with NVLink domain scaling up to four," he said. "Enterprises can use H200 NVL to accelerate AI and HPC applications while also improving energy efficiency through reduced power consumption." Like its SXM cousin, the H200 NVL comes with 141GB of high-bandwidth memory and 4.8 TB/s of memory bandwidth in contrast to the H100 NVL's 94-GB and 3.9-TB/s capacities, but its max thermal design power only goes up to 600 watts instead of the 700-watt maximum of the H200 SXM version, according to the company. This results in the H200 NVL having slightly less horsepower than the SXM module. For instance, the H200 NVL can only reach 30 teraflops in 64-bit floating point (FP64) and 3,341 teraflops in 8-bit integer (INT8) performance whereas the SXM version can achieve 34 teraflops in FP64 and 3,958 teraflops in INT8 performance. (A teraflop is a unit of measurement for one trillion floating-point operations per second.) When it comes to running inference on a 70-billion-parameter Llama 3 model, the H200 NVL is 70 percent faster than the H100 NVL, according to Nvidia. As for HPC workloads, the company said the H200 NVL is 30 percent faster for reverse time migration modeling. The H200 NVL comes with a five-year subscription to the Nvidia AI Enterprise software platform, which comes with Nvidia NIM microservices for accelerating AI development.

Nvidia GB200 packs 4 GPUs, 2 CPUs in to 5.4 kW board

SC24 Nvidia's latest HPC and AI chip is a massive single board computer packing four Blackwell GPUs, 144 Arm Neoverse cores, up to 1.3 terabytes of HBM, and a scorching 5.4 kilowatt TDP. In many respects, the all-new GB200 NVL4 form factor, detailed at Supercomputing in Atlanta this week, looks like two of Nvidia's upcoming Grace-Blackwell Superchips that have been stitched together. However, unlike the 2.7 kilowatt GB200 boards we've looked at previously, the speedy NVLink communications found on Nvidia's NVL4 configuration are limited to the four Blackwell GPUs and two Grace CPUs on the board. All I/O going on or off the board will be handled by your typical Ethernet or InfiniBand NICs. While this might seem like an odd choice, it actually aligns pretty neatly with how many HPC systems have been constructed. For example, the Cray EX Blades found in Frontier featured one third-gen Epyc CPU alongside four MI250X accelerators. This also means that the major HPC system builders such as HPE, Eviden, and Lenovo aren't limited to Nvidia's proprietary interconnects for scaling up and out. Both HPE and Eviden have their own interconnect tech. In fact, HPE has already teased new EX systems, due out in late 2025, that'll make use of Nvidia's GB200 NVL4 boards. The EX154n, announced last week, will pack up to 56 of the extra-wide Superchips - one per blade - into its massive liquid-cooled HPC cabinets. In this configuration, a single EX cabinet can churn out over 10 petaFLOPS of FP64 vector or matrix compute. That may sound like a lot, but if highly precise scientific computing is all you care about, HPE's AMD-based systems do offer higher floating point performance. The MI300A APUs found in Cray's EX255a blades each boast 61.3 teraFLOPS of vector FP64 or 122.6 teraFLOPS of matrix FP64 compared to the 45 teraFLOPS of double-precision vector/matrix performance seen in each Blackwell GPU. For AI-centric workloads, the performance gap is much narrower as each MI300A can output 3.9 petaFLOPS of sparse FP8 performance. So, for a fully packed EX cabinet, you'd be looking at roughly 2 exaFLOPS of FP8 grunt compared to about 2.2 exaFLOPS from the Blackwell system using less than half the GPUs - double that if you can take advantage of FP4 datatypes not supported by the MI300A. While HPE Cray is among the first to announce support for Nvidia's NVL4 form factor, we don't expect it'll be long before Eviden, Lenovo, and others start rolling out compute blades and servers of their own based on the design. Alongside Nvidia's double-wide GB200 NVL4, Nvidia has also announced general availability for its PCIe-based H200 NVL config. But before you get too excited, similar to the H100 NVL we got in early 2023, the H200 NVL is essentially just a bunch of double-width PCIe cards - up to four this time around - which have been glued together with an NVLink bridge. As with Nvidia's larger SXM-based DGX and HGX platforms, this NVLink bridge allows the GPUs to pool compute and memory resources to handle larger tasks without bottlenecking on slower PCIe 5.0 x16 interfaces that cap out at roughly 128 GBps of bidirectional bandwidth compared to 900 GBps for NVLink. Maxed out, the H200 NVL can support up to 564 GB of HBM3e memory and 13.3 petaFLOPS of peak FP8 performance with sparsity. Again, this is because it's just four H200 PCIe cards stuck together with a really fast interconnect bridge. However, all of that perf comes at the expense of power and thermals. Each H200 card in the four-stack is rated for up to 600 W of power or 2.4 kilowatts in total. With that said, the approach does have its advantages. For one, these cards can be deployed in just about any 19-inch rack server with sufficient space, power, and airflow to keep them cool. ®

Hopper Scales New Heights, Accelerating AI and HPC Applications for Mainstream Enterprise Servers

NVIDIA H200 NVL is ideal for lower-power, air-cooled enterprise rack designs and offers up to 1.7x faster large language model inference and 1.3x more performance on high-performance computing applications. Since its introduction, the NVIDIA Hopper architecture has transformed the AI and high-performance computing (HPC) landscape, helping enterprises, researchers and developers tackle the world's most complex challenges with higher performance and greater energy efficiency. During the Supercomputing 2024 conference, NVIDIA announced the availability of the NVIDIA H200 NVL PCIe GPU -- the latest addition to the Hopper family. H200 NVL is ideal for organizations with data centers looking for lower-power, air-cooled enterprise rack designs with flexible configurations to deliver acceleration for every AI and HPC workload, regardless of size. According to a recent survey, roughly 70% of enterprise racks are 20kW and below and use air cooling. This makes PCIe GPUs essential, as they provide granularity of node deployment, whether using one, two, four or eight GPUs -- enabling data centers to pack more computing power into smaller spaces. Companies can then use their existing racks and select the number of GPUs that best suits their needs. Enterprises can use H200 NVL to accelerate AI and HPC applications, while also improving energy efficiency through reduced power consumption. With a 1.5x memory increase and 1.2x bandwidth increase over NVIDIA H100 NVL, companies can use H200 NVL to fine-tune LLMs within a few hours and deliver up to 1.7x faster inference performance. For HPC workloads, performance is boosted up to 1.3x over H100 NVL and 2.5x over the NVIDIA Ampere architecture generation. Complementing the raw power of the H200 NVL is NVIDIA NVLink technology. The latest generation of NVLink provides GPU-to-GPU communication 7x faster than fifth-generation PCIe -- delivering higher performance to meet the needs of HPC, large language model inference and fine-tuning. The NVIDIA H200 NVL is paired with powerful software tools that enable enterprises to accelerate applications from AI to HPC. It comes with a five-year subscription for NVIDIA AI Enterprise, a cloud-native software platform for the development and deployment of production AI. NVIDIA AI Enterprise includes NVIDIA NIM microservices for the secure, reliable deployment of high-performance AI model inference. Companies Tapping Into Power of H200 NVL With H200 NVL, NVIDIA provides enterprises with a full-stack platform to develop and deploy their AI and HPC workloads. Customers are seeing significant impact for multiple AI and HPC use cases across industries, such as visual AI agents and chatbots for customer service, trading algorithms for finance, medical imaging for improved anomaly detection in healthcare, pattern recognition for manufacturing, and seismic imaging for federal science organizations. Dropbox is harnessing NVIDIA accelerated computing for its services and infrastructure. "Dropbox handles large amounts of content, requiring advanced AI and machine learning capabilities," said Ali Zafar, VP of Infrastructure at Dropbox. "We're exploring H200 NVL to continually improve our services and bring more value to our customers." The University of New Mexico has been using NVIDIA accelerated computing in various research and academic applications. "As a public research university, our commitment to AI enables the university to be on the forefront of scientific and technological advancements," said Prof. Patrick Bridges, director of the UNM Center for Advanced Research Computing. "As we shift to H200 NVL, we'll be able to accelerate a variety of applications, including data science initiatives, bioinformatics and genomics research, physics and astronomy simulations, climate modeling and more." Some systems are based on the NVIDIA MGX modular architecture, which enables computer makers to quickly and cost-effectively build a vast array of data center infrastructure designs. Platforms with H200 NVL will be available from NVIDIA's global systems partners beginning in December. To complement availability from leading global partners, NVIDIA is also developing an Enterprise Reference Architecture for H200 NVL systems. The reference architecture will incorporate NVIDIA's expertise and design principles, so partners and customers can design and deploy high-performance AI infrastructure based on H200 NVL at scale. This includes full-stack hardware and software recommendations, with detailed guidance on optimal server, cluster and network configurations. Networking is optimized for the highest performance with the NVIDIA Spectrum-X Ethernet platform. NVIDIA technologies will be showcased on the showroom floor at SC24, taking place at the Georgia World Congress Center through Nov. 22. To learn more, watch NVIDIA's special address.

Nvidia launches H200 NVL high-performance GPU to power AI supercomputing - SiliconANGLE

Nvidia launches H200 NVL high-performance GPU to power AI supercomputing Nvidia Corp. today announced the availability of its newest data center-grade graphics processing unit, the H200 NVL, to power artificial intelligence and high-performance computing. The company announced its new PCIe form factor card during the SC24 high-performance computing conference in Atlanta alongside a bevy of updates to enable high-performance AI for scientific research and engineering. The H200 NVL is the latest addition to Nvidia's Hopper family of GPUs designed for data centers running on lower-power, air-cooled racks. According to the company, about 70% of enterprise data center floors run at 20,000-kilowatt hours and below, which allows for air cooling. This makes PCIe GPUs essential for flexibility, since they don't require specialized cooling systems. The new GPU provides a 1.5-times memory efficiency increase and 1.2-times bandwidth increase over the H100 NVL, the company said, and businesses can use it to fine-tune large language models within a few hours with up to 1.7-times faster performance when deployed. The University of Mexico said it has tapped Nvidia's accelerated computing technology for scientific research and academic applications. "As a public research university, our commitment to AI enables the university to be on the forefront of scientific and technological advancements," said Prof. Patrick Bridges, director of the UNM Center for Advanced Research Computing. "As we shift to H200 NVL, we'll be able to accelerate a variety of applications, including data science initiatives, bioinformatics and genomics research, physics and astronomy simulations, climate modeling and more." To help build on and accelerate the use of AI in scientific research, Nvidia announced the release of the open-source Nvidia BioNeMo framework to assist with drug discovery and molecule design. The platform accelerated computing tools to scale up AI models for biomolecular research to bring supercomputing to biopharma engineering. "The convergence of AI, accelerated computing and expanding datasets offers unprecedented opportunities for the pharmaceutical industry, as evidenced by recent Nobel Prize wins in chemistry," said Kimberly Powell, vice president of healthcare at Nvidia. AI has quickly emerged as a potent discovery mechanism for new chemicals and materials. To explore potential materials within the nearly infinite combinations Nvidia announced ALCHEMI which allows researchers to optimize AI models via an NIM microservice. These microservices enable users to deploy prebuilt foundation models in the cloud, data center or workstations. Using ALCHEMI, researchers can test chemical compounds and materials in a simulated virtual AI lab without the need to spend extra time trying and failing to build structures in an actual lab. According to Nvidia, the new microservices can be run on an H100 using a pretrained model for materials chemistry testing for long-term stability 100 times faster than with the NIM off. For example, evaluating 16 million structures could take months, with the microservice it would take mere hours. Extreme weather also presents a problem for scientists, which is why Nvidia announced two new NIM microservices for Earth-2, a platform used for simulating and visualizing weather and climate conditions. With this technology, climate researchers can use AI assistance to forecast extreme weather events - from local to global occurrences. According to a report from Bloomberg, natural disasters have been responsible for around $62 billion in insured losses during the first half of 2024, or about 70% more than the 10-year average. This included increasing wildfires and severe thunderstorms across the United States with even more costly storms and torrential flooding in Europe. Nvidia is releasing CorDiff NIM and ForCastNet NIM, which will help weather technology companies build high-resolution and accurate predictions. CorDiff uses generative AI at kilometer-scale resolution for weather forecasting. Nvidia used the model's capability to gain better-localized predictions and analysis of typhoons over Taiwan earlier this year. CorDiff was trained on the Weather Research and Forecasting model's numerical simulation to generate patterns at 12 times higher resolution. According to Nvidia CorDiff NIM runs 500 times faster and is 100,000 times more energy efficient than traditional prediction using central processing units. FourCastNet provides global, medium-range, lower-resolution forecasts for systems that don't need highly localized information. For example, if a provider wants to get an idea of what might happen across the next two weeks given current conditions, FourCastNet can provide a forecast 5,000 times faster than traditional numerical weather models.