Nvidia's Blackwell AI GPU Cooling Issues: Overblown or Resolved?

Nvidia May Have Already Fixed Blackwell's Cooling Issues

Earlier this week, a report from The Information said Nvidia's Blackwell AI chips were delayed due to an overheating issue that developed when they were placed on server racks, but a third-party research firm claims the problem is overblown and was fixed months ago. Blackwell, which is designed for businesses looking to build out their AI data centers, has server racks that can fit up to 72 GPUs. But Semianalysis, a research firm that focuses on the semiconductor and AI industries, tells Business Insider that Nvidia suppliers reworked the server racks with "minor" changes to address the problem. According to the firm's chief analyst, cooling may be a concern in the future, but the specific server issue in question has been fixed. Nvidia said earlier this week that it is "working with leading cloud service providers as an integral part of our engineering team and process" regarding any potential issues and that "engineering iterations are normal and expected." Overheating GPUs can throttle performance and cause operational issues. Their immediate surroundings (like the number of nearby fans, type of case, or rack design) can directly impact GPU temperature, as well. The GPU's design can also result in higher average temperatures depending on the specific model. But Georgia Tech Professor Bara Cola -- who is also the founder of Carbice, which develops thermal computing solutions -- argues that heat itself isn't Blackwell's biggest challenge. "The real challenge is mechanical stress and not heat. I am confident that Nvidia will find a way to operate these chips for their customers. High-performance chips like this will always run hot, and it is just a matter of balancing how hot -- smart engineers will solve this," Cola tells PCMag via email. "But early failure happens when the interfaces cannot handle the thermal expansion stress that the heat brings. This is a hard materials science problem." Blackwell previously had a "design flaw" unrelated to the server overheating issue. Nvidia CEO Jensen Huang has also said this has since been resolved.

Nvidia's Blackwell AI GPU overheating issues are seemingly overhyped -- semiconductor analysts reveal cooling issues have been mostly addressed

Blackwell's cooling issues aren't as severe as some might have thought. Reports of Nvidia's GB200 NVL72 server racks overheating have purportedly been exaggerated. Business Insider reports that Blackwell's cooling design faults have already been addressed. Dylan Patel, chief analyst at Semianalysis, purportedly told Business Insider that Blackwell's design issues, which have been present for months, have been largely addressed, stating that the overheating issues are largely overblown. Semianalysis' five analysts monitoring the semiconductor industry reported that the cooling system issues triggering "reworks" from several suppliers were a "minor" change. Blackwell's cooling faults have been specifically problematic with Nvidia's massive 72-chip server rack, which can consume up to 120kW. Design flaws in the rack's design have forced Nvidia to reevaluate its design multiple times due to the GPUs inside overheating. This has setback shipments of Nvidia's GB200 hardware, causing additional delays due to the required design changes. Nvidia's B200 GPUs are the most potent processing chips for AI workloads. The GB200 superchip, for instance, has a configurable TDP in the thousands of watts, with a peak rating of up to 2,700 watts. These absurdly high power figures make air cooling virtually impossible to use in the constraints of a standard rack mount form factor. This physics problem has forced Nvidia to require liquid cooling on its latest Blackwell GPUs. It also requires data centers to revamp their server farms to accommodate the infrastructure needed to support liquid-cooled servers. Nvidia could solve this problem by creating slower air-cooled GPUs -- which the GPU manufacturer still does, in the form of GPUs such as the H200 NVL. However, to remain at the bleeding edge of the AI GPU arms race, Nvidia is prioritizing performance no matter the cost, which is why the company has opted to make GPUs that require thousands of watts of power at the expense of air-cooling. The good news is that Nvidia's 72-chip Blackwell cooling issues are apparently minor and have been largely addressed already. In addition, only Nvidia's flagship 72-chip server rack is having the problem.