NVIDIA's Thermal Challenge: Innovative Cooling Solutions for Next-Gen AI GPUs

NVIDIA could change cooling solution for Rubin Ultra AI GPUs for huge 2300W thermal concerns

TL;DR: NVIDIA's next-gen Rubin Ultra AI GPUs will consume up to 2300W, prompting a shift to advanced microchannel cold plate cooling to manage extreme thermals. This innovation responds to rising power demands in AI accelerators, driven by competition with AMD's high-TDP Instinct MI450X, signaling a new era of ultra-powerful AI GPUs. NVIDIA's next-gen Rubin Ultra AI GPUs will reportedly consume quite an insane 2300W of power or so, with rumors swirling that NVIDIA is switching to a totally new cooling solution for its bleeding-edge AI chips. In a new post from @QQ_Timmy on X, we're hearing that NVIDIA is contacting cooling solution partners to integrate "direct-to-chip" cooling through microchannel cold plates for its beefed-up Rubin Ultra AI GPUs. This would be quite the move from conventional liquid cooling solutions, and keep the up to 2300W of thermals in check to hit those performance numbers that NVIDIA wants. The market previously rumored that cooling for Rubin AI GPUs with a 2300W TDP would use a microchannel cover plate (MCL) but @QQ_Timmy's post suggests that there has been high mass production difficulty for these cover plates. Additionally, the leaker has learned that NVIDIA has reportedly requested Asia Vital Components to design a microchannel cold plate (MCCP) for Rubin Ultra (launching in 2027). It looks like there are multiple cooling solutions in development right now, with the liquid cooling plate market size expected to continue to grow over the coming years, especially when there are 2300W+ AI GPUs on the market. Recently, we reported that AMD's next-gen Instinct MI450X AI accelerator "forced" NVIDIA to increase the TDP of its VR200 Rubin AI GPU. Rubin had an 1800W TGP just a few months ago, and then it was upgraded to 2300W. Instinct MI450X was a 2300W TGP design, and to increase competitiveness, AMD boosted it to 2500W. The world of super-power-hungry AI GPUs is here, and it's only going to get even more power-hungry.

NVIDIA Might Switch to Direct-to-Chip Cooling Solutions For Next-Gen Rubin Ultra as It Battles Thermal Constraints

NVIDIA is reportedly planning to switch to an entirely new cooling solution for the next-gen Rubin Ultra AI lineup, as it tries to mitigate the growing power requirements and thermal concerns. For those unaware, having an adequate cooling solution for next-generation systems is one of the top concerns for firms like NVIDIA, as power draw levels rise tremendously with each iteration, necessitating sufficient cooling mechanisms onboard. According to @QQ_Timmy, it is believed that NVIDIA is contacting cooling solution partners to integrate 'direct-to-chip' cooling through microchannel cold plates for its Rubin Ultra AI GPUs, which could prove to be a significant shift from conventional liquid cooling solutions and enable NVIDIA to achieve top-tier performance. Let's discuss microchannel cover plates (MCCP) mentioned here. It's somewhat similar to direct-die cooling, which overclocking enthusiasts often employ with modern CPUs, and it involves a copper cold plate with microchannels embedded inside that allows coolant to flow around. This allows the plate to create local convection, reducing thermal resistance from the die to the fluid. It is similar to NVIDIA's current conventional cooling solutions, except that the changes are made directly to the liquid-cooled plates on the chip, enabling significantly more efficient thermal management. I could delve into more specifics about how MCCP works, but it's essential to understand why NVIDIA requires this technology in the first place. Team Green is in a really tight product cadence, and scaling up from Blackwell to Rubin brings in much higher power requirements, especially in complex rack-scale systems, which is why it is necessary for NVIDIA to 'think out of the box'. Even if the company doesn't desire it, architectural advancements force them to invest in advanced cooling solutions. It is claimed that NVIDIA has contacted Asia Vital Components, a Taiwanese thermal solution provider, to design the MCCP for Rubin Ultra. However, it was initially planned for Rubin; the tight timeline has given suppliers insufficient time to switch to an advanced solution. Interestingly, Microsoft recently announced microfluidic cooling, an approach similar to MCCP, with the key difference being a focus on 'in-chip cooling,' where the coolant is located either inside or on the backside of the silicon itself. This demonstrates that the industry definitely needs new cooling options.