Microsoft's Microfluidics Breakthrough: A Game-Changer in AI Chip Cooling

Microsoft Is Turning to the Field of Microfluidics to Cool Down AI Chips

One of the major reasons why AI data centers are sucking up so much power is the need to cool processors that run very hot. But Microsoft Corp. is trying out a possible solution: sending fluid directly through tiny channels etched into the chips. The technology is called microfluidics, and it's being used in prototype systems in test conditions at the company, said Husam Alissa, who oversees Microsoft's systems technology. The technique has been applied to server chips used for Office cloud apps and the graphics processing units that handle artificial intelligence tasks, he said.

Microsoft claims a 'breakthrough' in AI chip cooling

AI is an enormous energy drain, contributing to greenhouse gas emissions at a time when the planet desperately needs progress in the opposite direction. Although most of that comes from running GPUs, cooling them is another significant overhead. So, it's worth noting when a company of Microsoft's stature claims to have achieved a breakthrough in chip cooling. Microsoft's new system is based on microfluidics, a method long pursued but hard to implement. The company claims its approach could lead to three times better cooling than current methods. Many data centers rely on cold plates to prevent GPUs from overheating. Although effective to a degree, the plates are separated from the heat source by several layers of material, which limits their performance. "If you're still relying heavily on traditional cold plate technology [in five years], you're stuck," Microsoft program manager Sashi Majety is quoted as saying in the company's announcement. In microfluidics, the coolant flows closer to the source. The liquid in Microsoft's prototype moves through thread-like channels etched onto the back of the chip. The company also used AI to more efficiently direct the coolant through those channels. Another aspect separating this prototype from previous attempts is that it drew inspiration from Mother Nature. As you can see in the image above, the etchings resemble the veins in a leaf or a butterfly wing. Microsoft says the technique can reduce the maximum silicon temperature rise inside a GPU by 65 percent. (However, that number depends on the workload and chip type.) This would enable overclocking "without worrying about melting the chip down," Microsoft's Jim Kleewein said. It could allow the company to place servers closer together physically, reducing latency. It would also lead to "higher-quality" waste heat use. Although this sounds good for the environment in a general sense, Microsoft's announcement doesn't lean into that. The blog post primarily discusses the technique's potential for performance and efficiency gains. Green benefits are only alluded to briefly as "sustainability" and reduced grid stress. Let's hope that's only a case of a cynical observer overanalyzing framing. Our planet needs all the help it can get.

Microsoft tames intense chip heat with liquid cooling veins, designed by AI and inspired by biology

REDMOND, Wash. -- About 5 minutes before each hour, and 5 minutes after it, Microsoft Teams gets busy with people joining meetings early or late. Yes, you know who you are. Those predictable spikes in demand are an example of a larger infrastructure challenge facing the industry: either build data centers full of chips and servers that sit idle for much of the time, waiting to handle peak loads, or risk performance issues during moments of high demand. Microsoft has come up with a different approach. The company said Tuesday that it has successfully prototyped new advances in the field of microfluidics that let its servers run hotter and faster -- a technique known as overclocking -- without the risk of melting down the silicon. "We've got these really spiky workloads -- I would dearly love to be able to overclock ... because then we would need way fewer servers and deliver a better experience," said Jim Kleewein, a Microsoft technical fellow and development director for core Office 365 services, recounting the challenge he originally brought to Microsoft's silicon development teams. Microfluidics overall is not a new concept. Instead of placing a cooling plate on top of a chip, the approach brings liquid coolant directly inside the processor itself through tiny channels. But Microsoft says it has developed a new approach that uses AI to customize the cooling system for the unique heat signatures of different chips running specific workloads. The design, inspired by biology, ends up resembling the veins in a leaf, delivering coolant with greater precision to a chip's hottest spots. The company says it's up to three times more effective at removing heat than current cold plates, making overclocking more feasible. 'Surface of the sun' Some of the hot spots on a chip "could have the same heat flux as the surface of the sun when you look at the very small scale," said Husam Alissa, Microsoft's director of systems technology, referring to a technical measure of heat intensity, or density in a specific area. While the working prototype was demonstrated on a commercially available Intel Xeon chip, Microsoft says the innovation is part of a broader strategy to improve its entire hardware fleet. As a next step, the company is looking to incorporate the cooling technology into future versions of its own silicon, potentially including its Azure Cobalt chip and Maia AI accelerator. Those first-party chips, unveiled in 2023, reflect Microsoft's efforts to gain control over the final piece in its cloud platform. The company is competing to efficiently train and run cutting-edge AI models against Amazon, Google, and others, which are also making their own chips. Microsoft still uses third-party chips, including industry-standard GPUs from Nvidia and other suppliers, but it says a more integrated approach is necessary to meet the challenges of the AI era. The goal isn't to replace partners but to advance the broader ecosystem, said Rani Borkar, the corporate vice president who leads Azure hardware systems and infrastructure. "The demands of Al are such that, frankly, hardware on its own cannot keep up," Borkar said during a recent media briefing and tour of Microsoft's Silicon Lab in Redmond. "We really are all about co-designing, co-optimizing every layer of the stack." Networking and steel The company made two related announcements Tuesday morning: * Microsoft is working with Corning and Heraeus to move to industrial scale production of its hollow core fiber technology, which transmits pulses of light through a hollow channel vs. a conventional solid glass core, letting data move faster and with lower latency. * In an initiative focused on sustainability, Microsoft said it will procure "green steel" from partner Stegra for use in datacenter construction. The steel is produced in a plant designed to reduce carbon emissions by up to 95% vs. traditional steelmaking. As for the newly announced microfluidics advances, Microsoft says it will work with partners and the broader tech community to make the technology an industry standard. Longer term, the company sees the cooling technology as a foundational step toward a revolutionary new type of chip architecture: 3D stacking. While stacking layers of silicon dramatically reduces latency by shortening the distance data has to travel, it has been largely unfeasible because of the immense challenge of removing heat from the inner layers. Microfluidics could eventually solve that by allowing coolant to flow between each layer of the silicon brick, a breakthrough that Microsoft's Kleewein believes could be transformative. "That is where this could move from, 'it's an interesting change that the rest of the industry should adopt,' to a 'holy shit' moment in the evolution of technology," he said.

Microsoft in-chip cooling breakthrough cuts GPU heat rise by 65%

Microsoft has announced a breakthrough in cooling technology that could reshape the future of artificial intelligence hardware. The company successfully tested microfluidic cooling, a system that channels liquid directly inside a chip to remove heat. Early results show it can outperform today's advanced cold plate systems by as much as three times. AI chips are growing hotter with each generation. Current systems like cold plates sit on top of chips but remain separated by layers that limit efficiency.