AWS Unveils Innovative Liquid Cooling and Energy Efficiency Measures for AI Data Centers

How AWS's $75B plan uses water to cool AI data centers

Amazon.com Inc.'s cloud division, Amazon Web Services (AWS), is launching new designs aimed at enhancing data center efficiency to mitigate the increasing demand on the electrical grid. The updates include advanced cooling techniques, alternative fuel options for backup generators, and an improved server rack layout. Some of these components have already been implemented, with additional features set to debut as new data centers open. This initiative responds to the energy-intensive nature of server farms that power on-demand computing services. AWS plans to invest approximately $75 billion in capital expenditures this year, primarily directed toward server and chip upgrades, including cooling systems. The investment reflects AWS's commitment to addressing energy usage while enhancing its AI infrastructure. At its upcoming re:Invent conference, the company is expected to introduce its latest custom-designed chips, including advanced AI products that will compete with established offerings from Nvidia. One of the most significant updates is the shift to liquid cooling systems for AWS's AI servers. This technology is essential for maintaining optimal performance in high-powered chips from Nvidia and AWS's homegrown Trainium devices. AWS emphasizes that the liquid cooling integration is flexible, allowing for both air and liquid cooling in a single system. This multimodal design is intended to maximize performance and efficiency across various workloads, addressing the unique demands of AI applications. Furthermore, AWS is pursuing a simplified approach to electrical distribution and mechanical designs for its servers. This strategy could enhance infrastructure availability to 99.9999%, significantly reducing the number of server racks susceptible to electrical disturbances by up to 89%. This improvement is likely achieved by minimizing the conversions from AC to DC power, which typically lead to energy losses. AWS unveils next-level AI tools to modernize customer support These new cooling systems and streamlined designs aim to support a remarkable sixfold increase in rack power density over the next two years, with additional growth anticipated thereafter. By incorporating AI into its operational strategies, AWS is employing predictive analytics to optimize server rack positioning, thereby further reducing energy waste attributed to underutilized power. Ian Buck, Nvidia's vice president of hyperscale and high-performance computing, acknowledged that advanced liquid cooling solutions will efficiently cool AI infrastructure while minimizing energy usage. Both companies are working closely to refine the rack design specifically for liquid cooling applications, which is expected to benefit shared customers significantly. Prasad Kalyanaraman, AWS's vice president of Infrastructure Services, stated that these improvements are critical strides toward increasing energy efficiency and modularity. "AWS continues to relentlessly innovate its infrastructure to build the most performant, resilient, secure, and sustainable cloud for customers worldwide," stated Kalyanaraman. "These data center capabilities represent an important step forward with increased energy efficiency and flexible support for emerging workloads. But what is even more exciting is that they are designed to be modular, so that we are able to retrofit our existing infrastructure for liquid cooling and energy efficiency to power generative AI applications and lower our carbon footprint."

AWS integrates liquid cooling and simplifies electrical distribution to lower data center power consumption - SiliconANGLE

AWS integrates liquid cooling and simplifies electrical distribution to lower data center power consumption Amazon Web Services Inc. said today at its annual re:Invent conference that it's making some major changes to its cloud computing infrastructure. Among other things, it's introducing a new liquid cooling system and simplifying the way electricity is distributed throughout its facilities. The cloud computing giant said the updates help to make its infrastructure services four-times more efficient than typical on-premises environments, positioning it to handle the increased computing requirements of next-generation artificial intelligence workloads. Not only are its data centers more efficient, but they can also help to reduce the carbon footprint associated with AI and other computing workloads by up to 99%, the company claimed. The biggest change is that AWS is starting to roll out a new liquid cooling system for its AI servers and some other high-performance systems, including those powered by its homemade Trainium chips and Nvidia's powerful graphics processing units. Specifically, AWS said its new Trainium2 chips and Nvidia's rack-scale AI supercomputing chip, the GB200 NVL72, will both benefit from the new cooling system. According to AWS, the liquid cooling will help to reduce mechanical energy consumption by up to 46% during peak cooling conditions, without any additional water being required. What's interesting is that its cooling infrastructure will remain flexible, as the new systems can also use air for cooling when liquid is not required. So, the hundreds of servers in AWS's data centers that perform regular computing tasks, and those that handle networking and storage, for example, probably won't be cooled using liquid. Such systems are powered by much less energy-intensive central processing units, which don't generate nearly as much heat, meaning liquid cooling is probably overkill. According to the company, "this flexible multimodal cooling design allows AWS to provide maximum performance and efficiency at the lowest cost, whether running traditional workloads or AI models." Adding to the liquid cooling is a brand new, simplified electrical distribution system that's said to reduce the amount of potential failure points by up to 20%. It does this by minimizing the number of electrical conversions required, bringing backup power sources closer to its data center racks, and reducing the number of fans used to exhaust hot air from its servers. The company explained that it's using the natural pressure differential instead of fans to expel this hot air, meaning more energy can be directed to its servers, reducing overall power consumption. There's also a new power shelf design being implemented that will increase rack power density by approximately six-times in the next two years, followed by another three-times increase in the future. It will enable AWS to generate a 12% increase in compute power at every site it operates, helping it to reduce the total number of facilities it needs to operate. The company explained that many of these changes are being made to power AI workloads, but it's also using AI itself to try and improve the efficiency of its data center designs. The company has created AI models that can design more efficient rack configurations within each data center facility it owns, to reduce the amount of underutilized power. AI will also be integrated in a new control system for electrical and mechanical devices within its data centers, with integrated telemetry services, enabling real-time diagnostics. Another way AWS is reducing its carbon footprint is by switching to a new type of renewable diesel for its backup generators. In future, they'll run on what the company describes as a "biodegradable and non-toxic fuel" that results in 90% fewer carbon emissions over the course of its lifetime compared to regular diesel made from fossil fuels. Finally, AWS said it's making some changes to the way it builds its data centers, with plans to use lower-carbon steel and concrete. Apparently, the steel will be produced by electric arc furnaces rather than gas-fired furnaces, meaning fewer carbon emissions resulting from its production. Meanwhile, the carbon in its concrete mix has been reduced by up to 35% compared to the construction industry average. It will also reduce the overall amount of steel and concrete that it uses, thanks to the new, AI-optimized rack placement system it's using. AWS plans to implement these changes globally in every new data center it builds, with some also coming to its existing facilities. "These data center capabilities represent an important step forward with increased energy efficiency and flexible support for emerging workloads," said Prasad Kalyanaraman, head of AWS Infrastructure Services. "But what is even more exciting is that they are designed to be modular, so that we are able to retrofit our existing infrastructure for liquid cooling and energy efficiency to power generative AI applications and lower our carbon footprint."

AWS bets on liquid cooling for its AI servers

It's AWS re:Invent this week, Amazon's annual cloud computing extravaganza in Las Vegas, and as is tradition, the company has so much to announce, it can't fit everything into its five (!) keynotes. Ahead of the show's official opening, AWS on Monday detailed a number of updates to its overall data center strategy that are worth paying attention to. The most important of these is that AWS will soon start using liquid cooling for its AI servers and other machines, no matter whether those are based on its homegrown Trainium chips and Nvidia's accelerators. Specifically AWS notes that its Trainium2 chips (which are still in preview) and "rack-scale AI supercomputing solutions like NVIDIA GB200 NVL72" will be cooled this way. It's worth highlighting that AWS stresses that these updated cooling systems can integrate both air and liquid cooling. After all, there are still plenty of other servers in the data centers that handle networking and storage, for example, that don't require liquid cooling. "This flexible, multimodal cooling design allows AWS to provide maximum performance and efficiency at the lowest cost, whether running traditional workloads or AI models," AWS explains. The company also announced that it is moving to more simplified electrical and mechanical designes for its servers and server racks. "AWS's latest data center design improvements include simplified electrical distribution and mechanical systems, which enable infrastructure availability of 99.9999%. The simplified systems also reduce the potential number of racks that can be impacted by electrical issues by 89%," the company notes in its announcement. In part, AWS is doing this by reducing the number of times the electricity gets converted on its way from the electrical network to the server. AWS didn't provide many more details than that, but this likely means using DC power to run the servers and/or HVAC system and avoiding many of the AC-DC-AC conversion steps (with their default losses) otherwise necessary. "AWS continues to relentlessly innovate its infrastructure to build the most performant, resilient, secure, and sustainable cloud for customers worldwide," said Prasad Kalyanaraman, vice president of Infrastructure Services at AWS, in Monday's announcement. "These data center capabilities represent an important step forward with increased energy efficiency and flexible support for emerging workloads. But what is even more exciting is that they are designed to be modular, so that we are able to retrofit our existing infrastructure for liquid cooling and energy efficiency to power generative AI applications and lower our carbon footprint." In total, AWS says, the new multimodal cooling system and upgraded power delivery system will let the organization "support a 6x increase in rack power density over the next two years, and another 3x increase in the future." In this context, AWS also notes that it is now using AI to predict the most efficient way to position racks in the data center to reduce the amount of unused or underutilized power. AWS will also rool out its own control system across its electrical and mechanical devices in the data center, which will come with built-in telemetry services for real-time diagnostics and troubleshooting. "Data centers must evolve to meet AI's transformative demands," said Ian Buck, vice president of hyperscale and HPC at NVIDIA. "By enabling advanced liquid cooling solutions, AI infrastructure can be efficiently cooled while minimizing energy use. Our work with AWS on their liquid cooling rack design will allow customers to run demanding AI workloads with exceptional performance and efficiency."

AWS says its AI data centers just got even more efficient - here's how

At its annual re:Invent conference in Las Vegas on Monday, Amazon's AWS cloud computing service disclosed numerous changes to its computing infrastructure -- with a big emphasis on liquid cooling and renewable energy -- to make the case that its service is four times more efficient than on-premise computing. Also: The best web hosting services: Expert tested and reviewed "AWS's infrastructure is currently up to 4.1 times more efficient than on-premises infrastructure, and when workloads are optimized on AWS, the associated carbon footprint can be reduced by up to 99%," the company said in a press release, citing a report it commissioned and posted online. The innovations unveiled Monday include: Prasad Kalyanaraman, head of AWS Infrastructure Services, said that the changes represent "an important step forward with increased energy efficiency and flexible support for emerging workloads. They are designed to be modular, so that we are able to retrofit our existing infrastructure for liquid cooling and energy efficiency to power generative AI applications and lower our carbon footprint." Also: Amazon launches App Studio and new gen AI guardrails at AWS Summit The company said it has "achieved its goal to match all of the electricity consumed by its operations with 100% renewable energy - seven years ahead of its 2030 goal." The release includes numerous endorsements from partners and customers including Nvidia, Anthropic, and Lexis/Nexis. Re:Invent runs through Friday, December 6. You can register for free to watch the live stream on the re:Invent site. Keynotes include AWS CEO Matt Garman on Tuesday 8 a.m. PT.