Microsoft Unveils Custom Chips and Infrastructure Upgrades to Boost AI Performance and Efficiency

Microsoft launches two data center infrastructure chips to speed AI applications

SAN FRANCISCO (Reuters) - Microsoft has designed two additional infrastructure chips for its data centers that will help speed artificial intelligence operations and increase data security, it said on Tuesday at its Ignite conference. Microsoft has devoted significant resources to develop home-grown silicon for general purpose applications and artificial intelligence. Like rivals Amazon.com and Google, Microsoft's engineers say there is a performance and price benefit to designing chips that are customized for its needs. Designing custom chips can reduce Microsoft's reliance on processors made by Intel and Nvidia. Microsoft's two new chips are designed to be installed deep within the company's data center infrastructure. One chip is designed to increase security and the other is for data processing. The company makes the effort to design an array of data center processors because it aims to "optimize every layer of infrastructure" and ensures that Microsoft's data centers crunch information at the speed AI requires, said Rani Borkar, corporate vice president, Azure Hardware Systems and Infrastructure Engineers will install the new security chip called the Azure Integrated HSM in every new server destined for a data center beginning next year. The chip aims to keep crucial encryption and other security data inside the security module. The data processing unit, or DPU, aims to move multiple components of a server into a single chip that is focused on cloud storage data. The company said it can run these specific tasks at three times less power and four times the performance compared with its current hardware. Microsoft also announced a new version of a cooling system for data center servers that relies on liquid to reduce the temperature of nearby components. The cooling unit can be used to support large-scale AI systems. (Reporting by Max A. Cherney in San Francisco; Editing by Leslie Adler)

Microsoft debuts custom chips to boost data center security and power efficiency

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More At the Ignite developer conference today, Microsoft unveiled two new chips designed for its data center infrastructure: the Azure Integrated HSM and the Azure Boost DPU. Scheduled for release in the coming months, these custom-designed chips aim to address security and efficiency gaps faced in existing data centers, further optimizing their servers for large-scale AI workloads. The announcement follows the launch of Microsoft's Maia AI accelerators and Cobalt CPUs, marking another major step in the company's comprehensive strategy to rethink and optimize every layer of its stack -- from silicon to software -- to support advanced AI. The Satya Nadella-led company also detailed new approaches aimed at managing power usage and heat emissions of data centers, as many continue to raise alarms over the environmental impact of data centers running AI. Just recently, Goldman Sachs published research estimating that advanced AI workloads are poised to drive a 160% increase in data center power demand by 2030, with these facilities consuming 3-4% of global power by the end of the decade. The new chips While continuing to use industry-leading hardware from companies like Nvidia and AMD, Microsoft has been pushing the bar with its custom chips. Last year at Ignite, the company made headlines with Azure Maia AI accelerator, optimized for artificial intelligence tasks and generative AI, as well as Azure Cobalt CPU, an Arm-based processor tailored to run general-purpose compute workloads on the Microsoft Cloud. Now, as the next step in this journey, it has expanded its custom silicon portfolio with a specific focus on security and efficiency. The new in-house security chip, Azure Integrated HSM, comes with a dedicated hardware security module, designed to meet FIPS 140-3 Level 3 security standards. According to Omar Khan, the vice president for Azure Infrastructure marketing, the module essentially hardens key management to make sure encryption and signing keys stay secure within the bounds of the chip, without compromising performance or increasing latency. To achieve this, Azure Integrated HSM leverages specialized hardware cryptographic accelerators that enable secure, high-performance cryptographic operations directly within the chip's physically isolated environment. Unlike traditional HSM architectures that require network round-trips or key extraction, the chip performs encryption, decryption, signing, and verification operations entirely within its dedicated hardware boundary. While Integrated HSM paves the way for enhanced data protection, Azure Boost DPU (data processing unit) optimizes data centers for highly multiplexed data streams corresponding to millions of network connections, with a focus on power efficiency. The offering, first in the category from Microsoft, complements CPUs and GPUs by absorbing multiple components of a traditional server into a single piece of silicon -- right from high-speed Ethernet and PCIe interfaces to network and storage engines, data accelerators and security features. It works with a sophisticated hardware-software co-design, where a custom, lightweight data-flow operating system enables higher performance, lower power consumption and enhanced efficiency compared to traditional implementations. Microsoft expects the chip will easily run cloud storage workloads at three times less power and four times the performance compared to existing CPU-based servers. New approaches to cooling, power optimization In addition to the new chips, Microsoft also shared advancements made towards improving data center cooling and optimizing their power consumption. For cooling, the company announced an advanced version of its heat exchanger unit - a liquid cooling 'sidekick' rack. It did not share the specific gains promised by the tech but noted that it can be retrofitted into Azure data centers to manage heat emissions from large-scale AI systems using AI accelerators and power-hungry GPUs such as those from Nvidia. On the energy management front, the company said it has collaborated with Meta on a new disaggregated power rack, aimed at enhancing flexibility and scalability. "Each disaggregated power rack will feature 400-volt DC power that enables up to 35% more AI accelerators in each server rack, enabling dynamic power adjustments to meet the different demands of AI workloads," Khan wrote in the blog. Microsoft is open-sourcing the cooling and power rack specifications for the industry through the Open Compute Project. As for the new chips, the company said it plans to install Azure Integrated HSMs in every new data center server starting next year. The timeline for the DPU roll-out, however, remains unclear at this stage.

Microsoft cranks up its AI infrastructure offerings with new VMs, cooling and power delivery tech - SiliconANGLE

Microsoft cranks up its AI infrastructure offerings with new VMs, cooling and power delivery tech Microsoft Corp. wants to be the infrastructure foundation for the next generation of artificial intelligence, catering to everyone from agile startups to multinational corporations, and to do this it's introducing some significant advancements and upgrades to the Azure Cloud platform. The new infrastructure updates announced at Microsoft Ignite 2024 today are all about offering customers more choices in terms of power, performance and cost efficiency. They cover advancements in everything from AI accelerators, computer chips and liquid cooling to data integrations and flexible cloud deployments. Perhaps the biggest news is the imminent arrival of Nvidia Corp.'s latest graphics processing unit, the eagerly anticipated Blackwell GPUs. Although reports elsewhere claim Nvidia is still having issues with its newest accelerators, Microsoft Vice President of Azure Infrastructure Marketing Omar Khan revealed that the company has already started co-validating the new chips ahead of the launch of a new virtual machine series called the Azure ND GB200 v6. It will combine the Nvidia GB200 NVL 72 rack-scale design with advanced Quantum InfiniBand networking infrastructure, which makes it possible to link "tens of thousands" of Blackwell GPUs for AI supercomputing, Khan said. However, he declined to mention a launch date for the new offering. On the other hand, customers will be able to access various new silicon options right now, including new versions of the Azure Maia AI accelerators and the Azure Cobalt central processing units. Another new option is Azure Boost DPU (below), which is Microsoft's first in-house data processing unit, designed to provide high performance with low energy consumption for data-centric workloads. According to Khan, it absorbs multiple components of a traditional server into a single, dedicated silicon chip, and makes it possible to run cloud storage workloads with three times less power and up to four times the performance. The company is also introducing a custom-made security chip, called Azure Integrated HSM, a dedicated hardware security module that puts encryption and key management in the hands of customers. Khan said that from next year, Microsoft plans to install Azure Integrated HSM in every new server it deploys in its data centers globally, increasing security for both confidential and general-purpose compute workloads. Coming soon, there's a new VM series called the Azure HBv5, which will be powered by Advanced Micro Devices Inc.'s new EPYC 9V64H processors. It's designed for high-performance computing workloads and is said to be eight times faster than the latest bare-metal and cloud alternatives, and as much as 35 times faster than on-premises servers. Khan said this is because the VMs feature seven terabytes per second of high bandwidth memory. It will launch sometime next year, but customers can sign up for preview access now. Besides chips, Microsoft has also been focused on improving the technology needed to keep those processors running cool. Its efforts there have paid off with the launch of a next-generation liquid cooling "sidekick rack," or heat exchanger unit (pictured below) for AI systems. It's designed to be retrofitted into the company's existing data centers, and can help to cool off GPUs and other AI accelerators, including those from Nvidia and Microsoft's in-house chips, the Azure Maia GPUs. Cooling chips down helps save on energy, and so does more efficient power delivery, which has been another area of focus for the company. Khan explained that Microsoft has collaborated with Meta Platforms Inc. on a new, disaggregated power rack that can help support the addition of more AI accelerators into its existing data centers. According to Khan, each of the new racks delivers 400 volts of DC power to support up to 35% more AI accelerators per server rack. They also enable dynamic power adjustments, so it can scale up or down as necessary to support different kinds of AI workloads. The new racks are being open-sourced via the Open Compute Project, so everyone can benefit. In other announcements, Khan said the company is improving its Oracle Database@Azure service, launched earlier this year, with support for Microsoft Purview, which is a new offering that can help customers to better manage and secure their Oracle data and enhance data governance and compliance. In addition, Microsoft has been working with Oracle to support another new offering. The tongue-twisting Oracle Exadata Database Service On Exascale Infrastructure on Oracle Database@Azure is said to deliver hyper-elastic scaling with pay-per-use economics for customers that want more flexible Oracle database options. That service will become available in more regions too, as Microsoft is expanding availability of Oracle Database@Azure to a total of nine regions globally. For customers with hybrid cloud deployment requirements, there's a new cloud-connected infrastructure offering called Azure Local, which is said to bring the capabilities of Azure Stack into a single, unified platform. Azure Stack is the on-premises version of the Azure Cloud, and enables customers to access many of Azure's cloud features from within their own data centers. With Azure Local, customers can run containers and Azure Virtual Desktop on validated servers from companies including Dell Technologies Inc., Hewlett Packard Enterprise Co. and Lenovo Group Ltd., among others, helping them to meet their compliance requirements. Windows Server 2025, which became generally available earlier this year, is getting some new features too, including simpler upgrades, enhanced security and support for AI and machine learning. There's also a new "hotpatching" subscription offering in preview that makes it possible to install new updates with fewer restarts, saving organizations time. Finally, Khan said Microsoft is making SQL Server 2025 available in preview, giving customers access to an enhanced database offering that leverages Azure Arch to provide cloud agility in any location. The new release is all about supporting AI workloads, with integrated AI application development tools and support for retrieval-augmented generation patterns and vector search capabilities.