Red Hat Acquires Neural Magic to Enhance AI Performance Across Hybrid Cloud Environments

Red Hat Acquires Neural Magic to Boost Hybrid Cloud AI Performance

Red Hat, an open-source solutions provider, has announced its acquisition of Neural Magic, a Massachusetts-based company specialising in software and algorithms to accelerate generative AI (gen AI) inference workloads. The acquisition aims to make high-performance AI accessible across hybrid cloud environments, addressing key challenges in deploying large language models (LLMs), which typically require costly and resource-intensive infrastructure. Also Read: L&T Technology Services Acquires Intelliswift to Boost AI and Digital Engineering Capabilities "Neural Magic's expertise in inference performance engineering and commitment to open source aligns with Red Hat's vision of high-performing AI workloads that directly map to customer-specific use cases and data, anywhere and everywhere across the hybrid cloud," Red Hat said in a statement this week. Red Hat intends to address the challenges of building cost-efficient and reliable LLM services which requires significant computing power, energy resources and specialised operational skills by making gen AI more accessible to more organisations through the open-source vLLM. Developed by UC Berkeley, vLLM is a community-driven open-source project for open model serving (how gen AI models infer and solve problems), with support for all key model families, advanced inference acceleration research and diverse hardware backends, including AMD GPUs, AWS Neuron, Google TPUs, Intel Gaudi, Nvidia GPUs and x86 CPUs. Red Hat says Neural Magic's expertise in the vLLM project, combined with Red Hat's portfolio of hybrid cloud AI technologies, will offer organisations an open pathway to building AI strategies that meet their unique needs. Also Read: L&T Partners with E2E Networks to Drive GenAI Cloud Solutions in India Founded in 2018 as a spinout from MIT, Neural Magic has developed technologies for optimising AI models, particularly in model inference performance. Red Hat intends to leverage this expertise in the open-source vLLM project to democratise AI by offering scalable, cost-efficient options for businesses of all sizes, supporting various model types and hardware backends. "AI workloads need to run wherever customer data lives across the hybrid cloud; this makes flexible, standardised and open platforms and tools a necessity, as they enable organisations to select the environments, resources and architectures that best align with their unique operational and data needs," said Matt Hicks, President and CEO of Red Hat. Neural Magic uses its expertise in vLLM to build an enterprise-grade inference stack which enables customers to optimise, deploy and scale LLM workloads across hybrid cloud environments with full control over infrastructure choice, security policies and model lifecycle. Neural Magic also conducts model optimisation research, develops the LLM Compressor (a unified library for optimising LLMs with state-of-the-art sparsity and quantisation algorithms) and maintains a repository of pre-optimised models ready to deploy with vLLM. Also Read: HCLTech to Open AI, Cloud Native Lab in Singapore to Accelerate AI Innovation With Neural Magic's capabilities, Red Hat plans to enhance its AI portfolio, including Red Hat Enterprise Linux AI (RHEL AI) for developing foundation models, Red Hat OpenShift AI for machine learning across Kubernetes environments, and InstructLab, a collaborative project with IBM to advance open-source models. This expanded portfolio will enable enterprises to fine-tune and deploy AI models with flexibility and security across corporate data centers, cloud platforms, and edge locations, Red Hat said. Brian Stevens, CEO, Neural Magic added, "Open source has proven time and again to drive innovation through the power of community collaboration. At Neural Magic, we've assembled some of the industry's top talent in AI performance engineering with a singular mission of building open, cross-platform, ultra-efficient LLM serving capabilities." Dario Gil, IBM senior vice president and director of Research, said, "As our clients look to scale AI across their hybrid environments - virtualised, cloud-native LLMs built on open foundations will become the industry standard. Red Hat's leadership in open source combined with the choice of efficient, open source models like IBM Granite and Neural Magic's offerings for scaling AI across platforms empower businesses with the control and flexibility that they need to deploy AI across the enterprise." The transaction is subject to applicable regulatory reviews and other customary closing conditions.

Red Hat to acquire AI optimization startup Neural Magic - SiliconANGLE

Open-source software giant Red Hat Inc. announced today that it has agreed to acquire Neural Magic Inc., a machine learning startup that optimizes AI models to run whatever hardware is available, including central processing units or graphics processing units. As AI large language models grow in size and complexity, they also consume more compute power and energy. As a result, businesses have seen the need to build more cost-efficient LLMs designed for efficiency as well as accuracy that take advantage of the hardware they're deployed on. Founded in 2018 by Massachusetts Institute of Technology Research Scientist Alex Matveev and Professor Nir Shavit, Neural Magic's technology allows for the optimization of LLMs to run on off-the-shelf CPUs and GPUs at similar speeds to specialized AI chips. That means a company can fine-tune and optimize AI models to maximize for hardware efficiency across numerous hardware architectures. "AI workloads need to run wherever customer data lives across the hybrid cloud," said Matt Hicks, president and chief executive of Red Hat. "This makes flexible, standardized and open platforms and tools a necessity, as they enable organizations to select the environments, resources and architectures that best align with their unique operational and data needs." Neural Magic has contributed significantly to an open-source project for model deployment called vLLM, which Red Hat said brought the company to its attention. The project provides a fast, efficient library for serving and deploying AI models optimized for the cloud that supports numerous processor and GPU architectures. The company has also designed a specialized LLM compressor that provides faster inference with vLLM allowing faster delivery of AI models. The startup also keeps a repository of prebuilt, pre-optimized open-source LLMs ready to deploy with vLLM so businesses can quickly deploy high-performance models at scale. Red Hat said Neural Magic's leadership in vLLM will help build on the company's ability to support LLM deployments for the hybrid cloud with any hardware back end. This acquisition represents an important move for Red Hat's open-source AI strategy. The company recently launched Red Hat Enterprise Linux AI, which brings together the open-source Granite family of LLMs developed by IBM Corp. with Red Hat's InstructLab open-source community-driven model alignment tools, packaged as optimized, bootable server deployments. "Open source has proven time and again to drive innovation through the power of community collaboration," said Neural Magic CEO Brian Stevens. "At Neural Magic, we've assembled some of the industry's top talent in AI performance engineering with a singular mission of building open, cross-platform, ultra-efficient LLM serving capabilities."

Red Hat Acquiring Neural Magic To Bolster Open-Source AI Offerings

"Red Hat intends to address these challenges by making gen AI more accessible to more organizations through the open innovation of vLLM. Developed by UC Berkeley, vLLM is a community-driven open source project for open model serving (how gen AI models infer and solve problems), with support for all key model families, advanced inference acceleration research and diverse hardware backends including AMD GPUs, AWS Neuron, Google TPUs, Intel Gaudi, NVIDIA GPUs and x86 CPUs. Neural Magic's leadership in the vLLM project combined with Red Hat's strong portfolio of hybrid cloud AI technologies will offer organizations an open pathway to building. ... Neural Magic's technology leadership in vLLM will enhance Red Hat AI's ability to support LLM deployments anywhere and everywhere across the hybrid cloud with a ready-made, highly-optimized and open inference stack." "Neural Magic's core offering is based around the vLLM open source technology, with additional proprietary code. Red Hat has long shown its commitment to open-sourcing the technology it acquires, and we have no reason to expect a change in this approach. Our specific plans and timeline will be determined over the coming months."

Red Hat Announces Definitive Agreement to Acquire Neural Magic By Investing.com

Proposed transaction underscores Red Hat's commitment to enabling customers to deliver any application or workload " including AI " anywhere across the hybrid cloud, from on-premises to public clouds to the edge Neural Magic to bring expertise in generative AI performance engineering and state-of-the-art model optimization algorithms and high-performance GPU and CPU inference serving RALEIGH, N.C.--(BUSINESS WIRE)--Red Hat, Inc., the world's leading provider of open source solutions, today announced that it has signed a definitive agreement to acquire Neural Magic, a pioneer in software and algorithms that accelerate generative AI (gen AI) inference workloads. Neural Magic's expertise in inference performance engineering and commitment to open source aligns with Red Hat's vision of high-performing AI workloads that directly map to customer-specific use cases and data, anywhere and everywhere across the hybrid cloud. While the promise of gen AI dominates much of today's technology landscape, the large language models (LLMs) underpinning these systems continue to increase in size. As a result, building cost-efficient and reliable LLM services requires significant computing power, energy resources and specialized operational skills. These challenges effectively put the benefits of customized, deployment-ready and more security-conscious AI out of reach for most organizations. Red Hat intends to address these challenges by making gen AI more accessible to more organizations through the open innovation of vLLM. Developed by UC Berkeley (LON:BKGH), vLLM is a community-driven open source project for open model serving (how gen AI models infer and solve problems), with support for all key model families, advanced inference acceleration research and diverse hardware backends including AMD (NASDAQ:AMD) GPUs, AWS Neuron, Google (NASDAQ:GOOGL) TPUs, Intel (NASDAQ:INTC) Gaudi, NVIDIA (NASDAQ:NVDA) GPUs and x86 CPUs. Neural Magic's leadership in the vLLM project combined with Red Hat's strong portfolio of hybrid cloud AI technologies will offer organizations an open pathway to building AI strategies that meet their unique needs, wherever their data lives. Red Hat + Neural Magic: Enabling a future of hybrid cloud-ready gen AI Neural Magic spun out of MIT in 2018 with the goal of building performant inference software for deep learning. With Neural Magic's technology and performance engineering expertise, Red Hat aims to accelerate its vision for AI's future, powered by the Red Hat AI technology portfolio. Built to break through the challenges of wide-scale enterprise AI, Red Hat uses open source innovation to further democratize access to AI's transformative power via: vLLM leadership to enhance Red Hat AI Neural Magic uses its expertise and knowledge in vLLM to build an enterprise-grade inference stack which enables customers to optimize, deploy and scale LLM workloads across hybrid cloud environments with full control over infrastructure choice, security policies and model lifecycle. Neural Magic also develops model optimization research, builds LLM Compressor (a unified library for optimizing LLMs with state-of-the-art sparsity and quantization algorithms) and maintains a repository of pre-optimized models ready to deploy with vLLM. Red Hat AI aims to help customers lower AI's costs and skill barriers with powerful technologies, including: Neural Magic's technology leadership in vLLM will enhance Red Hat AI's ability to support LLM deployments anywhere and everywhere across the hybrid cloud with a ready-made, highly-optimized and open inference stack. The transaction is subject to applicable regulatory reviews and other customary closing conditions. AI workloads need to run wherever customer data lives across the hybrid cloud; this makes flexible, standardized and open platforms and tools a necessity, as they enable organizations to select the environments, resources and architectures that best align with their unique operational and data needs. We're thrilled to complement our hybrid cloud-focused AI portfolio with Neural Magic's groundbreaking AI innovation, furthering our drive to not only be the ~Red Hat' of open source, but the ~Red Hat' of AI as well. Brian Stevens, CEO, Neural Magic Open source has proven time and again to drive innovation through the power of community collaboration. At Neural Magic, we've assembled some of the industry's top talent in AI performance engineering with a singular mission of building open, cross-platform, ultra-efficient LLM serving capabilities. Joining Red Hat is not only a cultural match, but will benefit companies large and small in their AI transformation journeys. Dario Gil, IBM senior vice president and director of Research As our clients look to scale AI across their hybrid environments " virtualized, cloud-native LLMs built on open foundations will become the industry standard. Red Hat's leadership in open source combined with the choice of efficient, open source models like IBM Granite and Neural Magic's offerings for scaling AI across platforms empower businesses with the control and flexibility that they need to deploy AI across the enterprise. Red Hat is the world's leading provider of enterprise open source software solutions, using a community-powered approach to deliver reliable and high-performing Linux, hybrid cloud, container, and Kubernetes technologies. Red Hat helps customers integrate new and existing IT applications, develop cloud-native applications, standardize on our industry-leading operating system, and automate, secure, and manage complex environments. Award-winning support, training, and consulting services make Red Hat a trusted adviser to the Fortune 500. As a strategic partner to cloud providers, system integrators, application vendors, customers, and open source communities, Red Hat can help organizations prepare for the digital future. Forward-Looking Statements Except for the historical information and discussions contained herein, statements contained in this press release may constitute forward-looking statements within the meaning of the Private Securities Litigation Reform Act of 1995. Forward-looking statements are based on the company's current assumptions regarding future business and financial performance. These statements involve a number of risks, uncertainties and other factors that could cause actual results to differ materially. Any forward-looking statement in this press release speaks only as of the date on which it is made. Except as required by law, the company assumes no obligation to update or revise any forward-looking statements. Red Hat, Red Hat Enterprise Linux, the Red Hat logo, and OpenShift are trademarks or registered trademarks of Red Hat, Inc. or its subsidiaries in the U.S. and other countries. Linux ® is the registered trademark of Linus Torvalds in the U.S. and other countries.