IBM Teams Up with Groq to Accelerate Enterprise AI Deployment

IBM partners with Nvidia rival Groq to accelerate AI deployment - SiliconANGLE

IBM partners with Nvidia rival Groq to accelerate AI deployment IBM Corp. and Groq Inc. today announced a strategic partnership aimed at speeding enterprise deployment of agentic artificial intelligence by combining IBM's watsonx Orchestrate with Groq's hardware-accelerated inference technology. The collaboration will give IBM clients access to Groq's language processing unit through IBM's platform with the aim of reducing the cost of low-latency AI at large scale. Groq is a hardware-centric AI infrastructure company specializing in inference acceleration through its proprietary LPU architecture. IBM said Groq's custom LPU delivers more than five times faster and more cost-efficient inference than traditional graphics processing units such as those made by Nvidia Corp., maintaining consistent performance even as workloads scale up. The companies will also extend support for virtual large language models and Red Hat's llm-d framework for large-scale, distributed inference on Groq's LPU architecture. IBM's Granite models will run on GroqCloud, an inference platform that provides developers with access to high-speed, low-cost processing for large language models. IBM said Groq provides a secure, compliant platform for AI deployment that meets the most stringent regulatory and security requirements. Watsonx Orchestrate is used to automate business processes using prebuilt or custom-designed AI agents. Integration with watsonx Orchestrate is aimed at helping IBM customers speed purpose-built agents to production faster and at lower cost. Groq has raised $1.8 billion in funding, including a $750 million round last month that valued the company at $6.9 billion. IBM said a combination of speed and orchestration is critical for regulated industries such as healthcare and finance. It cited one health insurance provider that uses Groq technology to handle thousands of patient and provider questions simultaneously and in real time. "Many large enterprise organizations have a range of options with AI inferencing when they're experimenting, but when they want to go into production, they must ensure complex workflows can be deployed successfully to ensure high-quality experiences," Rob Thomas, IBM senior vice president of software and chief commercial officer, said in a statement. IBM said access to Groq's capabilities is available immediately.

IBM partners with Groq in agentic AI

On Monday, IBM announced the signing of a strategic and technological partnership with the Groq AI platform to enable its customers to access inference workloads on GroqCloud from watsonx Orchestrate, its agentic AI tool. Under the agreement, IBM plans to integrate RedHat's large open-source virtual language model with Groq's language processing unit (LPU) architecture. IBM Granite language models will also be supported on GroqCloud for IBM customers. IBM explains that the project aims to facilitate the transition from the pilot phase of agentic AI to production in areas such as healthcare, finance, public administration, distribution, and manufacturing by reducing costs and accelerating deployment.