Red Hat Unveils AI 3: A Leap Forward in Enterprise AI Deployment and Management

Red Hat AI 3 targets production inference and agents - SiliconANGLE

IBM Corp. subsidiary Red Hat today announced Red Hat AI 3, calling it a major evolution of its hybrid cloud-native artificial intelligence that can power enterprise projects in production at scale. Red Hat AI 3 is designed to manage AI workloads that span datacenters, clouds and edge environments while maintaining flexibility and control. A hybrid architecture is key to the strategy. "AI platforms aren't going to run a single model on a single inference server on a single machine," said Joe Fernandes, vice president and general manager of Red Hat's AI business unit. "You're going to have multiple models across multiple inference servers across a distributed environment." The release bundles enhancements across the open-source giant's OpenShift AI, Enterprise Linux AI and the AI Inference Server into one platform aimed at solving the messiness of enterprise AI inference using multiple tools and frameworks. The platform is designed to help organizations transition from experimentation to operational AI across platforms. "Generative AI adoption is exploding across enterprises in every vertical," Fernandes said. "We see challenges arising around the rising costs of generative AI, particularly as the number of use cases grows and those use cases begin to scale out to production deployments." At the heart of Red Hat AI 3 is inference, the compute-hungry cousin of model training, where applications run. With AI 3, Red Hat is investing heavily in this operational layer, building on open-source projects like the vLLM open-source library and introducing llm-d, a new distributed inference engine designed to bring intelligence to how LLMs are scheduled and served on the Kubernetes orchestrator for portable software containers. "With llm-d, customers can adopt an intelligent AI platform that integrates seamlessly with Kubernetes," said Steven Huels, Red Hat's vice president of AI engineering. "Kubernetes scheduling helps maximize model performance and utilization of the underlying [graphics processing unit hardware so they're not sitting there idle." Huels emphasized that LLM inference diverges from traditional application behavior. "Traditional apps are uniform and predictable," he said. "LLMs handle variable requests. Traditional app services are often stateless, but with LLMs, we're trying to maintain state and rely heavily on KVCache," a technique used in transformer-based language models to speed up the process of generating text. On top of the inference layer, Red Hat AI 3 includes a new model-as-a-service function that uses an integrated AI gateway powered by Red Hat Connectivity Link. The idea is to make models accessible as simple, scalable endpoints with usage tracking and plug-and-play support for third-party software. Enterprises that want to serve models internally for cost, privacy or compliance reasons can operate like a commercial model-serving service within their own environments. The platform also anticipates the rise of agent-based systems. To support agents, Red Hat AI 3 adds the Llama Stack application programming interface layer and integrates the emerging Model Context Protocol. "We give you the flexibility to choose whatever framework you prefer," Fernandes said. "We want to be the platform that can support all agents regardless of how they're built." Red Hat AI 3 also introduces a model customization toolkit based on the InstructLab open-source project that supports community contributions to large language models. It provides specialized Python libraries that give developers greater flexibility and control and uses the open-source Docling project to streamline the ingestion of unstructured documents into an AI-readable format. "This allows AI engineers to develop and tune models the way that they're most accustomed," Huels said. Red Hat is also enhancing retrieval-augmented generation, integration with Kubeflow workflow automation and launching Feature Store, a component within the OpenShift AI platform that provides a centralized repository for storing, managing and serving machine learning features.

Red Hat AI 3 Aims To Give Partners More Ways To Scale Workloads For Customers

Partners can 'become AI providers themselves for their enterprise customers,' says Joe Fernandes, vice president and general manager of Red Hat's AI business unit. Red Hat has unveiled the third version of its enterprise artificial intelligence platform, leveraging Models as a Service, a catalog of validated and optimized AI models, and a generative AI studio as part of its goal to allow organizations to rapidly scale and distribute AI workloads across hybrid, multi-vendor environments. Red Hat AI 3 gives solution providers another path to manage customers' AI use cases, Joe Fernandes, vice president and general manager of Raleigh, N.C.-based Red Hat's AI business unit, said in response to a CRN reporter's question during a press briefing. Even for customers starting with AI vendors such as OpenAI or Google, Red Hat AI 3 can enable flexibility to work across multiple clouds, at the edge, on-premises and in a mix of environments, he said. Red Hat AI 3 helps users go to "the next level" of multi-environment AI use cases beyond what they do with Red Hat OpenShift and Red Hat Enterprise Linux (RHEL). "This opens up another area where we can partner with our SI [systems integrator] or managed service provider partners to help customers deploy these platforms, manage these platforms and take advantage of these technologies," he said. "Rather than just consuming from the AI providers in the cloud, they become AI providers themselves for their enterprise customers." [RELATED: Red Hat Launches RHEL 10 With New Capabilities For Hybrid Cloud And AI Systems] The IBM division's latest version of Red Hat AI promises users improved cross-team collaboration on agents and other AI workloads leveraging a common platform. Red Hat AI 3's open standards foundation should support any model and hardware accelerator from data centers to public cloud and sovereign AI environments. Red Hat AI 3 adds Model-as-a-Service (MaaS) capabilities built on distributed inference. IT teams can provide their own MaaS and serve common models centrally with on-demand access for developers and applications, improving cost management. This should also allow for AI use cases that can't run on public services due to privacy and data concerns, according to the vendor. Red Hat AI 3 also now has a hub for exploring, deploying and managing AI assets. In addition, it has a catalog of validated and optimized generative AI models, a registry for managing model life cycles, and a deployment environment for configuring and monitoring AI assets running on Red Hat OpenShift AI. The platform's GenAI studio offers an environment for AI engineers to work with models, prototype GenAI apps and discover and consume available models and Model Context Protocol servers through an AI asset endpoint feature. Engineers can use the studio to experiment, test prompts and tune parameters for chat, retrieval-augmented generation and other use cases. OpenAI's gpt-oss, DeepSeek-R1, the Whisper speech-to-text model and Voxtral Mini voice-enabled agents model are among the new Red Hat-validated and optimized models available for Red Hatt AI 3 users, according to the vendor. Other recent AI innovations by the vendor include a developer preview of the offline version of RHEL command-line assistant, which uses AI to power guidance and suggestions to users, plus general availability of Red Hat Ansible Automation Platform 2.6, which integrates the Ansible Lightspeed intelligent assistant directly into the user interface to aid with troubleshooting, on-boarding and more.