Red Hat Expands AI Capabilities with RHEL AI 1.3 Release

Red Hat expands AI model support in new release of Linux AI platform - SiliconANGLE

Red Hat expands AI model support in new release of Linux AI platform IBM's Corp.'s Red Hat subsidiary today released to general availability the latest generation of Red Hat Enterprise Linux AI, a version of the company's core Linux platform optimized for developing, testing and running artificial intelligence large language models. RHEL AI provides a foundation model for building and running LLMs and includes image mode, which allows users to build, deploy and manage RHEL as a bootable container image. Version 1.3 supports the latest release of the Granite LLMs that IBM announced and released to open source in October. It also has improved data preparation features, expanded choices for hybrid cloud deployment and support for Intel Corp.'s Gaudi 3 AI accelerator. Citing a recent International Data Corp. report that found 61% of enterprises plan to use open-source foundation models for generative AI, Red Hat said it's orienting its operating system features to support smaller, open-source-licensed models, fine-tuning capabilities and inference performance engineering. The new release also pays homage to the company's corporate parent with support for Granite and InstructLab, an initiative birthed at IBM that aims to accelerate open-source contributions to generative AI development. The new release supports Granite 3.0 8b, an 8 billion-parameter converged model that supports more than a dozen natural languages and has code generation and function-calling capabilities. Non-English languages, code generation and function features are available as a developer preview within RHEL AI 1.3, with the expectation that the capabilities will be supported in future RHEL AI releases. It also includes support for Docling, an open-source project developed by IBM Research that allows PDFs, manuals and slide decks to be converted into specialized data formats such as JavaScript Object Notation and Markdown, a lightweight language that allows formatting elements to be added to plain text without using tags or a formal text editor. Users can convert documents into Markdown for simplified data ingestion for model tuning with InstructLab. Docling includes context-aware chunking, a method used in natural language processing to break down text or data into smaller, meaningful segments while considering the surrounding context. This helps resulting applications deliver more coherent and contextually appropriate responses to questions and tasks out of the box. Gaudi 3 is Intel's answer to Nvidia Corp.'s H100 graphics processing unit, which has since been succeeded by the H200 series. Intel has said Gaudi 3 can inference at up to 2.3 times the power efficiency of the H100 while speeding up LLM training times. Red Hat already supports GPUs from Nvidia and Advanced Micro Devices Inc. Red Hat OpenShift AI, which natively supports RHEL AI, now supports parallelized serving across multiple nodes with the vLLM runtimes, enabling multiple requests to be handled in real-time. VLLM is a high-performance inference engine for serving LLMs at low latency and high throughput. Users can dynamically alter an LLM's parameters when being served, such as sharding - or distributing - the model across multiple GPUs or quantizing it to a smaller footprint.

Red Hat Adds Granite, Gaudi Support In Latest AI Release

'Our services partners and systems integrators are the ones helping companies explore and integrate different use cases in a cost-effective way,' says Joe Fernandes, Red Hat's AI business unit vice president and general manager. Red Hat's latest release of its Enterprise Linux AI foundation model platform adds Granite large language model support and comes with previews for code generation and function calling capabilities as well as Intel Gaudi 3 accelerator support, bringing more opportunities to its services partners. The Raleigh, N.C.-based open-source tools vendor and IBM subsidiary's latest version of RHEL AI -- now generally available -- is a prime area for Red Hat services partners and systems integrators "helping customers implement and integrate that into their use cases and business," Joe Fernandes, Red Hat's AI business unit vice president and general manager, told CRN in an email. "AI in general and generative AI in particular is complicated, so our services partners and systems integrators are the ones helping companies explore and integrate different use cases in a cost-effective way," Fernandes said. "That maps directly to how Red Hat aims to reduce costs (with smaller models), eliminate complexity (around integrating those models with customer data and use cases) and enable flexibility (to deploy those models wherever they need to run across a hybrid environment)." [RELATED: The 10 Coolest GenAI Products And AI Tools Of 2024] According to CRN's 2024 Channel Chiefs, Red Hat has been working to increase the overall percentage of company revenue that comes through the channel and improve partner profitability. RHEL AI is meant for developing, testing and running GenAI models for enterprise applications, the vendor said in a statement Thursday. Version 1.3 adds support for the latest innovations in parent IBM's open-source licensed Granite large language models (LLMs) and leverages open-source technology for data preparation. RHEL AI brings together Granite with the InstructLab model alignment project created by Red Hat and IBM. Users can leverage the components for a packaged, bootable RHEL image for individual server deployments across hybrid clouds, according to Red Hat. Version 1.3 supports Granite 3.0 8b English language use cases. This version has a developer preview for using this model's non-English language, code generation and function calling capabilities, with full support coming in a future RHEL AI release. The new RHEL AI version also supports the Docling IBM Research open-source project for converting common document formats into Markdown, JSON and other formats for GenAI applications and training. The new version can do context-aware chunking, accounting for the structure and semantic elements of documents used for GenAI training with the goal of improving GenAI responses. RHEL AI added a technology preview for Intel Gaudi 3 accelerator support with this release and now supports parallelized serving across multiple nodes for multiple requests in real time, according to the vendor. Users can dynamically alter LLM parameters during serving. Other capabilities slated for future RHEL AI releases will support additional document formats for Docling, integration for retrieval-augmented generation pipelines and InstructLab knowledge tuning.