IBM Unveils Telum II: A Powerful AI-Focused Processor for Next-Gen Mainframes

IBM intros Telum II processor -- 5.5GHz chip with onboard DPU claimed to be up to 70% faster

IBM has announced its next-generation Telum II processor with a built-in AI accelerator for next-generation IBM Z mainframes that can handle both mission-critical tasks and AI workloads. The new processor can potentially improve performance "by up to 70% across key system components," compared to the original Telum, released in 2021, according to an email we received from IBM. The Telum II processor packs eight high-performance cores with improved branch prediction, store writeback, and address translation operating at 5.5GHz as well as 36MB of L2 cache, a 40% increase over its predecessor. The CPU also supports virtual L3 and L4 caches expanding to 360MB and 2.88GB, respectively. A key feature of Telum II is its improved AI accelerator, which delivers four times the computational power of its predecessor, reaching 24 trillion operations per second (TOPS) with INT8 precision. The accelerator's architecture is optimized for handling AI workloads in real time with low latency. In addition, Telum II has a built-in DPU for faster transaction processing. Telum II is made on Samsung's 5HPP process technology and contains 43 billion transistors. System-level improvements in Telum II allow each AI accelerator within a processor drawer to receive tasks from any of the eight cores, ensuring balanced workloads and maximizing the available 192 TOPS per drawer across all accelerators when fully configured. In addition to Telum II, IBM introduced its new Spyre AI accelerator add-in-card developed in collaboration with IBM Research and IBM. This processor contains 32 AI accelerator cores and shares architectural similarities with the AI accelerator in Telum II. The Spyre Accelerator can be integrated into the I/O subsystem of IBM Z through PCIe connections to boost the system's AI processing power. Spyre packs 26 billion transistors and is made on Samsung's 5LPE production node. Both the Telum II processor and the Spyre Accelerator are designed to support ensemble AI methods, which involve using multiple AI models to improve the accuracy and performance of tasks. An example of this is in fraud detection, where combining traditional neural networks with large language models (LLMs) can significantly enhance the detection of suspicious activities, according to IBM. Both the Telum II processor and Spyre Accelerator will be available in 2025, though IBM does not specify whether it will be early in the year or late in the year.

IBM unveils Telum II CPU with 8 cores at 5.5GHz, Spyre AI accelerator: 300+ TOPS, 128GB LPDDR5

IBM has just unveiled its new Telum II processor and Spyre AI accelerator, which it plans to use inside of its new IBM Z mainframe systems powering AI workloads. The company provided details of the architecture of its new Telum II processor and Spyre AI accelerator, which are designed for AI workloads on the next-gen IBM Z mainframes. The new mainframes will accelerate traditional AI workloads, as well as LLMs using a brand new ensemble method of AI. IBM's new Telum II processor features 8 high-performance cores running at 5.5GHz, with 36MB L2 cache per core and a 40% increase in on-chip cache capacity for a total of 360MB. The virtual level-4 cache of 2.88GB per processor drawer provides a 40% increase over the previous generation. The integrated AI accelerator allows for low-latency, high-throughput in-transaction AI inferencing, for example enhancing fraud detection during financial transactions, and provides a fourfold increase in compute capacity per chip over the previous generation. The new I/O Acceleration Unit DPU is integrated into the Telum II chip. It is designed to improve data handling with a 50% increased I/O density. This advancement enhances the overall efficiency and scalability of IBM Z, making it well suited to handle the large-scale AI workloads and data-intensive applications of today's businesses. Spyre Accelerator: A purpose-built enterprise-grade accelerator offering scalable capabilities for complex AI models and generative AI use cases is being showcased. It features up to 1TB of memory, built to work in tandem across the eight cards of a regular IO drawer, to support AI model workloads across the mainframe while designed to consume no more than 75W per card. Each chip will have 32 compute cores supporting int4, int8, fp8, and fp16 datatypes for both low-latency and high-throughput AI applications

IBM Intros Telum II Processor & Spyre AI Accelerator: 8 Cores Clocked at 5.5 GHz With 360 MB Cache

IBM has unveiled its next-gen Telum II Processor & Spyre AI Accelerators for the latest IBM Z mainframe systems powering AI. IBM Offers Two Brand New Chips For Its Next-Gen Z Mainframe AI Systems: Telum II Processor With 8 Cores at 5.5 GHz & Spyre Accelerator With 128 GB Memory Today, IBM is revealing the first architectural details of its Telum II processor and Spyre Accelerator which are meant to advance AI workloads on the next-gen IBM Z mainframes that are designed for AI workloads. These new AI mainframes will accelerate traditional AI workloads along with LLMs using a brand new ensemble method of AI. Telum II processor: Featuring eight high-performance cores running at 5.5GHz, with 36MB L2 cache per core and a 40% increase in on-chip cache capacity for a total of 360MB. The virtual level-4 cache of 2.88GB per processor drawer provides a 40% increase over the previous generation. The integrated AI accelerator allows for low-latency, high-throughput in-transaction AI inferencing, for example enhancing fraud detection during financial transactions, and provides a fourfold increase in compute capacity per chip over the previous generation. The new I/O Acceleration Unit DPU is integrated into the Telum II chip. It is designed to improve data handling with a 50% increased I/O density. This advancement enhances the overall efficiency and scalability of IBM Z, making it well suited to handle the large-scale AI workloads and data-intensive applications of today's businesses. Spyre Accelerator: A purpose-built enterprise-grade accelerator offering scalable capabilities for complex AI models and generative AI use cases is being showcased. It features up to 1TB of memory, built to work in tandem across the eight cards of a regular IO drawer, to support AI model workloads across the mainframe while designed to consume no more than 75W per card. Each chip will have 32 compute cores supporting int4, int8, fp8, and fp16 datatypes for both low-latency and high-throughput AI applications. via IBM Starting with the details, we first have the IBM Telum II processor which features an 8-core design with increased frequencies of up to 5.5 GHz, increased caches with 36 MB L2 dedicated per core, & a 40% increase in on-chip cache, leading to a total pool count of 360 MB. The chip also features a virtual L4 cache of 2.88 GB per processor drawer, which also marks a 40% increase over the first-generation Telum chips. Each Telum II processor comes with an integrated AI accelerator which offers low-latency and high throughput AI inferencing performance. Other new additions include a I/O Acceleration Unit DPU which has been integrated into the Telum II chip and can improve data handling with a 50% uplift in I/O density. The second AI chip that IBM is introducing today for its IBM Z mainframe is the Spyre AI accelerator which is an enterprise-grade solution with 128 GB of memory capacity and 1 TB of memory across 8 cards that can be plugged into the IBM Z mainframe running the Telum II processor. Each Spyre AI accelerator features 32 compute cores which support INT4, INT8, FP8 & FP16 data types & come in 75W TDP cards. Each card is designed for low-latency and high-throughput AI applications. IBM expects its Z mainframe AI systems with Telum II processors to be available to client in 2025 while the Spyre AI accelerator is currently in tech preview and is also expected to be made available by 2025.

IBM unveils new Telum II AI processor set to power its next-generation mainframe systems - SiliconANGLE

IBM unveils new Telum II AI processor set to power its next-generation mainframe systems IBM Corp. today revealed the specifications of its upcoming Telum II Processor, which is set to power the next generation of its iconic mainframe systems and boost their relevance in the artificial intelligence processing industry. Details of the new chip were unveiled at the Hot Chips 2024 event taking place at Stanford University this week. The company said its enhanced processing capabilities will help to accelerate both traditional AI models and emerging large language models using a new technique known as "ensemble AI." The IBM Telum II Processor (pictured) notably features a completely new data processing unit, which is used to offload certain computing tasks and improve the overall computing efficiency of the chip. According to the company, the new DPU is designed to accelerate complex input/output protocols for networking and storage on the company's mainframe systems. In addition to the new chip, the company provided details of its all-new IBM Spyre Accelerator that's meant to be used in tandem with the Telum chips, providing additional oomph for AI workloads. IBM is promising quite a boost in overall compute performance when its next-generation mainframe launches later in the year. The new Telum chip, which is built on Samsung Foundry's most advanced 5-nanometer process, will sit at the heart of the new IBM Z mainframe, providing increased frequency and memory capacity, which enables it to deliver a 40% improvement in cache and integrated AI accelerator core performance. Digging deeper, IBM said the new chip, the successor to the original Telum processor that debuted in 2021, features eight high-performance cores running at 5.5 gigahertz, with 36 megabytes of memory per core. That amounts to an increase of 40% in on-chip cache capacity, for a total of 360 megabytes. In addition, the Telum II chip comes with an enhanced integrated AI accelerator for low-latency and high-throughput in-transaction AI inference operations, which makes it more suitable for application such as real-time fraud detection in financial transactions Meanwhile, the integrated I/O Acceleration Unit DPU should result in a significant improvement in the chip's data handling capabilities, with IBM promising a 50% increase in overall I/O density. As for the Spyre Accelerator (pictured, right), it's a purpose-built and enterprise-grade accelerator that's specifically designed for customers that want to use their mainframe systems for AI processing. It's designed to ramp up the performance of the most complex AI models, IBM said, including generative AI applications. To do this, it packs 1 terabyte of memory spread across eight cards of a regular I/O drawer. It has 32 compute cores that support int4, int8, fp8 and fp16 data types, enabling it to reduce latency and improve throughput for any kind of AI application. IBM explained that Telum II and Spyre have been designed to work in tandem, providing a scalable architecture for ensemble methods of AI modeling. Ensemble methods involve combining multiple machine learning and deep learning AI models with encoder LLMs. By drawing on the strengths of each model architecture, ensemble models can deliver more accurate results compared to using a single model type alone. Tina Tarquinio, IBM's vice president of product management for IBM Z and LinuxONE, said the new chips enable the company to remain "ahead of the curve" as it strives to cater to the escalating demands of AI. "The Telum II processor and Spyre accelerator are built to deliver high-performance, secured and more power efficient enterprise computing solutions," she promised. The company said Telum II is suitable for a range of specialized AI applications that are traditionally powered by its Z mainframe systems. For instance, it said ensemble methods of AI are particularly well suited for enhancing insurance fraud detection. The chips can also support money laundering detection systems, powering advanced algorithms that can spot suspicious financial activities in real-time, reducing the risk of financial crime. In addition, Telum II is said to be an ideal foundation for generative AI assistants, supporting knowledge transfer and code explanation, among other necessary tasks. The company said the Telum II chips will make their debut in the next version of the IBM Z mainframe and IBM LinuxONE systems, which are set to be launched later this year.

New IBM Chips Aim To Advance AI Adoption

'Bringing that kind of AI capability to that kind of enterprise environment - it's really only something IBM can do,' IBM VP Tina Tarquinio tells CRN. IBM unveiled its Telum II processor and Spyre accelerator chip during the annual Hot Chips conference, promising partners and solution providers new tools for bringing artificial intelligence use cases to life when the chips become available for Z and LinuxOne in 2025. The Armonk, N.Y.-based tech giant positions the chips as helping users adopt traditional AI models, emerging large language models (LLMs) and use the ensemble AI method of combining multiple machine learning (ML) and deep learning AI models with encoder LLMs. The conference runs through Tuesday in Stanford, Calif. "Bringing that kind of AI capability to that kind of enterprise environment - it's really only something IBM can do," Tina Tarquinio, IBM's vice president of product management for Z and LinuxOne, told CRN in an interview. [RELATED: Red Hat Partner Program Updates Include Incentives, New Digital Experience] Phil Walker, CEO of Manhattan Beach, Calif.-based solution provider Network Solutions Provider, told CRN in an interview that he is looking to grow his IBM practice and take advantage of the vendor's AI portfolio. "WatsonX will be key in business advice and recommendations," Walker said. He believes IBM will be a "key enabler for midmarket business growth." Tarquinio told CRN that IBM continues to see strong mainframe demand for its capabilities in sustainability, cyber resilience and of course AI. She said that mainframes have helped customers meet various data sovereignty regulations such as the European Union's Digital Operational Resilience Act (DORA) and General Data Protection Regulation (GDPR). "Our existing clients are leveraging the platform more and getting more capacity, which is great to see," she said. She added that "the more our partners can understand what we're bringing and start to talk about use cases with their end clients," the better. The Telum II processor has eight high-performance cores running at 5.5 gigahertz (GHz) with 36 megabytes (MB) secondary cache per core. On-chip cache capacity total 360MB, according to IBM. Samsung Foundry manufactures the processor and the Spyre accelerator, according to IBM. Telum II's virtual level-four cache of 2.88 gigabytes (GB) per processor drawer promises a 40 percent increase compared to the previous processor generation. The Telum II also promises a fourfold increase in compute capacity per chip compared to the prior generation. The Telum II chip is integrated with an input/output (I/O) acceleration data processing unit meant to improve data handling with 50 percent more I/O density. The DPU aims to simplify system operations and improve key component performance, according to IBM. Christian Jacobi, IBM fellow and chief technology officer of systems development, told CRN in an interview that fraud detection, insurance claims processing and code modernization and optimization are some common use cases for the new chip technology and LLMs. IBM clients can have millions of lines of code to contend with, Jacobi said. And IBM is at work on more specific AI models - along with general-purpose ones - that should appeal to IBM customers that rely on COBOL, for example. "Everybody sees the impact of AI," he said. "Everybody knows, 'AI is disrupting my industry, and if I'm not playing in that space, I'm going to get disrupted.'" The Spyre accelerator - now in tech preview - has up to a terabyte (TB) of memory and works across the eight cards of a regular I/O drawer. IBM bills the Spyre as an add-on complement to the Telum II for scaling architecture to support the ensemble AI method. Jacobi said that the accelerator can help improve accuracy. When a Telum II model "has a certain level of uncertainty - like I'm only 80 percent sure that this is not fraudulent, for example - then you kick the transaction to the Spyre accelerator, where you could have a larger model running that creates additional accuracy." "You're getting the benefits of the speed and energy efficiency of the small model. And then ... you use the bigger model to validate that," he said. Spyre attaches with a 75-watt Peripheral Component Interconnect Express (PCIe) adapter. The accelerator should support AI model workloads across the mainframe while consuming 75 watts or less of energy per card. Each chip has 32 compute cores that support int4, int8, fp8, and fp16 datatypes, according to IBM.