Broadcom's Tomahawk 6: A Game-Changer for AI Networking

Broadcom takes a Tomahawk to Nvidia's AI networking empire

Broadcom began shipping its answer to Nvidia's upcoming Quantum-X and Spectrum-X switches on Tuesday: the Tomahawk 6. The chip doubles the bandwidth of its predecessor and comes in both standard and co-packaged optics flavors. The launch comes as Broadcom looks to play a bigger role in AI networks -- a space where Ethernet has historically trailed Nvidia's InfiniBand interconnect tech. That's starting to change. Over the past year, we've seen a number of high-profile AI clusters pop up using Ethernet for their scale-out fabrics -- the network used to connect GPU nodes to one another so that work can be distributed across them. xAI's 200,000 GPU Colossus supercomputer in Memphis, Tennessee, is just one example. In fact, as of this year, Dell'Oro Group analyst Sameh Boujelbene estimates that Ethernet has surpassed InfiniBand's share, she told The Register. Broadcom not only faces competition from Nvidia's Quantum-X InfiniBand switches, but also its Spectrum-X family of Ethernet kit, which our sibling site The Next Platform projects could rival Cisco's datacenter networking revenues within a year or so. To meet this challenge, Broadcom is accelerating the rollout of its Tomahawk 6 (TH6) switch ASIC. Normally, it takes about two years from the time that Broadcom starts "shipping" chips to integrators to when the first appliances actually hit the market. This time around, Pete Del Vecchio, product line manager for Broadcom's Tomahawk line, tells El Reg that we can expect to see the first TH6-based switches as early as this summer with volume production ramping in the first half of next year. In terms of speeds and feeds, TH6 looks quite similar to Nvidia's new photonic Spectrum ASICs due out next year. Both chips offer 102.4Tbps of aggregate bandwidth and make use of 200Gbps PAM4 serializer/deserializers (SerDes). This means a single switch package will now be able to support up to 512 ports at 200Gbps, or, if you're prioritizing bandwidth over density, up to 64 ports at 1.6Tbps. We say package because, unlike Tomahawk 5, which was monolithic, Broadcom's latest flagship has moved to a chiplet architecture, which breaks out the SerDes from the package processing and traffic management. And that means you'll be able to get this thing with 1,024 SerDes running at 100Gbps if you really want to. According to Del Vecchio, Tomahawk 6's greater density means Broadcom can now support scale-out networks at 200Gbps with up to 128,000 GPUs using a two-layer topology, where three tiers would have been needed previously. And TH6 isn't limited to plain old RDMA over converged Ethernet (RoCE) either - we're told the chip will also support the emerging Ultra Ethernet standard. This has the benefit of minimizing hops and therefore latency, simplifying congestion control, and halving the power required by the network, he added. This is mainly a benefit of the chip's increased bandwidth and not unique to Broadcom. Other 102.4Tbps ASICs, like those from Nvidia, should also enjoy similar improvements in efficiency. Ethernet fabrics are usually associated with the scale-out networks, but Broadcom is also positioning Tomahawk 6 as an alternative to Nvidia's NVLink for scale-up networks. These networks usually offer higher aggregate bandwidth and are used to pool the compute and memory of multiple physical accelerators into one logical one. For example, Nvidia's HGX B200 platform used NVLink to stitch eight GPUs together, while its NVL72 racks, as their name suggests, utilize multiple NVLink switches to connect 72 GPUs. Broadcom argues the same can be achieved using Ethernet on TH6, which can support scale-up fabrics up to 512 GPUs at 200Gbps. If you need more than that, you'll need to add more switches or start aggregating ports. This is actually what Intel did with its Gaudi accelerators. Each Gaudi3 OAM module featured 24 200GbE ports. Three ports were dedicated to the scale-out network, while the remainder enabled each of the eight accelerators to communicate with each other at roughly 1TBps of bidirectional bandwidth. That said, Ethernet-based scale-up networks may not catch on, as the emerging Ultra Accelerator Link standard is gaining momentum among chipmakers like Intel and AMD. In addition to conventional switches, Tomahawk 6 will also be available with co-packaged optics (CPO) on board. The technology essentially takes the lasers, digital signal processors (DSPs), and retimers traditionally found in pluggable transceivers -- for which you'd have one per port -- and packages them alongside the switch ASIC. Broadcom has been investing in CPO for years now, having first introduced the tech with its Humboldt family of switches back in 2021. However, it only began shipping CPO switches with its Tomahawk 5 Bailly platform last year. Broadcom's latest CPO platform, teased back at Computex, enables up to 200Gbps of optical bandwidth per lane, which aligns perfectly with TH6. For large-scale AI clusters, CPO offers a number of advantages. Most notably, it eliminates the need for expensive and power-hungry pluggable optics, at least at the switch. So, it's no surprise Broadcom and Nvidia are so keen on the tech: The less power spent on networking, the more you have for GPUs. CPO also has the advantage of allowing for greater port densities. You can pack more fiber ports onto the front of a switch than you can QSFP cages. Along with lower power and higher density, eliminating pluggables means fewer points of failure and lower overall latencies, since there are fewer interconnects between the fiber and ASIC. With that said, CPO is still in its infancy, and there remain valid concerns over the increased complexity, issues with serviceability, and the overall blast radius of such a vertically integrated system. If a pluggable fails, it's relatively cheap to swap out. But, what happens if the photonic chiplet or one of the lasers fails? That could take down the entire switch fabric. This is one of the reasons most CPO switch designs we've seen to date still use user-replaceable lasers that can be swapped out in the event of a failure. However, Manish Mehta, VP of marketing for Broadcom's optical systems group, says early evidence suggests many of these concerns are unfounded. "The continuous wave laser source has been disaggregated, but even there it's not really failing," he told us. At GTC this spring, Nvidia revealed that its next-generation of Quantum-X InfiniBand and Spectrum-X Ethernet switches would ditch QSFP cages for CPO and gobs of MPO fiber ports. Nvidia's Quantum-X photonic switches are expected to hit the market later this year and will offer 144 ports of 800Gbps InfiniBand connectivity, which you might have noticed is more than Broadcom's TH6. However, it's important to remember that the Quantum 3450-LD is an appliance, while Tomahawk 6 is just an ASIC. Nvidia actually uses four Quantum-X CPO packages to achieve its 115Tbps of bandwidth. As The Next Platform, pointed out in its GTC coverage, there should probably be six ASICs in there for a full non-blocking configuration, but somehow Nvidia managed to make it work with four. A far better comparison to TH6 is Nvidia's Spectrum SN6810, which uses a single ASIC, and, as we mentioned earlier, boasts many of the same speeds and feeds, including support for up to 128 ports at 800Gbps. If you need more than that, Nvidia's SN6800 quadruples the bandwidth to 409.6Tbps, which works out to 512 MPO ports good for 800Gbps apiece. That's enough bandwidth for a single appliance to support a cluster of 64 DGX B200s. Broadcom's Del Vecchio was keen to point out that, while Nvidia's Spectrum switches rely on its own ConnectX superNICs or BlueField data processing units (DPU) to achieve its InfiniBand-like latencies, TH6's Cognitive Routing functionality means it can achieve similar results using any modern NIC. Nvidia says its Quantum-X photonic switches should hit the market in the second half of this year, while its Spectrum-X CPO kit will arrive in 2026. That's roughly in line with when Broadcom expects its partners to begin shipping its first CPO-enhanced TH6 Ethernet switches. But if you need a CPO switch today, it should be noted that several network vendors, including Micas Networks, already offer switches based on Broadcom's 51.2Tbps Tomahawk 5-Bailly platform. ®

Broadcom Ships New Gear Meant to Improve AI Chip Performance

Broadcom Inc. began shipping a new version of its data center switch chips that can boost the efficiency of AI accelerators, aiming to take a bigger role in the booming market for artificial intelligence computing. The company started delivering the Tomahawk 6 switch chips to customers over the weekend, and the product will be broadly available in July, said Ram Velaga, senior vice president and general manager of Broadcom's Core Switching Group. Switches, a central piece of networking equipment, allow computers to communicate with one another. A single new Tomahawk 6 can do the work of six of the previous versions, Broadcom said.

Broadcom ships latest networking chip to speed AI

SAN FRANCISCO, June 3 (Reuters) - Broadcom (AVGO.O), opens new tab has begun to ship its latest networking chip that aims to speed AI, the company said on Tuesday. The chip, called the Tomahawk 6, boasts double the performance compared with the prior version and other traffic control features that make the networking chip significantly more efficient, Ram Velaga, a Broadcom senior vice president, told Reuters in a Monday interview. The speed boost means that fewer networking switches are needed to perform the same task, Velaga said. Broadcom's networking chips have gained increased importance because of AI. When constructing the necessary data centers for AI applications, infrastructure builders must string together hundreds or thousands of chips. Building large-scale clusters of networked chips requires specialized networking gear and chips, of which the Tomahawk series of processors is one such component. With the Tomahawk 6, Broadcom's engineers have boosted its speed and capabilities to the point where it can be used to construct the larger data centers that are necessary for AI, which can be over 100,000 graphics processors (GPUs) strung together, Velaga said. "In a couple of years, you will start to see a million GPUs housed inside a physical building," he said. Broadcom's networking chips use the Ethernet networking protocol, which has been a networking standard for decades. Nvidia (NVDA.O), opens new tab produces hardware that uses a rival tech called InfiniBand and several products based on Ethernet. "All of these networks can be very simply done on Ethernet, you don't need esoteric technologies," Velaga said. The Tomahawk 6 is the first product in that line that will use several chips combined into a single package, a tech known as chiplets that is widely adopted by other chip designers such as Advanced Micro Devices (AMD.O), opens new tab. Adding chiplets roughly doubled the amount of silicon area used in the design, Velaga said. Broadcom is producing the Tomahawk switch on Taiwan Semiconductor Manufacturing Co's three nanometer process. Reporting by Max A. Cherney; Editing by Leslie Adler Our Standards: The Thomson Reuters Trust Principles., opens new tab Suggested Topics:Disrupted Max A. Cherney Thomson Reuters Max A. Cherney is a correspondent for Reuters based in San Francisco, where he reports on the semiconductor industry and artificial intelligence. He joined Reuters in 2023 and has previously worked for Barron's magazine and its sister publication, MarketWatch. Cherney graduated from Trent University with a degree in history.

Broadcom introduces Tomahawk 6 networking chip for large-scale AI clusters - SiliconANGLE

Broadcom introduces Tomahawk 6 networking chip for large-scale AI clusters Broadcom Inc. today debuted a new chip lineup, the Tomahawk 6 series, that's optimized to power Ethernet switches in data centers. The company says that customers can expect bandwidth of up to 102.4 terabits per second. That's nearly twice the performance of the second-fastest Ethernet switch on the market. According to Broadcom, the Tomahawk 6's speed makes it particularly well-suited for powering large artificial intelligence clusters. The process of training a large language model involves spreading it across multiple graphics cards. Each chip performs a different subset of the tasks involved in the training workflow. As a result, those tasks can be performed in parallel, which is significantly faster than completing them one after one. The graphics cards involved in an LLM training project must keep their work coordinated. They do so by regularly sending data to one another, a task that consumes a significant amount of bandwidth. Inference is also bandwidth-intensive because it often requires the graphics cards that run an LLM to retrieve data over the network from remote storage equipment. According to Broadcom, Tomahawk 6 optimizes network speeds using a set of AI features dubbed Cognitive Routing 2.0. The technology detects when a network link is congested and reroutes data to other connections, which helps avoid performance bottlenecks. It doubles as an observability tool that can collect data about technical issues. Data center operators often link together the servers in their AI clusters using fiber-optic cables. Such cables are significantly faster than the copper wires historically used for the task. For customers with optical networks, Broadcom offers a version of Tomahawk 6 that ships with co-packed optics. Before data from an AI server can be sent over a fiber-optic cable, it has to be turned into light. This task is usually performed by devices called pluggable transceivers that have to be attached to an AI cluster's switches. Co-packaged optics, or CPO, technology integrates the features of a transceiver directly into a switch's processor. That removes the need for standalone transceiver devices, which avoids the associated hardware costs and lowers power consumption. The Tomahawk 6 can also be used in copper-based networks. Standard copper cables have fairly limited range, which means that AI servers must be placed near one another to ensure reliability connectivity. That constraint, in turn, can make it challenging for engineers to design AI clusters. Tomahawk 6 ships with support for long-reach passive copper cables that can ease the design process. Broadcom says that the chip is capable of powering clusters with up to 512 processors when it's used in a scale-out configuration. This configuration includes limited amount of network equipment. In two-tier scale-out networks that include a larger number of switches, the Tomahawk 6 can link together more than 100,000 processors.

Broadcom: Tomahawk 6 Switch Is A 'Turning Point' For AI Data Centers

Designed to support more than 1 million accelerator chips in AI data centers, the Tomahawk 6 Ethernet switch series is being called a 'turning point in AI infrastructure design' by Broadcom, which says deployments with more than 100,000 accelerator chips are being planned. Broadcom is calling its newly launched 102.4-Tbps Tomahawk 6 Ethernet switch series a "breakthrough" and a "turning point in AI infrastructure design" that is "poised to make a rapid and dramatic impact on the deployment" of AI data centers. The IT infrastructure giant announced on Tuesday that it has begun shipping the Tomahawk 6 switch series, representing what it called the "world's first 102.4 Tbps of switching capacity in a single chip, double the bandwidth of any Ethernet switch" available now. [Related: Intel Spin-Off: Our InfiniBand Alternative For AI Data Centers Has A 'Devastatingly Good' Edge] "Tomahawk 6 is not just an upgrade -- it's a breakthrough," said Ram Velaga, senior vice president and general manager of Broadcom's Core Switching Group, in a statement. "It marks a turning point in AI infrastructure design, combining the highest bandwidth, power efficiency, and adaptive routing features for scale-up and scale-out networks into one platform." The Tomahawk 6 announcement was lauded by major IT channel vendors such as Juniper Networks, Arista Networks and AMD. "We are excited to leverage its low-power, high- radix, 1.6-Tbps port speeds, and advanced packet processing capabilities for AI-aware routing, based on open Ethernet standards," said Hardev Singh, general manager of cloud titans and AI at Arista, in a statement. Velega called demand from customers and partners "unprecedented," with Broadcom noting there are "multiple deployments" planned with more than 100,000 accelerator chips "using Tomahawk 6 for both the scale-out and scale-up interconnect." "Tomahawk 6 is poised to make a rapid and dramatic impact on the deployment of large AI clusters," Velega said. Broadcom said Tomahawk 6, which is compliant with specifications by the widely supported Ultra Ethernet Consortium, features the "industry's most comprehensive set of AI routing features and interconnect options," which will allow the switch series to scale AI data centers to nodes with more than 1 million accelerator chips. The switch chip architecture also enables "unified networks for AI training and inference at unprecedented scale," thanks to capabilities like advanced telemetry, dynamic congestion control and rapid failure detection as part of Broadcom's Cognitive Routing 2.0 feature. Thanks to its use of "industry-leading" 200-Gbps SerDes technology, Tomahawk 6 "provides the longest reach for passive copper interconnect, enabling high-efficiency, low-latency system design with the highest reliability and lowest total cost of ownership," according to Broadcom. Tomahawk 6 also supports 1,024 100-Gbps SerDes ports on a single chip, "enabling AI clusters with extended copper reach and efficient use of XPUs with native 100G interfaces," the company said. Broadcom said the new switch chip family will provide customers with an option for co-packaged optics to enable connectivity at the "lowest power and latency while reducing link flaps and improving long-term reliability," which it called essential for hyperscalers.

Broadcom Tomahawk 6 Flexes AI Muscle, Hyperscalers Buy In - Broadcom (NASDAQ:AVGO)

Broadcom Inc AVG announced on Tuesday that it is now shipping the Tomahawk 6 switch series, which delivers 102.4 Terabits/sec of switching capacity in a single chip. Tomahawk 6 is built to power the next generation of scale-up and scale-out AI networks, delivering unmatched flexibility with support for 100G/200G SerDes and co-packaged optics (CPO). It offers a comprehensive set of AI routing features and interconnect options designed to meet the demands of AI clusters with more than one million XPUs. Also Read: Broadcom Unveils Next-Gen Optical Chip Tech To Power AI Data Centers Kunjan Sobhani, lead semiconductor analyst at Bloomberg Intelligence, said that by breaking the 100Tbps barrier and unifying scale-up and scale-out Ethernet, Broadcom's Tomahawk 6 gives hyperscalers an open, standards-based fabric and a clear, flexible path to the next wave of AI infrastructure. Ram Velaga, senior vice president and general manager of Broadcom's Core Switching Group, told Bloomberg that the new chip's cost is more than double that of the old one. Customers will pay about twice as much, he said. Velaga said it would be under $20,000 apiece. Broadcom stock surged over 90% in the last 12 months, backed by the AI frenzy. BofA Securities analyst Vivek Arya chose Broadcom as a favorite, citing demand for AI computing with an additional tailwind from the ongoing shift toward compute-hungry test-time computing and reasoning models. Price Action: AVGO stock is trading higher by 2.01% to $253.71 at the last check on Tuesday. Read Next: Intel Eyes Exit From Networking And Edge Business To Refocus On PC, Data Center Dominance: Report Image: Shutterstock AVGOBroadcom Inc$255.072.56%Stock Score Locked: Edge Members Only Benzinga Rankings give you vital metrics on any stock - anytime. Unlock RankingsEdge RankingsMomentum95.79Growth38.74Quality80.92Value8.65Price TrendShortMediumLongOverviewMarket News and Data brought to you by Benzinga APIs