Broadcom CEO Predicts Massive Growth in AI Infrastructure

Broadcom CEO Hock Tan predicts million-accelerator AI rigs

Hock Tan reckons the silicon sales cycle is about to swing up, sharply, too Broadcom CEO Hock Tan has predicted his hyperscale semiconductor customers will continue building AI clusters for another three to five years, with each generation of machines to double in size. Speaking to Jim Cramer of CNBC's Mad Money on Tuesday, Tan was asked to explain the recent performance of Broadcom stock. The share price dipped markedly after its Q3 results were felt by some to indicate its chips biz was off the boil as ardor for AI eased. It rebounded a week later, after other market signals indicated demand for custom AI silicon could be strong. Tan liked that argument, telling Cramer he's aware of hyperscalers' roadmaps that suggest a three-to-five year plan "to build out these large clusters" that allow development of new large language models. The Broadcom boss said those builds could be annual, and each will require two or three times the compute power of their predecessors. That will create a larger "compute opportunity" that Tan predicted will be met by "XPUs" - AI acceleration silicon for the network and other components - that the chip shop has predicted will grow faster than GPUs in coming years. XPUs are also a product Broadcom will happily make for hyperscalers - in bulk. Tan noted that the AI clusters hyperscalers are building today use 100,000 accelerators, but future rigs will require a million. Broadcom wants to design them just for hyperscale customers - a segment Tan feels is his best potential target. And fair enough: Nvidia and AMD look set to fight it out for the GPU market and associated software stacks. And Meta has already signalled a desire to build custom AI silicon to run alongside 600,000 Nvidia GPUs. Nvidia has promised its supply chain can deliver the GPUs the world wants, but has form of sputtering deliveries - as is reasonable during these boom times. If Broadcom can help hyperscalers to design rigs that don't depend on known chokepoints, it could thrive. Tan also observed that the semiconductor market bottomed out this year, as part of the industry's normal cycles - But COVID-19 created an unusual upcycle that turned downwards in late 2023 and early 2024. He predicted that 2025 and 2026 will be an upcycle for non-AI silicon. Asked if that turnabout means sales of Wi-Fi and storage connectivity silicon will improve - after respectively achieving stasis and a slump last quarter - Tan said that will "absolutely" be the case. "We are already seeing strong sequential growth from the bottom of Q2 this year," Tan said, thanks to demand from enterprises. Tan was also asked about VMware, and opined that his reforms at the virtualization giant are going well. "What we've done is improved the products and made it much more usable from just being compute virtualization to basically creating a virtualization of entire datacenter, on-prem, and creating the cloud experience, on-prem." For what it's worth, VMware has offered compute, storage, and network virtualization for a decade. Tan was probably referring to Cloud Foundation 5.2, which for the first time allows virtual compute, storage, and networks to be brought under a single management umbrella. That represents around 80 percent of the work Broadcom says is needed to realize its vision of a fully virtualized datacenter - which it claims is coming, without committing to a timeline. ®

Chip market high isn't over says Broadcom CEO, with million-accelerator clusters on the horizon

Broadcom CEO Hock Tan has predicted that demand for AI clusters will continue for another three to five years as companies continue to battle with the resource- and power-intensive compute needs of AI. Speaking with Jim Cramer in an interview on CNBC's Mad Money, Tan outlined how customers plan to build increasingly larger AI clusters, doubling in size with each new generation of machines. The prediction suggests a steady rise in demand for custom AI silicon and other advanced semiconductors, which Tan sees as beneficial for Broadcom. Tan explained that ongoing developments and advancements of large language models will mean that large-scale builds will be required annually in order to keep up with pace. As such, hyperscale AI clusters, which currently use 100,000 accelerators, may one day grow to include one million accelerators, believes Tan. Looking to get a slice of the action, the company aims to design and deliver essential components, including custom-built XPUs, to meet the demand. XPUs are anticipated to grow faster than traditional GPUs in the future, and as has been proven time and time again, early entry to the market counts for a lot. Nvidia, which was one of the first companies to invest heavily in AI chips for data centers, briefly spent some time as the world's most valuable company. Currently occupying third place, it has a market cap of $2.78 trillion. Broadcom's recent AI revenue expectation was raised to $12 billion for fiscal year 2024, reflecting ongoing hot demand from the tech sector. Moreover, Tan addressed the pandemic-induced chip shortage and the state of non-AI semiconductor production and demand globally, indicating that recovery for the segment could occur in 2025 or 2026. The CEO also noted strong demand for Wi-Fi and storage connectivity products.