Nvidia Embraces Optical Technology for Networking, But Not Yet for GPUs

Nvidia CEO says power-saving optical chip tech will need to wait for wider use

SAN JOSE, California, March 18 (Reuters) - A promising new chip technology that aims to cut energy usage is not yet reliable enough for use in Nvidia's flagship graphics processing units (GPUs), Nvidia's CEO Jensen Huang said Tuesday. Co-packaged optics, as the emerging technology is called, uses beams of laser light to send information on fiber optic cables between chips, making connections faster and with superior energy efficiency to those through traditional copper cables. During a keynote address to Nvidia's (NVDA.O), opens new tab annual developer conference at a packed hockey stadium in San Jose, California on Tuesday, Huang said his company would use the co-packaged optical technology in two new networking chips that sit in switches on top of its servers, saying the technology would make the chips three and a half times more energy efficient than their predecessors. The switch chips will come out later this year and into 2026 in a small but significant step toward advancing the technology. But Huang told a group of journalists after his speech that while Nvidia examined using it more widely in its flagship GPU chips it had no current plans to do so, because traditional copper connections were "orders of magnitude" more reliable than today's co-packaged optical connections. "That's not worth it," Huang said of using optical connections directly between GPUs. "We keep playing with that equation. Copper is far better." Huang said that he was focused on providing a reliable product roadmap that Nvidia's customers, such as OpenAI and Oracle (ORCL.N), opens new tab , could prepare for. "In a couple years, several hundred billion dollars of AI infrastructure is going to get laid down, and so you've got the budget approved. You got the power approved. You got the land built," Huang said. "What are you willing to scale up to several hundred billion dollars right now?" Silicon Valley entrepreneurs and investors have pinned their hopes on the optics technology, which they believe will be central to building ever-larger computers for AI systems, which Huang said on Tuesday would still be necessary even after advances by companies like DeepSeek because AI systems would need more computing power to think through their answers. Startups such as Ayar Labs, Lightmatter and Celestial AI have raised hundreds of millions of dollars in venture capital - some of it from Nvidia itself - to try and put co-packaged optical connections directly onto AI chips. Lightmatter and Celestial AI are both targeting public offerings. Copper connections are cheap and fast, but can only carry data a few meters at most. While that might seem trivial, it has had a huge impact on Nvidia's product lineup over the past half decade. Nvidia's current flagship product contains 72 of its chips in a single server, consuming 120 kilowatts of electricity and generating so much heat that it requires a liquid cooling system similar to that of a car engine. The flagship server unveiled on Tuesday for release in 2027 will pack hundreds of its Vera Rubin Ultra Chips into a single rack and will consume 600 kilowatts of power. Cramming more than double the number of chips into the same space over two years will require massive feats of engineering from Nvidia and its partners. Those feats are driven by the fact that AI computing work requires moving a lot of data back and forth between chips, and Nvidia is trying to keep as many chips as it can within the relatively short reach of copper connections. Mark Wade, the CEO of Ayar Labs, which has received venture backing from Nvidia, said the chip industry was still navigating how to manufacture co-packaged optics at lower costs and with higher reliability. While the transition may not come until 2028 or beyond, Wade said, the chip industry will have little choice but to ditch copper if it wants to keep building bigger and bigger servers. "Just look at the power consumption going up and up on racks with electrical connections," Wade told Reuters in an interview on the sidelines of Nvidia's conference. "Optics is the only technology that gets you off of that train." Reporting by Stephen Nellis in San Jose, California; Editing by Kate Mayberry Our Standards: The Thomson Reuters Trust Principles., opens new tab Suggested Topics:Technology

Nvidia embraces optical tech - but not for GPUs (yet)

The big picture: Nvidia's approach to co-packaged optics is a nuanced reflection of the technology's current limitations and future potential. By focusing on specialized applications with less demanding reliability requirements, Nvidia is positioning itself to lead the development of optical technologies for networking and data center applications while continuing to rely on traditional copper connections for its high-performance GPUs. Nvidia CEO Jensen Huang recently addressed the reliability of co-packaged optics, a technology that uses laser light to transmit data between chips, and its potential use in the company's flagship GPUs. During the company's annual developer conference, Huang noted that while co-packaged optics offer superior energy efficiency and speed compared to traditional copper connections, they are not yet reliable enough for widespread use in Nvidia's GPUs. This reliability gap is significant, with copper connections being "orders of magnitude" more reliable than current co-packaged optical connections, making them the preferred choice for Nvidia's GPUs, Huang told a group of journalists after his keynote speech. However, Nvidia is exploring using co-packaged optical technology in more specialized applications. Huang said that the company plans to integrate the technology into two new networking chips designed for switches on top of its servers. These chips, set to be released later this year and in 2026, will be three and a half times more energy-efficient than their predecessors. This marks a significant step forward in advancing the technology, albeit in a more niche application where the reliability requirements might be less stringent than those for high-performance GPUs. Nvidia has also made significant strides in silicon photonics, with the introduction of its Quantum-X and Spectrum-X networking switches last week. These switches integrate optical communications into electronic circuits at a massive scale, enabling AI factories to connect millions of GPUs across sites while reducing energy consumption and operational costs. Nvidia claims these optics innovations use 4x fewer lasers to deliver 3.5x more power efficiency, 63x greater signal integrity, 10x better network resiliency at scale, and 1.3x faster deployment than traditional methods. Huang highlighted the importance of the new networking switches, stating, "AI factories are a new class of data centers with extreme scale, and networking infrastructure must be reinvented to keep pace." By integrating silicon photonics into switches, Nvidia is pushing the boundaries of what is possible in hyperscale and enterprise networks, paving the way for "million-GPU AI factories," the CEO said. The industry's shift towards optical technology is driven by the need to address the increasing power consumption and heat generation associated with traditional copper connections. As AI computing demands grow, companies like Nvidia and startups such as Ayar Labs are exploring ways to make co-packaged optics more reliable and cost-effective. Mark Wade, CEO of Ayar Labs, told Reuters that while transitioning to widespread use of co-packaged optics may not occur until 2028 or beyond, optics is essential for building larger servers without being constrained by copper's limitations. "Just look at the power consumption going up and up on racks with electrical connections," Wade said. "Optics is the only technology that gets you off of that train."

Jensen Huang says no to lasers on GPUs for now

Nvidia is diving into the world of optical data transmission, but don't expect to see it in their GPUs just yet. CEO Jensen Huang confirmed that while the tech boasts superior energy efficiency and speed, it's not quite reliable enough for Nvidia's flagship products, as stated during the company's annual developer conference. Co-packaged optics, which transmit data between chips using laser light, still lag significantly behind traditional copper connections in terms of reliability. Huang described copper as "orders of magnitude" more dependable, making it the go-to choice for Nvidia's high-stakes GPU operations. Nvidia isn't sidelining optical tech entirely. The company plans to roll out two new networking chips, designed for server switches, later this year and in 2026. These chips will leverage co-packaged optics to deliver a hefty 3.5 times boost in energy efficiency compared to their predecessors. This move indicates a strategic, focused application, acknowledging that reliability needs might differ from those of their top-tier GPUs. Nvidia RTX 5090 price drops but good luck finding one Last week, Nvidia unveiled its Quantum-X and Spectrum-X networking switches. These products push silicon photonics, blending of optical communications with electronic circuits, providing large-scale, efficient connections between lots of GPUs, driving cost and energy savings. "AI factories are a new class of data centers with extreme scale, and networking infrastructure must be reinvented to keep pace," Huang stated. He emphasized that integrating silicon photonics into switches positions the company to pave the way for "million-GPU AI factories." The tech industry's gradual shift toward optical technology stems from the persistent headaches of power consumption and heat generation with copper. Mark Wade, CEO of Ayar Labs, mentioned to Reuters that fully transitioning to co-packaged optics might take until 2028 or later. He pointed that, "Optics is the only technology that gets you off of that train," referring to the escalating power demands of electrically-connected servers.

Nvidia CEO says power-saving optical chip tech will need to wait for wider use

During a keynote address to Nvidia's annual developer conference at a packed hockey stadium in San Jose, California on Tuesday, Huang said his company would use the co-packaged optical technology in two new networking chips that sit in switches on top of its servers, saying the technology would make the chips three and a half times more energy efficient than their predecessors.A promising new chip technology that aims to cut energy usage is not yet reliable enough for use in Nvidia's flagship graphics processing units (GPUs), Nvidia's CEO Jensen Huang said Tuesday. Co-packaged optics, as the emerging technology is called, uses beams of laser light to send information on fiber optic cables between chips, making connections faster and with superior energy efficiency to those through traditional copper cables. During a keynote address to Nvidia's annual developer conference at a packed hockey stadium in San Jose, California on Tuesday, Huang said his company would use the co-packaged optical technology in two new networking chips that sit in switches on top of its servers, saying the technology would make the chips three and a half times more energy efficient than their predecessors. The switch chips will come out later this year and into 2026 in a small but significant step toward advancing the technology. But Huang told a group of journalists after his speech that while Nvidia examined using it more widely in its flagship GPU chips it had no current plans to do so, because traditional copper connections were "orders of magnitude" more reliable than today's co-packaged optical connections. "That's not worth it," Huang said of using optical connections directly between GPUs. "We keep playing with that equation. Copper is far better." Huang said that he was focused on providing a reliable product roadmap that Nvidia's customers, such as OpenAI and Oracle, could prepare for. "In a couple years, several hundred billion dollars of AI infrastructure is going to get laid down, and so you've got the budget approved. You got the power approved. You got the land built," Huang said. "What are you willing to scale up to several hundred billion dollars right now?" Silicon Valley entrepreneurs and investors have pinned their hopes on the optics technology, which they believe will be central to building ever-larger computers for AI systems, which Huang said on Tuesday would still be necessary even after advances by companies like DeepSeek because AI systems would need more computing power to think through their answers. Startups such as Ayar Labs, Lightmatter and Celestial AI have raised hundreds of millions of dollars in venture capital - some of it from Nvidia itself - to try and put co-packaged optical connections directly onto AI chips. Lightmatter and Celestial AI are both targeting public offerings. Copper connections are cheap and fast, but can only carry data a few meters at most. While that might seem trivial, it has had a huge impact on Nvidia's product lineup over the past half decade. Nvidia's current flagship product contains 72 of its chips in a single server, consuming 120 kilowatts of electricity and generating so much heat that it requires a liquid cooling system similar to that of a car engine. The flagship server unveiled on Tuesday for release in 2027 will pack hundreds of its Vera Rubin Ultra Chips into a single rack and will consume 600 kilowatts of power. Cramming more than double the number of chips into the same space over two years will require massive feats of engineering from Nvidia and its partners. Those feats are driven by the fact that AI computing work requires moving a lot of data back and forth between chips, and Nvidia is trying to keep as many chips as it can within the relatively short reach of copper connections. Mark Wade, the CEO of Ayar Labs, which has received venture backing from Nvidia, said the chip industry was still navigating how to manufacture co-packaged optics at lower costs and with higher reliability. While the transition may not come until 2028 or beyond, Wade said, the chip industry will have little choice but to ditch copper if it wants to keep building bigger and bigger servers. "Just look at the power consumption going up and up on racks with electrical connections," Wade told Reuters in an interview on the sidelines of Nvidia's conference. "Optics is the only technology that gets you off of that train."