Nvidia Considers Socketed Design for Next-Gen Blackwell Ultra B300 AI GPUs

Nvidia eyes socketed design for next-gen GB300 chips

In brief: Nvidia is reportedly contemplating a move to a socketed design for its next-generation GB300 chips. AMD implemented a similar change, but if Team Green throws its considerable weight behind this configuration, it could set a new standard for high-performance AI accelerators. Nvidia is contemplating a significant change in its design approach for the upcoming GB300 AI chips. The company is exploring the possibility of adopting a socketed design for these next-generation products, a departure from its current practice of mounting GPUs directly onto motherboards, according to reports from MoneyDJ and the Economic Daily News that were spotted by Trendforce. The GB300, an upgrade to the recently launched GB200 Blackwell AI hardware, is expected to debut in 2025. If implemented, this design change would mark a first for Nvidia in its GPU products. The proposed socketed design would involve a motherboard with one CPU and four GPU sockets, doubling the number of GPUs compared to the current GB200 model. This would allow individual replacement or upgrade capability for each GPU. There are both benefits and drawbacks to this potential shift. The advantages include improved motherboard yields, a simplified supply chain for Nvidia and partners, easier post-sale support and maintenance, and the potential for future upgrades without replacing the entire board. The challenges include possible reduction in overall platform performance and increased power and cooling demands. The move to a socketed design would also benefit Nvidia's manufacturing partners, such as Foxconn. AI server manufacturers might not need to be equipped with SMT production lines, and there could be a potential boost for Taiwanese supply chain companies like LOTES. While this would be a first for Nvidia, rival AMD introduced a socketed design in 2023 with its Instinct MI300A AI Accelerator, which uses an SH5 socket similar to its SP5 socket for Epyc server CPUs. But with Nvidia now contemplating this change, it raises questions about the future of GPU design. It's unlikely to affect consumer GeForce cards due to different memory configurations. Still, it could influence other AI chip manufacturers like Intel and may lead to new cooling solutions for high-performance GPUs. The rumored change has sparked discussions in the tech industry. Analysts suggest it could simplify after-sales service and server board maintenance, while some speculate it might optimize computing board manufacturing yields. Others caution about potential performance trade-offs. Whether it features a socketed design or not, the B300 series is expected to become Nvidia's mainstream product in the second half of 2025, with demand expected to be significant due to its adoption of FP4, which is well-suited for inference scenarios. Meanwhile, Nvidia's current Blackwell GPUs are experiencing a 12-month backlog fueled by major tech giants like AWS, CoreWeave, Google, Meta, Microsoft, and Oracle. These companies have purchased every Blackwell GPU Nvidia and its manufacturing partner TSMC can produce for the next four quarters.

NVIDIA's next-gen Blackwell Ultra 'B300' AI GPU for GB300 AI servers: socketed design rumored

NVIDIA is rumored to move towards a socketed design for its next-gen GB300 AI servers, based on the upcoming Blackwell Ultra AI chips coming in 2025. In a new report by TrendForce, we're learning that in the second half of 2025 to expect the B300 series to "become the mainstream product for NVIDIA. The main attraction of the B300 series is said to be its adaption of FP4, which is well-suited in inference scenarios". This change in design is also expected to boost the yield rates of the B300 AI GPUs, with TrendForce noting that it "might probably reduce performance". The Economic Daily News says that using the socketed design will help simplify after-sales service and server board maintenance, as well as optimize the yield of computing board manufacturing. The shift into a socketed design for GB300 AI servers would allow more flexibility in production, as AI server manufacturers might not need to be equipped with an SMT production line. If these rumors are true, it would be the first time that NVIDIA has used a socket design for its GPU products, while competitor AMD used a socket design in 2023 with its Instinct MI300A AI Accelerator.

NVIDIA Blackwell Ultra "B300" AI GPUs For GB300 Servers Might Utilize A Socketed Design

NVIDIA's Blackwell Ultra B300" GPUs may introduce a docketed design on GB300 servers, which will make maintenance and upgrades easier. The current data center high-performance chips from NVIDIA bring the OAM design, which is an on-board solution. With this design, the GPU chips are soldered permanently to the server motherboard such as the GB200, where the users will find both Grace CPUs and Blackwell GPUs on a single board. However, this could be the last series featuring the onboard design as several reports suggest that NVIDIA could move to a different design with the Blackwell B300 "Ultra" GPUs for GB300 servers. As per MoneyDJ and Economic Daily News (via Trendforce), the B300 GPUs could feature a socket-based design, which will allow the users to install or uninstall the GPUs from the motherboards. The socketed approach on NVIDIA's Blackwell Ultra "B300" AI GPUs approach is said to simplify the manufacturing process for NVIDIA and can benefit several companies, especially the Taiwan-based Foxconn and LOTES, which produce interconnect components and sockets. The current Blackwell GPUs are soldered directly to the motherboard and with the transition to a socket-type design, the B300 GPUs could be removed from the motherboard just like CPUs. With this transition, there will be several benefits including improved yield rate and flexible production as the GPU won't have to be soldered into the socket and NVIDIA wouldn't need to rely on Surface Mount Technology. Moreover, the process will simplify the maintenance and after-sales services as the whole motherboard won't have to be replaced in cases of GPU-related problems. As a result, the upgrades could reduce the overall time for downtime when the GPUs are changed and it will help companies to offer more reliable servers to their customers. However, it is expected that the new socket design will introduce some performance reduction as this will introduce some higher latency. Nonetheless, if maintenance, upgrades, and better yields are improved, the trade-off will be worth the design transition. Another important change with the B300 is the adoption of FP4(Floating Point 4), which benefits inference. Inference is how the trained models make predictions on data and serves as a crucial aspect of AI computation. The B200 is already exceptional in AI workloads and has already been deployed by various companies. Meanwhile, the B300 "Blackwell Ultra" is expected to enhance its performance significantly but surprisingly, it won't be the first to feature the socket-based design as AMD already introduced this design with its MI300A chips introduced in 2023.

Nvidia reportedly mulls socketed design for Blackwell B300 AI GPUs -- next-gen Blackwell GPUs may be removable by the user

GPUs that could be removed from the motherboard, similarly to CPUs. Nvidia is considering adopting a socket design for at least some of its upcoming Blackwell B300 GPUs for AI and HPC applications, according to a report from TrendForce that cites Economic Daily News and MoneyDJ. The company is said to adopt the new socketed design for something codenamed GB300, and for now, the information looks unconvincing, to put it mildly. Yet, given the fact that there is supply chain chatter, it is at least worth considering. MoneyDJ reports that considering the failure rates of AI GPUs under high loads, the replacement costs of motherboards, and cooling challenges, Nvidia and other AI GPU designers might consider using socket designs for their next generation of GPUs instead of soldering GPUs to motherboards. EDN cites Chen Shuowen, an analyst with CLSA, as saying that based on supply chain checks, Nvidia has been designing GPU sockets for its products, possibly starting with the GB200 Ultra. Chen reportedly mentioned a 4-way Nvidia GPU design with one Nvidia CPU. Neither of the reports mentions anything called GB300, so TrendForce has added this part, possibly based on some additional chatter. Several things about the reports should be noted. Socketed designs would instead add to power and cooling challenges rather than help solve them, so the first report is inaccurate. The most power-hungry GPUs usually use BGA packaging. A 4-way Blackwell GPU with one CPU motherboard does not look extraordinary, considering that with DGX servers, we see an 8-way GPU baseboard and a 2-way CPU motherboard, yet such a design looks incredible. Nvidia's data center nomenclature divides the company's GPU (A100, H100, B100/B200) and Grace CPU + GPU platforms (GH100, GB200). For now, GB200 platforms use BGA packaging for both CPU and GPU; we are not sure something has to change with the B200 Ultra refresh, especially with the possible GB200 Ultra refresh sometime in the second half of the year. We all love standard CPU sockets for their easy repairs and upgradeability. But in servers, they take up more space and have more power and thermal constraints than BGA packages or SXM/OAM modules. While the modules provide reparability, the process might vary depending on the specific motherboard design, and removing an OAM/SXM module requires careful handling, so they are not as good as sockets. There is another point to make. Add-in cards, SXM, and OAM modules are hard and expensive to make, and for now, most Nvidia SXM modules are made by Foxconn. Migrating from a card or a module to a socket cuts costs but limits performance. Before moving on to the alleged Blackwell-based data center product (GB300, GB200 Ultra, whatever) featuring a socketed GPU, let us recall which Blackwell-based data center GPUs Nvidia has already introduced. By now, Nvidia has formally introduced its B200 GPU (1,000W+) that will be used on GB200 boards (codenamed Bianca with one Grace CPU and two Blackwell GPUs as well as Ariel with one Ariel CPU and one Blackwell GPU) and will come in a BGA form-factor. In addition, Nvidia also has Umbriel GPU boards supporting eight B200 (1000W) and B100 (700W) SXM module form factors. In addition, there are codenamed Miranda (adds performance (think higher TDP), PCIe 6.0, and 800G networking) and codenamed Oberon GB200 platforms, according to SemiAnalysis. While there are Nvidia H100 and even H200 add-in cards (based on the Hopper architecture) with lowered performance to fit into typical power and thermal budgets provided by classic servers, Nvidia has never announced any add-in cards featuring Blackwell-based GPUs. Yet, based on unofficial information, we know that Nvidia is prepping its codenamed B200A product based on the monolithic B102 processor with four HBM3E memory stacks connected using TSMC's CoWoS-S packaging technology. This is in contrast to dual-die B100/B200 designs that are packaged together using TSMC's CoWoS-L and then connected to eight stacks of HBM3E memory. Given that with the alleged B200A, we are dealing with a single-die product not designed to be a performance champion, this one could adopt multiple form factors. This includes an SXM modular design (especially in its China-specific B20 form) and an add-in-card form factor. Could it be a socket? Perhaps. We are going to see about that. Intel has done its socketed Xeon CPU Max 9480 'Sapphire Rapids' with HBM onboard, and it was not a success beyond selected supercomputing auditory. Does Nvidia want to build something similar? We will see about that.