Micron develops stacked GDDR memory as cost-effective HBM alternative for AI inference

Micron Explores Vertically Stacked GDDR Memory as HBM Alternative

Micron is reportedly developing a new memory architecture based on vertically stacked GDDR, targeting a space between traditional GDDR and high-bandwidth memory (HBM). According to industry reports, the company could introduce an early prototype by 2027, signaling a longer-term effort to diversify memory solutions for GPUs and AI accelerators. The concept behind stacked GDDR is relatively straightforward. By vertically stacking memory dies within a single package, Micron aims to increase both capacity and bandwidth without relying on the complex silicon interposer designs required by HBM. This could result in a more scalable and cost-effective solution, particularly for applications that do not require the full performance envelope of HBM. HBM currently dominates high-end AI training hardware due to its extremely high bandwidth and efficiency. However, it also comes with higher production costs and more demanding packaging requirements. GDDR, on the other hand, remains widely used in GPUs and certain AI inference accelerators thanks to its lower cost and simpler integration. Professional graphics cards and various AI-focused designs continue to rely on GDDR, demonstrating that it still has a place in the broader memory ecosystem. The move toward vertically stacked GDDR appears to be driven by growing demand in AI inference workloads. Unlike training, inference often benefits more from increased memory capacity and cost efficiency than from peak bandwidth. By stacking memory dies, Micron could significantly boost per-package density while maintaining a relatively accessible price point compared to HBM-based solutions. This development could also intensify competition among memory manufacturers, as the industry looks for new ways to balance performance, cost, and scalability. If successful, stacked GDDR may become a viable alternative for mid-range AI accelerators and future GPU designs, offering a flexible option between conventional GDDR and premium HBM. While still early in development, Micron's approach highlights a broader trend in memory innovation, where vendors are exploring hybrid solutions to meet the evolving requirements of AI and high-performance computing workloads.

Micron Is Looking to Stack Gaming GPU GDDR Modules Like HBM for the First Time Ever, and AI's Memory Hunger Is Entirely to Blame

Micron is now looking to gobble up general-purpose GDDR memory modules by creating an HBM-like solution with them, and this might further fuel shortages for gamers. The memory industry is working to meet the demands of modern-day AI workloads, and companies are exploring various methods to overcome this challenge. While traditional HBM technologies were adequate for training frontier models, as we move towards inference, memory has become the next major focus. A report by ETNews reveals that Micron is now looking to stack GDDR modules, likely to create a solution with significantly higher capacities than current solutions, which could prove to be an interesting venture. Initial GDDR stacking of around four layers is expected. Prototypes (samples) could be released as early as next year. - ETNews GDDR is a segment that hasn't been influenced by the AI frenzy as dramatically as LPDDR or DDR, since its use case had previously been confined to gaming GPUs. Given that Micron plans to vertically stack GDDR to offer the industry a solution, it may be that the company has sufficient capacity on board and, instead of easing GPU market shortages, finds it better to cater to enterprise demand. The report noted that the stacked GDDR solution won't be on par with HBM in terms of performance, but it would offer higher capacities, complementing modern-day inference workloads. GDDR stacking is a newly explored concept, and while the report doesn't provide technical details on how the solution could look. Micron's SOCAMM2 has already explored the prospect of stacking general-purpose DRAM, such as LPDDR5X, up to 16-Hi, and the firm has achieved up to 256 GB per module. However, stacking LPDDR5X is much easier than GDDR, given that the former is a power-efficient module with manageable thermals. With GDDR, the primary issues Micron could face are maintaining thermal and signal integrity if it sticks to wire bonding. There are multiple ways Micron could address the issues with GDDR stacking by trading off clock speeds, but the report highlights that the memory giant is pursuing innovation in the memory segment. We have seen that Micron experienced hiccups with HBM4 following a delay in NVIDIA's certification, and while the firm has provided modules for Vera Rubin, supply allocation has dropped, while competitors like Samsung have seen an increase. A GDDR-stacked solution could stand out in the memory market if it proves cost-effective relative to HBM.