Nvidia unveils NVHBM custom memory promising 30% higher bandwidth and 15% lower power than HBM4e

Nvidia Introduces NVHBM for AI Accelerators

Nvidia has announced NVHBM, a custom high-bandwidth memory solution designed to address the escalating demands of AI infrastructure as trillion-parameter workloads and AI agents become mainstream1

1

2

2

. The technology represents an expansion of the NVLink Fusion program, which provides partners with building blocks to connect custom chips within Nvidia's rack-scale platform architecture2

2

. NVHBM delivers 30% higher bandwidth per stack compared to standard HBM4e, directly translating to higher tokens-per-second rates for AI inference workloads that are memory-bandwidth-bound1

1

3

3

.

Revolutionary Architecture Moves Memory Controller to Base Die

Traditional HBM architectures incorporate the memory controller into the primary silicon die on the package, consuming valuable real estate that could be dedicated to compute functions2

2

. NVHBM fundamentally changes this approach by integrating Nvidia's custom memory controller directly into the base die of the 3D HBM stack1

1

3

3

. This architectural shift frees up to 25% more area on XPU compute dies, allowing designers to add up to 30% more compute capabilities on the primary silicon die1

1

2

2

. The technology also simplifies interposer routing used to join multiple chips together in designs using advanced packaging techniques1

1

.

Power Efficiency Gains Enable Performance Scaling

NVHBM achieves 15% lower power consumption compared to commodity HBM4e, addressing a critical concern for AI infrastructure operators1

1

2

2

. These power savings can be reallocated into more functional units for custom AI chip designs, translated into higher sustained performance within the same power budget, or banked for improved performance-per-watt metrics1

1

. For hyperscalers deploying thousands of AI accelerators, these savings multiply significantly when moving massive data structures like model weights and KV caches1

1

. The energy saved on data movement can support larger numbers of accelerators within the same fixed power envelope1

1

.

Amazon Annapurna Labs Becomes First NVHBM Partner

Amazon's Annapurna Labs will be the first partner to collaborate on NVHBM technology as part of its broader work with Nvidia around NVLink Fusion2

2

3

3

. Nafea Bshara, vice president of Annapurna Labs at Amazon, stated that "NVHBM represents a new architectural approach to advancing high-bandwidth memory performance and efficiency" and expressed anticipation for how "this technology collaboration to benefit future AWS infrastructure designs"2

2

3

3

. Annapurna's next-generation Trainium 4 chips will support NVLink Fusion, allowing Amazon chips and Nvidia GPUs to work together within a common rack-scale architecture1

1

2

2

.

Standardized Implementation Accelerates Time-to-Market

Nvidia is establishing a standard NVHBM implementation that will be validated and offered by leading memory vendors, reducing the engineering effort required to integrate and qualify memory across multiple suppliers2

2

3

3

. This standardization provides NVLink Fusion customers with a faster path for bringing custom AI chips to market compared to implementing commodity HBM from the ground up1

1

2

2

. The technology will be available exclusively to Nvidia's custom silicon partners through the NVLink Fusion program, which also provides access to NVLink chiplets, NVLink-C2C, NVLink Switches, and Nvidia MGX systems and racks2

2

. Nvidia has indicated that NVHBM will be incorporated into future GPUs, with speculation pointing to the Feynman GPU generation expected in 2028 as a potential first adopter3

3

.