Micron ships 256GB SOCAMM2 modules to customers, bringing 2TB AI memory capacity per CPU

Micron Ships Industry's First 256GB SOCAMM2 to Transform AI Data Centers

Micron has begun shipping customer samples of what it claims is the industry's first 256GB LPDRAM SOCAMM2 memory module, a development that positions AI data centers to dramatically expand their memory capacity while reducing power consumption 1

1

. The new high-density memory module represents a 33% capacity increase over the previous generation's 192GB modules released just six months ago, enabling AI servers to reach 2TB of LPDRAM per 8-channel CPU 2

2

. This advancement addresses a critical bottleneck as large language models and inference workloads increasingly demand enormous memory pools to handle expanding context windows and complex AI tasks.

Monolithic 32Gb LPDDR5X Dies Power Unprecedented Density

The 256GB SOCAMM2 module achieves its capacity through Micron's industry-first monolithic 32Gb LPDDR5X dies, where all memory and relevant circuitry are integrated into a single die 3

3

. Each module packs 64 of these 32GB chips into a compact form factor that consumes approximately one-third the power of comparable RDIMMs while occupying only one-third of the physical footprint 5

5

. This power efficiency translates directly into reduced thermal load and infrastructure costs for data center operators deploying hundreds of billions of dollars in AI infrastructure. The smaller footprint also improves rack density, allowing more computing power to fit within existing data center space constraints.

NVIDIA Collaboration Drives AI Memory Innovation

Micron developed the 256GB SOCAMM2 in collaboration with NVIDIA, building on the SOCAMM2 standard that emerged from a partnership between NVIDIA and memory manufacturers Micron, Samsung, and SK hynix 1

1

. The original SOCAMM standard was designed by NVIDIA, but the company faced challenges with overheating on high-density servers, prompting collaboration with memory specialists to create the improved SOCAMM2 format. Ian Finder, Head of Product for Data Center CPUs at NVIDIA, noted that "Micron's achievements in delivering massive memory capacity and bandwidth using less power than traditional server memory with 256GB SOCAMM2 is enabling the next generation of AI CPUs" 4

4

. The modular SOCAMM2 design supports liquid-cooled server architectures and enables future capacity expansion as AI memory requirements continue to grow.

Performance Gains Target Time To First Token and Agentic AI

The increased memory capacity directly impacts AI performance metrics that matter most to users. Micron reports that the 256GB SOCAMM2 improves Time To First Token (TTFT) by more than 2.3 times for long-context, real-time LLM inference when used for key value cache offloading compared to currently available solutions 5

5

. This improvement was demonstrated in internal testing using the Llama3 70B model with FP16 quantization, 500K context length, and 16 concurrent users. In standalone CPU applications focused on High-Performance Computing (HPC) workloads, LPDRAM delivers more than 3 times better performance per watt than mainstream memory modules. The solution particularly benefits agentic AI workloads where standalone CPU applications play a key role in processing complex, multi-step tasks that require sustained access to large context windows 4

4

.

Datacenter Deployment and Industry Adoption

For typical NVIDIA NVL72 rack configurations, the new modules enable deployment of 72TB of RAM across 36 CPUs, a substantial increase that allows AI systems to accommodate larger model parameters and more demanding inference tasks 1

1

. Raj Narasimhan, senior vice president and general manager of Micron's Cloud Memory Business Unit, emphasized that "Micron's 256GB SOCAMM2 offering enables the most power-efficient CPU-attached memory solution for both AI and HPC" 3

3

. Micron continues to play a leading role in the JEDEC SOCAMM2 specification definition and maintains deep technical collaborations with system designers to drive industry-wide improvements. The company now offers the industry's broadest data center LPDRAM portfolio, spanning 8GB to 64GB components and 48GB to 256GB SOCAMM2 modules, with the 256GB version set to be showcased at GTC 2026 4

4

. As AI workloads continue to scale and memory becomes an increasingly critical constraint on system performance and scalability, these high-capacity modules address the convergence of AI training, inference, and general-purpose compute that is reshaping data center system architectures.