Micron ships 256GB SOCAMM2 modules to power next-generation AI infrastructure

Micron Ships Out the "World's First" 256GB SOCAMM2 Modules Targeted Toward the Agentic AI Frenzy

Micron's latest breakthrough in the memory industry is the debut of the more capable SOCAMM2 memory modules, featuring leading capacity and power efficiency. With the 'applications' layer of AI, the memory bottleneck is growing as workloads continue to scale, which is why DRAM manufacturers have paid special attention to advancements being made with HBM and other AI-specific memory products. In Micron's latest announcement, the firm has set a "new benchmark" with SOCAMM2 memory modules, as they ramp up the per-module capacity to 256 GB, marking a massive leap from the previous 192 GB threshold. According to the company, SOCAMM2 will be integrated into modern-day AI infrastructure equipment and help address memory constraints. Micron's achievements in delivering massive memory capacity and bandwidth using less power than traditional server memory with 256GB SOCAMM2 is enabling the next generation of AI CPUs. - Ian Finder, Head of Product, Data Center CPUs at NVIDIA With the latest SOCAMM2 iteration, Micron has increased the capacity of a single LPDRAM monolithic die to 32 GB. With the 256 GB model, 2 TB of LPDRAM per 8-channel CPU is provided, enabling AI servers to process long-context windows with ease. Micron also says that with the new SOCAMM2 solution, the TTFT has increased by 2.3 times for long-context inference, significantly helping complement agentic workloads, where standalone CPU applications are a key focus. SOCAMM2 is a solution developed in cooperation with NVIDIA, and in a previous post, we discussed how Vera Rubin will be one of the first AI infrastructure offerings to utilize the new memory standard. In the AI world, memory has become a significant asset in workloads where latency and context are important, but at the same time, SOCAMM2 is one of the products that would take up a decent portion of the DRAM supply, likely eating into allocations for general-purpose products like GDDR7. Micron says that 256GB SOCAMM2 samples have already been shipped to customers, and that the solution will be on showcase at this year's GTC 2026.

Micron Technology, Inc. Sets New Benchmark with the World's First High-Capacity 256Gb LPDRAM SOCAMM2 for Data Center Infrastructure

Micron Technology, Inc. extended its leadership in low-power server memory by shipping customer samples of the industry?s highest-capacity LPDRAM module ? 256GB SOCAMM2. Enabled by the industry's first monolithic 32Gb LPDDR5X design, this milestone represents a transformational step forward for AI data centers, delivering low-power memory capacity that can unlock new system architectures. The convergence of AI training, inference, agentic AI and general-purpose compute are driving more demanding memory requirements and reshaping data center system architectures. Modern AI workloads drive large model parameters, expansive context windows and persistent key value (KV) caches, while core compute continues to scale in data intensity, concurrency and memory footprint. Across these workloads, memory capacity, bandwidth efficiency, latency and power efficiency have become primary system level constraints, directly influencing performance, scalability and total cost of ownership. LPDRAM?s unique combination of these attributes position it as a cornerstone solution for both AI and core compute servers in increasingly power and thermally constrained data center environments. Micron is collaborating with NVIDIA to co-design sophisticated memory for the needs of advanced AI infrastructure. Micron?s 256GB SOCAMM2 delivers higher memory capacity, substantially lower power consumption and faster performance for a variety of AI and general-purpose computing workloads. With one-third more capacity than the prior highest capacity 192GB SOCAMM2, 256GB SOCAMM2 provides 2TB of LPDRAM per 8-channel CPU for larger context windows and complex inference workloads. SOCAMM2 consumes one-third of the power compared with equivalent RDIMMs, while using only one-third of the footprint, improving rack density and reducing the total cost of ownership. In unified memory architectures, 256GB SOCAMM2 improves time to first token by more than 2.3 times for long context, real-time LLM inference when used for KV cache offload compared to currently available solutions. In standalone CPU applications, LPDRAM delivers more than 3 times better performance per watt than mainstream memory modules for high-performance computing workloads. The modular SOCAMM2 design improves serviceability, supports liquid-cooled server architectures and enables future capacity expansion as AI and core compute memory requirements continue to grow. Micron continues to play a leading role in the JEDEC SOCAMM2 specification definition and maintains deep technical collaborations with system designers to drive industry-wide improvements in power efficiency and performance for next-generation data center platforms. Micron is now shipping customer samples of its 256GB SOCAMM2 and offers the industry?s broadest data center LPDRAM portfolio, spanning 8GB to 64GB components and 48GB to 256GB SOCAMM2 modules. One-third of the power consumption calculated based on watts of power used by one 128GB, 128-bit bus width SOCAMM2 module compared to two 64GB, 64-bit bus width DDR5 RDIMMs. One-third footprint calculation compares SOCAMM2 area (14x90mm) versus a standard server RDIMM. Results are based on Micron internal testing of real-time inference with Llama3 70B model (with FP16 quantization) using 500K context length and 16 concurrent users. The projected TTFT latency improvement is based on a latency of 0.12s for 2TB LPDRAM per CPU vs. 0.28s for 1.5TB LPDRAM per CPU. Micron internal testing measuring Pot3D solar physics HPC code performance on identical capacities of LPDDR5X and DDR5.