NVIDIA Open Sources cuFile API to Enable Microsecond GPU Data Access for AI Storage

As AI Increases Demands on Memory, Storage Steps Up

At FMS, NVIDIA shows how accelerated computing enables AI applications to access storage directly -- fast enough to act like memory and secure by design. Surging AI demands are driving the need for massive datasets and context windows that burst past the confines of system memory. But rising needs aren't met by simply adding more storage capacity. What's needed is useful, grounded insights from AI factories and efficient, secure storage architectures that enable those insights. At this week's Future of Memory and Storage (FMS) conference, NVIDIA is unveiling new storage advancements and showcasing how the next leap in AI depends as much on the storage infrastructure feeding accelerated computing as on the computing power itself. The pressure on that infrastructure is intensifying as AI agents consume massive amounts of data -- and GPUs can now initiate storage requests directly, generating thousands of concurrent operations. To serve those requests, storage systems must continuously encrypt, compress, verify and reconstruct data. These critical data services can become bottlenecks when thousands of agents access storage simultaneously. Benchmarks highlighted in this NVIDIA technical blog show that the NVIDIA Vera CPU, part of NVIDIA Vera BlueField-4 STX, delivers up to 3.21x higher throughput than an x86 CPU in a two-stage compression and encryption pipeline. This means that with Vera, storage platforms can absorb the flood of AI data more efficiently -- delivering greater throughput with significantly less compute infrastructure. With accelerated computing, storage stops being a passive place to keep data and becomes an active part of the data path. This upends the old economics of determining when data belongs in memory (where applications can fetch it faster) versus on a storage drive (where it can be held in cheap and plentiful space). The tradeoff was first framed 40 years ago, when the answer was measured in accessing that data in minutes. On today's GPUs, paired with AI storage solutions from NVIDIA and partners, the same tradeoff now plays out in microseconds. Closing the gap between AI's needs and memory shortage depends on extreme codesign across the whole ecosystem, from memory and storage manufacturers to the software built on them. Open Source NVIDIA cuFile APIs Enable Interoperability for Storage Solutions At FMS, NVIDIA announced it is open sourcing its cuFile application programming interfaces (APIs) -- and the vertical storage software stack underneath them -- which let GPUs, not just CPUs, read from and write to storage directly. cuFile is an open source component of NVIDIA GPUDirect Storage. Using hundreds of thousands of GPU threads, fast high-bandwidth memory and other methodologies, cuFile enables securely accessing data from storage in just microseconds. This represents how the industry is unifying a security-first storage stack based on Linux best practices, providing interoperability between GPUs and data. In addition, fast, secure access to data and storage is a foundational element to powering preventive and detective cybersecurity measures. Making cuFile openly available will help make security context, data and storage accessible at the speed AI-powered defenses need. Such open technologies support initiatives such as the new Open Secure AI Alliance. This site is the new home for APIs that are open to contributions -- with Google, Intel, NVIDIA and Meta as inaugural maintainers -- and can be optimized for use across various software and hardware platforms, driving innovation and efficiency for developers and enterprises. NVIDIA and Industry Leaders Advance New Frontier of AI Storage In addition, NVIDIA and storage industry leaders are optimizing memory and storage solutions through an initiative called Storage-Next. The NVIDIA-driven initiative brings together storage makers, controller vendors, thermal design, cooling and orchestration operators, and standards bodies to align on how GPU-driven storage should behave -- then turn these advancements into interoperable, open industry standards. Storage-Next includes over 40 leading storage and flash vendors -- including DDN, KIOXIA and Micron -- each contributing to the next generation of AI storage technologies with NVIDIA. The initiative is grounded in accelerated data access for large AI datasets. To support this, NVIDIA offers SCADA -- short for scaled, accelerated data access -- a framework that lets massively parallel GPUs pull only the data necessary for the application directly from storage into their own high-speed memory. For example, DDN is integrating SCADA with Infinia, its software-defined, AI-native data intelligence platform built to eliminate storage bottlenecks at scale. "AI success will be defined not by how much infrastructure organizations own, but by how productively they use it," said Sven Oehme, chief technology officer at DDN. "Our collaboration with NVIDIA is helping create a more direct, efficient connection between GPUs and data -- keeping accelerated computing resources productive, speeding time to insight and enabling customers to achieve stronger business and financial returns from their AI investments." Storage-Next and SCADA extend NVIDIA's longstanding work on AI storage infrastructure, including on NVIDIA Vera BlueField-4 STX -- a modular, rack-scale foundation powered by the NVIDIA Vera Rubin platform, NVIDIA Vera BlueField-4 storage processors and NVIDIA Spectrum-X Ethernet networking. Defining a new class of AI-native data platforms, NVIDIA STX uses the unified NVIDIA DOCA security stack to let enterprises enable continuous policy enforcement in the AI data path. Plus, NVIDIA CMX Context Memory Storage provides an AI‑native context tier for long‑context, multi‑turn, agentic AI inference, built on NVIDIA STX. SCADA Enables Fast AI Storage That Stays Secure Speed at the storage layer comes with a catch. Letting an application talk straight to a drive is quick, but done carelessly, it can scribble over other processes' memory -- a security hole, not a feature. NVIDIA SCADA uses a safe, robust method to achieve scaled direct access by splitting the job in two: * The user parts of an application that need raw speed stay outside the trusted computing base. * A separate, privileged component configures protected access between the user application and its approved storage at setup, adhering to standard Linux protocols for security enforcement while efficiently safeguarding data. It's all part of how advancements in fast, massively parallel, efficient, secure AI storage infrastructure can feed better data to applications and AI factories -- so they can produce more useful, accurate, grounded intelligence at scale. Join NVIDIA sessions at FMS, running Aug. 4-6 in Santa Clara, California, and learn more about NVIDIA AI storage. See notice regarding software product information.

Nvidia open sources cuFile API, accelerating GPU read/write capability for high-speed storage

As artificial intelligence applications become ever hungrier for faster access to data, Nvidia Corp. today announced it is open-sourcing the application programming interface for its powerful cuFile vertical data storage stack, enabling millisecond data access. The company also announced a large-scale industry initiative with technology leaders in optimizing memory and storage solutions with Storage-Next. The initiative aims to bring storage makers, controller vendors, thermal design, cooling, and orchestration providers together with standard bodies to find opportunities for superior graphics processing unit-driven storage. During Nvidia's announcement at the Future of Memory and Storage conference, the company said the cuFile API enables secure access from storage in just milliseconds. CuFile was first officially launched into general availability in July 2021 alongside the company's GPU access CUDA Toolkit 11.4. Prior to the launch, it acted as a core component of the company's GPUDirect Storage software. It provides direct, high-speed data access between local distributed storage and GPU memory; this allows GPUs to bypass the central processing unit and system main memory, which greatly reduces memory access delays. Using direct memory access, or DMA, it can move data directly from storage devices such as NVMe drives into GPU memory. The primary source of AI inference happens on GPUs and specialized cards that interact primarily with GPU memory; refreshing that memory is often completed by another software layer that uses the GPU to orchestrate and make decisions. CuFile cuts out the "middle man," so to speak, providing a direct access path and enabling GPU orchestration, meaning it can rapidly pull more data. This allows the synchronization of massive data sets at rates that can keep up with elite GPU speeds. If the bandwidth to this data is delayed or pinched off by a bottleneck, it causes something called the "GPU starvation" loop, where some GPUs in a distributed array sit idle waiting for data to be ready. Increased speeds and low millisecond delays allow massive datasets from retrieval-augmented generation and agentic AI to run faster by providing them a low-latency "superhighway" between deep storage and the GPU itself. Bringing open standards to GPU accessible storage Storage-Next includes 40 leading storage and flash memory vendors, including DataDirect Networks, Inc., Kioxia Corp. and Micron Technology Inc. Each will contribute to the next generation of AI storage technologies with Nvidia. The company will build its grounded, high-speed data access for storage off SCADA, short for scaled, accelerated data access. This solution provides the groundwork for massively parallel GPUs to pull data at high bandwidth and extremely low delay. Modern AI training requires massive data access all at once and AI inference now includes colossal mixture-of-experts models, which "think" extremely quickly, rapidly pulling data and firing tools for agentic workflows. "AI success will be defined not by how much infrastructure organizations own, but by how productively they use it," said DDN Chief Technology Officer Sven Oehme. "Our collaboration with Nvidia is helping create a more direct, efficient connection between GPUs and data -- keeping accelerated computing resources productive, speeding time to insight and enabling customers to achieve stronger business and financial returns from their AI investments." Nvidia said SCADA allows high-speed storage layers to operate securely. Technologies such as SCADA allow direct access to storage media by the AI; this punches through safeguards designed to prevent clobber - what happens when two processes attempt to write to the same data at once, with one scribbling over the other's work. This is a security problem, not a proper feature. The other side of the coin means that this might bypass security safeguards such as encrypted and privileged access. This opens up another security hole that malicious parties could exploit. SCADA overcomes this challenge by scaling direct access by splitting access across different jobs: the user parts of the application receive raw speed but stay outside the encrypted, secure and trusted computing base; and a separate privileged computing component configures protected access following Linux protocols to prevent clobber.