AI data storage emerges as the industry's biggest challenge beyond compute power

AI's biggest challenge is not compute - it's data storage

SPONSORED FEATURE: As AI evolves from novelty to autonomy, the real bottleneck isn't processing power -- it's where to put all that data. AI continues to evolve at pace. The novelty of generative models producing their own content is already giving way to the buzz around agentic systems that can set goals autonomously and execute multi-step workflows without human involvement. Each staging post on AI's journey raises the bar for compute resources and demands more powerful processing. But in the headlong dash towards newer and more specialized architectures, and amid the fuss that greets each silicon innovation, the industry is overlooking something important: beyond a certain scale, AI's biggest challenge is not compute but data storage. AI's trajectory towards human-like reasoning is driving exponential growth in data volume. Better AI needs better data in ever-increasing quantities, which raises the question of what to do with all that information. Picturing today's AI datacenters as giant compute systems is only half right. They are also data storage systems. The principal battleground in the AI arms race lies just as much at the storage layer as in chip development. The focus on compute capabilities alone in AI discourse is a legacy of the technology's origins, argues Nicolas Frapard, senior manager and regional lead, EMEAI at WD (also known as Western Digital). "Early on, AI's challenge was simply to make models work reliably and at scale," he points out. "That naturally put the spotlight on GPUs, accelerators, interconnect bandwidth, and overall compute density. Questions centered on how quickly models could be trained, how large they could become, and how efficiently clusters could scale. In that context, compute became a clear and measurable indicator of progress." Sustained, large-scale deployment expands the focus to the management and housing of information resources. The profit and loss implications of AI also change with scale: "As AI adoption expands to billions of interactions, data growth becomes structural rather than incidental," believes Frapard. "That has direct economic consequences. While compute tends to become more efficient over successive generations, data volumes continue to expand, driving sustained storage demand." Total cost of ownership (TCO) and return on investment (ROI) increasingly depend on how well organizations store and manage large-scale data estates. At exabyte scale, even small inefficiencies become magnified, which makes a lifecycle-driven approach to storage essential. Getting back to a balanced approach Those exploring AI-friendly storage strategies will have to grapple with a common myth. In recent years, IT decision-makers have been drip-fed the idea that storage tiers are irrelevant, and that hybrid architectures spreading data across different media according to need are redundant. For enterprises operating at scale, HDDs remain the backbone of storage architecture, providing the only viable way to store vast and growing datasets economically. Despite persistent narratives around all-flash environments, disk continues to underpin majority of enterprise data. But just as with the debate over compute versus storage, economics poses awkward questions once teams reach a certain scale of deployment. On a properly measured cost-per-terabyte basis, flash can clock in at up to 20 times more expensive than HDD. This does not negate the value of flash, but reinforces its role as one component within a broader architecture. The cost gap between flash and HDD was once expected to narrow until it would threaten the logic of disk drives. That never happened . In fact the gap has widened. Flash doesn't look set to become applicable across the board in any foreseeable timeframe . "The 'flash everywhere' approach emerged for similar reasons as the compute-centric view - at small scale when performance was the primary concern and data volumes were manageable, it was a good choice," explains Frapard. Flash offers low latency and high throughput, both critical for latency-sensitive workloads such as real-time inference. In early architectures, these requirements were often generalized across the entire system. But scale and maturity change the calculus. "At production scale, only a small proportion of data requires high-speed access," he points out. "The majority, such as logs, historical outputs, and training artifacts, should be stored reliably, accessed predictably, and retained economically over long periods. This is where HDDs becomes essential." High-performance storage such as flash has its place in any tiered arrangement. It sits close to compute, but the bulk of data is best housed in capacity-optimized HDD layers, where cost efficiency, density, durability, and energy consumption become the deciding factors. AI that delivers any meaningful return on investment requires a strategy that recognizes this and aligns storage technologies with data lifecycles, rather than applying a single performance standard across all data regardless of cost. Striking that balance reconciles scalability with long-term sustainability. HDD solutions optimized for the age of AI Data storage veteran WD has repositioned itself to help guide IT decision makers through this AI data conversation. Roughly 90 percent of its revenue now comes from AI and cloud. WD's latest storage roadmap reinvents the hard drive for AI needs. It has already produced a new generation of storage technologies spanning scalable capacity, performance optimizations, and power efficiency innovations. It also offers a API that will help accelerate platform storage deployment with cost-effective economics. "WD's position is rooted in a simple but important principle," explains Frapard: "At scale, AI is fundamentally a data system, and high-capacity HDD storage is what enables it to function sustainably and economically. Our strength lies in supporting that foundation across different tiers, particularly where scale and economics intersect." WD's portfolio, spanning high-capacity, energy-efficient drives as well as performance-oriented solutions, is designed with hyperscale and cloud use cases in mind. It focuses on balancing density, durability, and cost efficiency for large-scale AI workloads. The vendor's mission centers on enabling a system-level approach in which data can move between tiers, be retained economically, and remain accessible for ongoing use. "Ultimately, the long-term success of AI will not be defined solely by peak compute performance, but by how effectively organizations can manage and build value from their data over time," concludes Frapard, "That is the layer where WD is focused." Looking to the future At scale, IT decision-makers must prioritize data management rather than fixating on how much compute they need to deploy. Getting this balance right determines whether AI delivers sustained business value or drains resources for no obvious return. This is the central AI challenge for enterprises, and equally for emerging AI scalers such as neoclouds, sovereign clouds, and AI labs. All must control their data storage now or face bottlenecks, economic shortfalls, and failed AI initiatives further down the line. Hyperscalers and large cloud datacenters operating mixed-fleet storage architectures are already demonstrating the way forward. These systems incorporate enough flash to drive performance-sensitive workloads, combined with high-capacity HDD storage to support the vast amount of data that underpins AI systems at scale. The winners have figured out that IT infrastructure at scale is a nuanced, multi-tiered matter. The prize for getting this balance right goes beyond keeping a lid on accumulated data. It is the chance to lift the whole organization to a new level. "The real opportunity lies not simply in governing data, but in building systems where data remains accessible, reusable, and economically viable over time," explains Frapard. "When this is achieved, data becomes a strategic asset rather than an operational burden." AI systems, he says, improve through iteration when supported by the right data strategy. The ability to retain and reintroduce historical data enables continuous refinement, richer context, and better outcomes. Over time, this creates a compounding effect that can set organizations apart competitively. "However, this depends on architecture," he says. "If storage becomes constrained - either economically or operationally - organizations are forced into trade-offs about what to retain or discard. Those decisions directly impact the effectiveness of AI systems." The organizations that lead, he concludes, will be those that design their infrastructure with data longevity in mind, enabling them to build knowledge progressively rather than losing their critical institutional data over time .

Storage technology gets a promotion in age of agentic AI

The year 2026 could be remembered as the moment when storage technology received a massive promotion. The reason is that the current transition from simple chatbots to agentic AI systems has raised the stakes for context memory, the relevant information autonomous systems needs to understand and process a task. Supplying agents with data to support contextual working memory requires storage architecture with greater token throughput and improved efficiency. Storage's support of AI clusters has traditionally been confined to GPU servers or over a network in shared environments. Now, the ballgame has suddenly changed, according to Ace Stryker, director of AI and ecosystem marketing at Solidigm Inc. "It feels like storage kind of got a promotion," Stryker said, during a recent interview with theCUBE, SiliconANGLE Media's livestreaming studio. "What's new now is the third job. And that third job is new dedicated nodes specifically for storing context memory or KV cache. That's a completely new tier of storage in an AI cluster. We're going to have to come to terms with a whole lot more data and be able to store that with a combination of world-class hardware and software." This feature is part of SiliconANGLE Media's exploration of the architectural shifts powering continuous, production-grade AI. Be sure to check out SiliconANGLE's extensive coverage of RAISE Summit in Paris, July 8-9, featuring interviews with Solidigm executives and industry experts from d-Matrix, AMD, Neo4j, Tensordyne, Argentum AI, Cerebras, DDN and Canva, among others. (* Disclosure below.) Storage technology gets boost with CMX One market catalyst for storage's new assignment was the announcement of Nvidia Corp.'s BlueField-4 STX storage architecture in March. The release introduced Context Memory Storage, or CMX, a high-performance context layer that expanded GPU memory across the rack. The architecture's key engine is the BlueField-4 data processing unit, or DPU, that Nvidia unveiled in January. DPUs facilitate the offloading of infrastructure management tasks from a server's main processor, freeing more capacity for applications. BlueField-4 also handles tasks such as processing data traffic between GPUs and flash storage. "As AI infrastructure moves from proof of concept to production at enterprise scale, storage is becoming a strategic differentiator rather than a supporting component," said Paul Nashawaty, practice lead and principal analyst, application development, at theCUBE Research. "Organizations are discovering that GPU performance alone does not determine AI success. The ability to feed models with high-quality data, sustain throughput across distributed environments, and optimize infrastructure economics is equally important." STX sets the table for enterprises to store and reuse the massive key-value, or KV, cache that large language models and agentic AI inference can generate. AI workloads are moving from single prompts to agentic sessions with million-token context windows, increasing the volume of data into petabytes that exceed what standard GPU and DRAM memory tiers can handle. "You have to vectorize all this data and make it quickly searchable and accessible by AI models. All that has a storage cost, it's got to live somewhere," Stryker explained. "These models with these context windows that are just growing and growing, and these longer loops, more iterations ... all of that has incredible storage implications. It does not appear that this is a cyclical thing, that this is likely to wane anytime soon. That's where we find ourselves in 2026." Emerging role for context graph The storage industry's pursuit of solutions that address AI's quest for context highlights the growing need for backend systems and databases that work really well with autonomous technology. This will require an ability to cluster things together in ways that AI can use. One of the emerging tools for making this possible is the context graph, an accumulated structure of decision traces woven among entities and time so that precedent is searchable. Two researchers from Foundation Capital recently posted an analysis that suggested that the context graph could be AI's "trillion-dollar opportunity." One company seeking to capitalize on this architectural trend is Neo4j Inc. The company provides a command-line interface tool for generating full-stack applications with AI agents backed by graph databases for contextual memory. "In 2026, I think the big light bulb went off within the VC community," said Stephen Chin, vice president of developer relations at Neo4j, in an interview with theCUBE. "Basically, what they realized is that the reason why agents can't be successful is because they don't have the right context, they're so split on different data sources, on all of this tribal knowledge. And if you can bring those together and actually give agents full knowledge of the entire system, you get better decisions, you get better outcomes." Focus on inference One of the key factors driving the need for contextual memory is inference, the process of using a trained AI model to make real-time predictions or decisions on data. As theCUBE Research has noted, this is an area where the transformation of storage has become particularly significant. Large language models rely on KV cache to store intermediate data and maintain context, making it an essential ingredient in the inference process. It has also been a headache because the expansion of context windows can make KV cache expensive and slow down performance. The transformation of storage architecture disaggregates the need for data to reside in expensive GPU memory or move through less efficient CPUs. Storage is becoming part of the inference engine itself, and STX "is making storage workload-aware with specialized intelligence to make AI run better," according to theCUBE Research's Dave Vellante. The market implications for this are significant. A forecast from Deloitte notes that inference will account for approximately two-thirds of AI compute in 2026. "There is clear acknowledgement that the next big wave of AI computing is going to be around inference," said Sid Sheth, founder, president and CEO at d-Matrix Corp., in a recent conversation with theCUBE. "And I think people are just trying to figure out what does that really look like because it's not one size fits all. You can't really leverage a single computing platform for all of inference, because inference is done at different points in the network. You do it in big data centers, small data centers. We do it in edge applications. It's just going to be really spread out." Changing dynamics for developers With the current emphasis on inference and the significant shifts taking place in IT infrastructure, where does this leave today's developer? Findings from theCUBE Research show that developer experience directly impacts business outcomes. Organizations with high-quality developer experiences are 33% more likely to achieve their business goals and 31% more likely to improve software delivery flow. Yet, the tech industry is moving into a heterogeneous world where there will be a coexistence of different forms of compute and varying types of architectural solutions. In the past, developers didn't really need to worry about what the underlying hardware looked like. Now they do. "It's changing, you're seeing a very quick and dynamic shift that is happening in the underlying infrastructure, the underlying compute, which basically means if you want to write applications for that type of infrastructure, you really need application developers and programmers to understand what the underlying infrastructure looks like and how to program for it," Sheth noted. AI's rapidly developing ability to write code is also changing the equation. This is changing the very definition of developer responsibility when AI agents make changes and fixes. It is something that the developer community must come to terms with, according to Chin, and he believes the answer is clear. "It's the human who's accountable and responsible, but it's a human with a lot more capabilities, with a lot more tools at their disposal to actually participate and be an effective contributor," Chin told theCUBE. "Us humans, we're the community. But we are more capable, more empowered humans who have a lot of agents and tools at our disposal." (* Disclosure: TheCUBE is a paid media partner for RAISE Summit coverage. Neither Solidigm, the headline sponsor of theCUBE's event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)