5 Sources
[1]
AI's biggest challenge is not compute - it's data storage
SPONSORED FEATURE: As AI evolves from novelty to autonomy, the real bottleneck isn't processing power -- it's where to put all that data. AI continues to evolve at pace. The novelty of generative models producing their own content is already giving way to the buzz around agentic systems that can set goals autonomously and execute multi-step workflows without human involvement. Each staging post on AI's journey raises the bar for compute resources and demands more powerful processing. But in the headlong dash towards newer and more specialized architectures, and amid the fuss that greets each silicon innovation, the industry is overlooking something important: beyond a certain scale, AI's biggest challenge is not compute but data storage. AI's trajectory towards human-like reasoning is driving exponential growth in data volume. Better AI needs better data in ever-increasing quantities, which raises the question of what to do with all that information. Picturing today's AI datacenters as giant compute systems is only half right. They are also data storage systems. The principal battleground in the AI arms race lies just as much at the storage layer as in chip development. The focus on compute capabilities alone in AI discourse is a legacy of the technology's origins, argues Nicolas Frapard, senior manager and regional lead, EMEAI at WD (also known as Western Digital). "Early on, AI's challenge was simply to make models work reliably and at scale," he points out. "That naturally put the spotlight on GPUs, accelerators, interconnect bandwidth, and overall compute density. Questions centered on how quickly models could be trained, how large they could become, and how efficiently clusters could scale. In that context, compute became a clear and measurable indicator of progress." Sustained, large-scale deployment expands the focus to the management and housing of information resources. The profit and loss implications of AI also change with scale: "As AI adoption expands to billions of interactions, data growth becomes structural rather than incidental," believes Frapard. "That has direct economic consequences. While compute tends to become more efficient over successive generations, data volumes continue to expand, driving sustained storage demand." Total cost of ownership (TCO) and return on investment (ROI) increasingly depend on how well organizations store and manage large-scale data estates. At exabyte scale, even small inefficiencies become magnified, which makes a lifecycle-driven approach to storage essential. Getting back to a balanced approach Those exploring AI-friendly storage strategies will have to grapple with a common myth. In recent years, IT decision-makers have been drip-fed the idea that storage tiers are irrelevant, and that hybrid architectures spreading data across different media according to need are redundant. For enterprises operating at scale, HDDs remain the backbone of storage architecture, providing the only viable way to store vast and growing datasets economically. Despite persistent narratives around all-flash environments, disk continues to underpin majority of enterprise data. But just as with the debate over compute versus storage, economics poses awkward questions once teams reach a certain scale of deployment. On a properly measured cost-per-terabyte basis, flash can clock in at up to 20 times more expensive than HDD. This does not negate the value of flash, but reinforces its role as one component within a broader architecture. The cost gap between flash and HDD was once expected to narrow until it would threaten the logic of disk drives. That never happened . In fact the gap has widened. Flash doesn't look set to become applicable across the board in any foreseeable timeframe . "The 'flash everywhere' approach emerged for similar reasons as the compute-centric view - at small scale when performance was the primary concern and data volumes were manageable, it was a good choice," explains Frapard. Flash offers low latency and high throughput, both critical for latency-sensitive workloads such as real-time inference. In early architectures, these requirements were often generalized across the entire system. But scale and maturity change the calculus. "At production scale, only a small proportion of data requires high-speed access," he points out. "The majority, such as logs, historical outputs, and training artifacts, should be stored reliably, accessed predictably, and retained economically over long periods. This is where HDDs becomes essential." High-performance storage such as flash has its place in any tiered arrangement. It sits close to compute, but the bulk of data is best housed in capacity-optimized HDD layers, where cost efficiency, density, durability, and energy consumption become the deciding factors. AI that delivers any meaningful return on investment requires a strategy that recognizes this and aligns storage technologies with data lifecycles, rather than applying a single performance standard across all data regardless of cost. Striking that balance reconciles scalability with long-term sustainability. HDD solutions optimized for the age of AI Data storage veteran WD has repositioned itself to help guide IT decision makers through this AI data conversation. Roughly 90 percent of its revenue now comes from AI and cloud. WD's latest storage roadmap reinvents the hard drive for AI needs. It has already produced a new generation of storage technologies spanning scalable capacity, performance optimizations, and power efficiency innovations. It also offers a API that will help accelerate platform storage deployment with cost-effective economics. "WD's position is rooted in a simple but important principle," explains Frapard: "At scale, AI is fundamentally a data system, and high-capacity HDD storage is what enables it to function sustainably and economically. Our strength lies in supporting that foundation across different tiers, particularly where scale and economics intersect." WD's portfolio, spanning high-capacity, energy-efficient drives as well as performance-oriented solutions, is designed with hyperscale and cloud use cases in mind. It focuses on balancing density, durability, and cost efficiency for large-scale AI workloads. The vendor's mission centers on enabling a system-level approach in which data can move between tiers, be retained economically, and remain accessible for ongoing use. "Ultimately, the long-term success of AI will not be defined solely by peak compute performance, but by how effectively organizations can manage and build value from their data over time," concludes Frapard, "That is the layer where WD is focused." Looking to the future At scale, IT decision-makers must prioritize data management rather than fixating on how much compute they need to deploy. Getting this balance right determines whether AI delivers sustained business value or drains resources for no obvious return. This is the central AI challenge for enterprises, and equally for emerging AI scalers such as neoclouds, sovereign clouds, and AI labs. All must control their data storage now or face bottlenecks, economic shortfalls, and failed AI initiatives further down the line. Hyperscalers and large cloud datacenters operating mixed-fleet storage architectures are already demonstrating the way forward. These systems incorporate enough flash to drive performance-sensitive workloads, combined with high-capacity HDD storage to support the vast amount of data that underpins AI systems at scale. The winners have figured out that IT infrastructure at scale is a nuanced, multi-tiered matter. The prize for getting this balance right goes beyond keeping a lid on accumulated data. It is the chance to lift the whole organization to a new level. "The real opportunity lies not simply in governing data, but in building systems where data remains accessible, reusable, and economically viable over time," explains Frapard. "When this is achieved, data becomes a strategic asset rather than an operational burden." AI systems, he says, improve through iteration when supported by the right data strategy. The ability to retain and reintroduce historical data enables continuous refinement, richer context, and better outcomes. Over time, this creates a compounding effect that can set organizations apart competitively. "However, this depends on architecture," he says. "If storage becomes constrained - either economically or operationally - organizations are forced into trade-offs about what to retain or discard. Those decisions directly impact the effectiveness of AI systems." The organizations that lead, he concludes, will be those that design their infrastructure with data longevity in mind, enabling them to build knowledge progressively rather than losing their critical institutional data over time .
[2]
Inference needs memory: how context is becoming AI infrastructure
As enterprise AI systems evolve, the limiting factor is shifting. Model quality still matters, but it's no longer the main issue holding systems back. Increasingly, what constrains performance, scalability, and cost is context. Large language models are now expected to support long conversations, multi step reasoning, and complex workflows that span time, users, and systems. Every one of those interactions generates tokens, and those tokens produce key value (KV) cache -- the working memory that allows models to reason efficiently without constantly recomputing prior steps. Most AI architectures still treat this context as temporary. KV cache typically lives in GPU memory, is tied to a single inference process, and is discarded as soon as resources are exhausted. That approach might be acceptable for small scale experimentation, but it quickly breaks down in enterprise environments where context lengths grow, concurrency increases, and recomputation becomes expensive. Inference context has quietly become one of the largest bottlenecks in enterprise AI. KV cache as AI native data To understand why this matters, it helps to stop thinking about KV cache as "just a cache." Enterprises have spent decades building strategies around structured data and unstructured data, but AI introduces a third class that deserves just as much attention: AI native data. This is data generated by model execution itself, and KV cache is one of its most important forms. KV cache directly determines inference latency, throughput, energy consumption, and cost. As context windows get longer and reasoning chains become deeper, the volume and importance of this data grow faster than token counts alone. When KV cache is constantly thrown away, systems pay for it through rising latency, lower GPU utilization, lost reasoning context, and higher inference costs. At scale, this inefficiency becomes structural rather than incidental. Why existing infrastructure assumptions don't hold KV cache also exposes a mismatch with traditional infrastructure design. GPU memory delivers exceptional performance, but it is scarce and local to a single server. CPU memory extends capacity but remains volatile. Local NVMe storage adds scale yet keeps context trapped at the node level. Traditional shared storage provides durability and resilience, but it wasn't designed for highly dynamic, inference time state. This leaves enterprises with a fragmented memory hierarchy where context is either fast but fragile, or persistent but difficult to access efficiently. No amount of tuning can fully resolve this, because the problem isn't optimization -- it's architecture. What enterprise AI needs is a way to treat inference context as system memory rather than disposable state. Introducing an inference context memory layer That shift is what we describe as an inference context memory layer. Instead of forcing all KV cache to live and die inside GPU memory, this approach allows context to be created close to the GPU for low latency, then managed across a hierarchy of memory and storage tiers designed explicitly for inference workloads. Inactive context can move out of high cost memory without being discarded, while relevant context can be restored on demand without recomputation. This changes the behavior of inference systems in a fundamental way. Inference is no longer a series of isolated executions that start from scratch each time. It becomes a continuous, stateful process where knowledge accumulates, moves, and is reused across sessions, agents, and nodes. When storage becomes part of AI memory Making this work places new demands on storage. Inference context is large, mostly immutable, and technically recomputable -- but regenerating it at scale is costly and inefficient. A storage architecture for inference context must preserve locality when performance matters, enable sharing without manual replication, and provide resilience so context isn't lost when hardware fails. When storage is designed this way, it stops being just a place to store data and becomes an extension of AI memory itself. That shift has real economic consequences: faster time to first token, higher GPU utilization, support for much longer sessions, and dramatically lower cost per query. For enterprise workloads like tax advisory, legal analysis, healthcare reasoning, financial planning, and customer support, this is critical. These systems depend on preserving reasoning history and conversational context, not repeatedly rebuilding it from scratch. Context is now infrastructure Enterprise AI is entering a new phase. Models will continue to advance, but the systems that scale successfully will be defined by how well they manage the intelligence those models produce. Tokens are no longer fleeting artifacts, and context is no longer something enterprises can afford to lose. KV cache is AI native data. It represents system state. And increasingly, it must be treated as infrastructure. The architectural principle is simple: generate context once, manage it intelligently, and reuse it wherever possible. That shift is foundational to making enterprise AI reliable, efficient, and scalable -- and it's why storage once again plays a central role in the future of computing. Use the best business cloud storage to manage your data. This article was produced as part of TechRadar Pro Perspectives, our channel to feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/pro/perspectives-how-to-submit
[3]
Intelligence layer memory extension accelerates agentic AI
Solidigm targets the intelligence layer as agentic inference pushes storage to center stage The shift from model training to agentic inference is forcing a fundamental rethink of how artificial intelligence infrastructure is built and which components carry the most strategic weight. What was once treated as commodity plumbing is now being recognized as the intelligence layer where raw data becomes actionable intelligence. The rise of sovereign AI deployments and enterprise AI clusters is compounding that pressure, as organizations from hyperscalers to regional governments race to build out high-performance compute stacks that can sustain continuous inference at scale. Storage has quietly moved from afterthought to forethought in those conversations, according to Greg Matson (pictured), senior vice president and head of marketing and products at Solidigm, a trademark of SK Hynix NAND Product Solutions Corp. "It started a couple of years ago with training, where the need for high-capacity, high-performance storage very adjacent to the GPUs was all of a sudden center stage," Matson said. "But now at the inference, as we go from last year into this year inference phase into agentic inference, it's exploding even more. ... Storage is actually a whole new storage tier that's being created to extend the memory for the system. ... I almost call it the intelligence layer is now being housed on storage." Matson spoke with theCUBE's John Furrier and Dave Vellante at RAISE Summit 2026, during an exclusive broadcast on theCUBE, SiliconANGLE Media's livestreaming studio. They discussed how high-capacity SSDs are becoming essential intelligence layer infrastructure for AI inference, token economics and sovereign compute deployments. (* Disclosure below.) Intelligence layer economics reshape the AI infrastructure stack As inference workloads scale from single prompts to sprawling agentic sessions, the gap between GPU memory and the data feeding it has become a critical bottleneck. Hyperscalers recognized the problem roughly 12 to 18 months ago and began replacing legacy storage infrastructure -- some of it more than a decade old -- with high-capacity solid-state drives to keep GPUs fully utilized, Matson noted. "While the GPU is the most expensive part of your infrastructure, you want that thing to be humming 100% of the time generating tokens," he said. "And if it's down, right, waiting for data, then you're wasting your money on GPUs." The economics of token generation make that case concrete. A simple 15-word prompt can generate as many as 40,000 tokens -- representing five to 10 gigabytes of context data -- and multiplying that across enterprise workforces quickly pushes storage requirements into the petabytes, Matson explained. Solidigm has responded with drives that now reach 122 terabytes of capacity per unit, alongside the industry's first cold-plate-cooled enterprise SSD designed for fully fanless Nvidia GPU servers -- a product built as AI racks shed fans entirely in favor of liquid cooling. "Everything that's coming out over the next couple of years, nothing will be air-cooled," Matson said. "The infrastructure is the data layer. The data layer is the storage layer. That's the Solidigm layer." Here's the complete video interview, part of SiliconANGLE's and theCUBE's coverage of RAISE Summit 2026: (* Disclosure: TheCUBE is a paid media partner for RAISE Summit 2026. Sponsors of theCUBE's event coverage do not have editorial control over content on theCUBE or SiliconANGLE.)
[4]
Storage technology gets a promotion in age of agentic AI
The year 2026 could be remembered as the moment when storage technology received a massive promotion. The reason is that the current transition from simple chatbots to agentic AI systems has raised the stakes for context memory, the relevant information autonomous systems needs to understand and process a task. Supplying agents with data to support contextual working memory requires storage architecture with greater token throughput and improved efficiency. Storage's support of AI clusters has traditionally been confined to GPU servers or over a network in shared environments. Now, the ballgame has suddenly changed, according to Ace Stryker, director of AI and ecosystem marketing at Solidigm Inc. "It feels like storage kind of got a promotion," Stryker said, during a recent interview with theCUBE, SiliconANGLE Media's livestreaming studio. "What's new now is the third job. And that third job is new dedicated nodes specifically for storing context memory or KV cache. That's a completely new tier of storage in an AI cluster. We're going to have to come to terms with a whole lot more data and be able to store that with a combination of world-class hardware and software." This feature is part of SiliconANGLE Media's exploration of the architectural shifts powering continuous, production-grade AI. Be sure to check out SiliconANGLE's extensive coverage of RAISE Summit in Paris, July 8-9, featuring interviews with Solidigm executives and industry experts from d-Matrix, AMD, Neo4j, Tensordyne, Argentum AI, Cerebras, DDN and Canva, among others. (* Disclosure below.) Storage technology gets boost with CMX One market catalyst for storage's new assignment was the announcement of Nvidia Corp.'s BlueField-4 STX storage architecture in March. The release introduced Context Memory Storage, or CMX, a high-performance context layer that expanded GPU memory across the rack. The architecture's key engine is the BlueField-4 data processing unit, or DPU, that Nvidia unveiled in January. DPUs facilitate the offloading of infrastructure management tasks from a server's main processor, freeing more capacity for applications. BlueField-4 also handles tasks such as processing data traffic between GPUs and flash storage. "As AI infrastructure moves from proof of concept to production at enterprise scale, storage is becoming a strategic differentiator rather than a supporting component," said Paul Nashawaty, practice lead and principal analyst, application development, at theCUBE Research. "Organizations are discovering that GPU performance alone does not determine AI success. The ability to feed models with high-quality data, sustain throughput across distributed environments, and optimize infrastructure economics is equally important." STX sets the table for enterprises to store and reuse the massive key-value, or KV, cache that large language models and agentic AI inference can generate. AI workloads are moving from single prompts to agentic sessions with million-token context windows, increasing the volume of data into petabytes that exceed what standard GPU and DRAM memory tiers can handle. "You have to vectorize all this data and make it quickly searchable and accessible by AI models. All that has a storage cost, it's got to live somewhere," Stryker explained. "These models with these context windows that are just growing and growing, and these longer loops, more iterations ... all of that has incredible storage implications. It does not appear that this is a cyclical thing, that this is likely to wane anytime soon. That's where we find ourselves in 2026." Emerging role for context graph The storage industry's pursuit of solutions that address AI's quest for context highlights the growing need for backend systems and databases that work really well with autonomous technology. This will require an ability to cluster things together in ways that AI can use. One of the emerging tools for making this possible is the context graph, an accumulated structure of decision traces woven among entities and time so that precedent is searchable. Two researchers from Foundation Capital recently posted an analysis that suggested that the context graph could be AI's "trillion-dollar opportunity." One company seeking to capitalize on this architectural trend is Neo4j Inc. The company provides a command-line interface tool for generating full-stack applications with AI agents backed by graph databases for contextual memory. "In 2026, I think the big light bulb went off within the VC community," said Stephen Chin, vice president of developer relations at Neo4j, in an interview with theCUBE. "Basically, what they realized is that the reason why agents can't be successful is because they don't have the right context, they're so split on different data sources, on all of this tribal knowledge. And if you can bring those together and actually give agents full knowledge of the entire system, you get better decisions, you get better outcomes." Focus on inference One of the key factors driving the need for contextual memory is inference, the process of using a trained AI model to make real-time predictions or decisions on data. As theCUBE Research has noted, this is an area where the transformation of storage has become particularly significant. Large language models rely on KV cache to store intermediate data and maintain context, making it an essential ingredient in the inference process. It has also been a headache because the expansion of context windows can make KV cache expensive and slow down performance. The transformation of storage architecture disaggregates the need for data to reside in expensive GPU memory or move through less efficient CPUs. Storage is becoming part of the inference engine itself, and STX "is making storage workload-aware with specialized intelligence to make AI run better," according to theCUBE Research's Dave Vellante. The market implications for this are significant. A forecast from Deloitte notes that inference will account for approximately two-thirds of AI compute in 2026. "There is clear acknowledgement that the next big wave of AI computing is going to be around inference," said Sid Sheth, founder, president and CEO at d-Matrix Corp., in a recent conversation with theCUBE. "And I think people are just trying to figure out what does that really look like because it's not one size fits all. You can't really leverage a single computing platform for all of inference, because inference is done at different points in the network. You do it in big data centers, small data centers. We do it in edge applications. It's just going to be really spread out." Changing dynamics for developers With the current emphasis on inference and the significant shifts taking place in IT infrastructure, where does this leave today's developer? Findings from theCUBE Research show that developer experience directly impacts business outcomes. Organizations with high-quality developer experiences are 33% more likely to achieve their business goals and 31% more likely to improve software delivery flow. Yet, the tech industry is moving into a heterogeneous world where there will be a coexistence of different forms of compute and varying types of architectural solutions. In the past, developers didn't really need to worry about what the underlying hardware looked like. Now they do. "It's changing, you're seeing a very quick and dynamic shift that is happening in the underlying infrastructure, the underlying compute, which basically means if you want to write applications for that type of infrastructure, you really need application developers and programmers to understand what the underlying infrastructure looks like and how to program for it," Sheth noted. AI's rapidly developing ability to write code is also changing the equation. This is changing the very definition of developer responsibility when AI agents make changes and fixes. It is something that the developer community must come to terms with, according to Chin, and he believes the answer is clear. "It's the human who's accountable and responsible, but it's a human with a lot more capabilities, with a lot more tools at their disposal to actually participate and be an effective contributor," Chin told theCUBE. "Us humans, we're the community. But we are more capable, more empowered humans who have a lot of agents and tools at our disposal." (* Disclosure: TheCUBE is a paid media partner for RAISE Summit coverage. Neither Solidigm, the headline sponsor of theCUBE's event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)
[5]
Moving Beyond the GPU: Seagate's Jason Feist on Balancing Compute and Storage for Long-Term AI Growth
As organizations rapidly transition from AI pilots to full-scale production, the conversation surrounding infrastructure must expand beyond GPUs and raw compute power to address the massive influx of data. Efficiently scaling AI requires a fundamental shift toward intelligent, software-defined storage architectures that can handle exponentially larger datasets, continuous data reuse, and global distribution without creating costly bottlenecks. Maximizing the value of these next-generation AI investments hinges on an enterprise's ability to build a persistent, shared data foundation where compute and storage scale independently. Achieving this balance means rethinking traditional data storage not as a passive IT resource, but as a strategic layer critical to balancing performance, economics, and long-term resilience. This evolution is particularly crucial in rapidly expanding tech ecosystems like India, where robust data infrastructure must evolve at the same pace as digital ambitions to unlock sustainable business value. For a deeper look at how modern data architectures are redefining the future of enterprise technology, we turn to Jason Feist, Senior Vice President, Cloud Business, Seagate. CXOToday: AI infrastructure discussions often focus on GPUs, chips and compute. Why should storage receive equal attention as organizations scale AI? Jason: While GPUs and compute power are essential for AI, they're only as effective as the data they can access. As organizations move from AI pilots to production, they're working with exponentially larger datasets that require more frequent access, longer context windows, greater parallelism, longer retention, and continuous reuse. Without the right storage foundation, even the most advanced compute infrastructure can become a bottleneck. Storage is no longer simply where data resides. It's increasingly becoming the persistent system of record that AI workloads continuously return to, even as compute resources scale independently. That's a fundamental shift in how AI infrastructure is being designed. Today, 87% of exabytes in large data centers are stored on hard drives, reflecting the continued role of high-capacity storage in AI and cloud environments. Organizations that optimize compute and storage together will be better positioned to maximize AI performance and infrastructure efficiency. CXOToday: How are AI and cloud workloads redefining storage, and what does this mean for enterprises adopting AI? Jason: For cloud providers, hyperscalers and AI-led enterprises, AI is fundamentally redefining what primary storage means. Traditionally, primary storage relied on tightly coupled block and file systems located close to compute. Today, cloud-native environments increasingly rely on software-defined, globally distributed object storage as the persistent system of record, while compute scales independently based on workload demands. This reflects the reality that AI depends on data being continuously stored, shared and reused across multiple applications and teams. Modern storage architectures are designed to efficiently manage massive datasets while ensuring they're available wherever AI workloads need them. For enterprises, this means rethinking storage as a strategic layer of AI infrastructure rather than an IT resource. Organizations that adopt flexible, software-defined architectures will be better equipped to scale AI while improving operational efficiency. CXOToday: As AI workloads become more data-intensive, how should organizations balance performance, scalability, cost and data protection? Jason: One of the biggest misconceptions is that AI infrastructure should be optimized for performance alone. In reality, organizations need to balance performance with scalability, economics and resilience as AI datasets continue to grow. The answer isn't about choosing a single storage technology. It's about designing architectures where different technologies work together. Flash or memory can absorb bursty data ingest and checkpointing, while high-capacity hard drives efficiently manage large-scale, sequential data workloads. Software orchestration should then intelligently place data where each storage technology delivers the most value. This allows organizations to balance performance, scalability and cost, rather than optimizing for just one objective. By taking this approach, organizations can improve resource utilization, control infrastructure costs, and build resilient storage environments that support AI growth over the long term. CXOToday: AI depends on data being continuously stored, accessed and reused. How should enterprises rethink their data infrastructure? Jason: The success of AI increasingly depends on how effectively organizations manage their data, not just how much of it they have. As datasets grow larger and are reused across training, inference and analytics, enterprises should focus on creating shared data environments that provide AI applications with secure access to a trusted, persistent data foundation rather than repeatedly copying datasets across multiple silos. Modern cloud architectures support this through approaches such as object immutability and versioned updates, where changes create new versions of data instead of modifying existing ones. This improves consistency while enabling multiple AI workloads to securely access the same trusted datasets. Combined with strong governance and the right mix of storage technologies, this approach reduces unnecessary data movement, improves operational efficiency, and enables organizations to scale AI without significantly increasing complexity or cost. CXOToday: India is rapidly expanding its AI capabilities, cloud infrastructure and data centre footprint. What should organizations prioritise next? Jason: India has many of the ingredients to become a global AI leader, including deep technical talent, growing investment, supportive policy initiatives and strong digital adoption. The next priority is ensuring that its data infrastructure evolves at the same pace as those ambitions. Much of the conversation today focuses on compute. Powerful GPUs are essential for AI. But compute is only one part of the equation. As organizations move AI from experimentation to production, they'll also need robust data infrastructure that can store, manage and continuously reuse vast amounts of data. Without that foundation, AI innovation becomes difficult to scale sustainably and deliver long-term business value. The goal shouldn't simply be adding more infrastructure, but building the right data infrastructure. Cloud-native, software-defined architectures allow compute and storage to scale independently. This improves infrastructure efficiency and helps organizations maximize the long-term value of their AI investments. CXOToday: How do you see AI, cloud and data infrastructure evolving over the next few years, and what changes do you expect enterprises to make in their storage strategies? Jason: AI, cloud, and data infrastructure will become even more closely integrated over the coming years. As AI models continue to evolve, organizations will increasingly build architectures where data remains persistent while compute becomes more elastic and scales according to workload demands. We're also likely to see broader adoption of software-defined storage architectures that intelligently manage data across different performance tiers and environments. Rather than maintaining separate storage silos, organizations will increasingly rely on shared data foundations that support AI, analytics, and business applications from the same trusted source. Ultimately, storage strategies will become less about managing hardware and more about building intelligent data architectures that maximize the value of data, improve efficiency, reduce bottlenecks, and enable AI to scale sustainably.
Share
Copy Link
As AI evolves from simple chatbots to autonomous agentic systems, the industry faces a critical infrastructure challenge that has nothing to do with compute power. The real bottleneck is AI storage—specifically, managing massive context memory and KV cache that agentic AI generates. Storage technology is receiving what industry experts call a 'promotion' as enterprises discover that even the most powerful GPUs become idle when starved of data.
The AI infrastructure conversation is undergoing a fundamental shift. While GPUs and specialized accelerators have dominated headlines, a more pressing challenge is emerging: AI storage capacity and architecture. As enterprises transition from experimental generative models to production-scale agentic AI systems, storage technology for AI is moving from afterthought to strategic priority
1
.The evolution from simple chatbots to autonomous agents that execute multi-step workflows has created exponential growth in data volume. According to Nicolas Frapard, senior manager at Western Digital, "As AI adoption expands to billions of interactions, data growth becomes structural rather than incidental"
1
. This structural shift carries direct economic consequences that enterprises can no longer ignore.
Source: The Register
Agentic AI introduces a critical new requirement: persistent context memory that allows models to maintain reasoning across sessions, users, and systems. Every interaction generates tokens that produce KV cache—the working memory enabling models to reason efficiently without constantly recomputing prior steps
2
. A simple 15-word prompt can generate as many as 40,000 tokens, representing five to 10 gigabytes of context data3
.Most AI infrastructure still treats this inference context memory as temporary, storing KV cache in GPU memory and discarding it when resources are exhausted. "That approach might be acceptable for small scale experimentation, but it quickly breaks down in enterprise environments where context lengths grow, concurrency increases, and recomputation becomes expensive," according to industry analysis
2
.The solution requires what experts describe as an inference context memory layer—a dedicated storage tier specifically designed to extend memory for AI systems. Ace Stryker, director of AI and ecosystem marketing at Solidigm, notes that "storage kind of got a promotion" with this third job beyond traditional GPU servers and shared environments
4
.
Source: SiliconANGLE
The economics of balancing compute and storage for AI are straightforward but often overlooked. Greg Matson, senior vice president at Solidigm, explains: "While the GPU is the most expensive part of your infrastructure, you want that thing to be humming 100% of the time generating tokens. And if it's down, right, waiting for data, then you're wasting your money on GPUs"
3
.This storage bottleneck in AI has prompted hyperscalers to replace legacy storage infrastructure—some more than a decade old—with high-capacity SSDs. Solidigm now offers drives reaching 122 terabytes of capacity per unit, alongside the industry's first cold-plate-cooled enterprise SSD designed for fully fanless Nvidia GPU servers
3
. The shift reflects a broader infrastructure transformation where liquid cooling replaces air-cooled systems entirely.Market catalysts include Nvidia's BlueField-4 STX storage architecture announced in March, which introduced Context Memory Storage (CMX)—a high-performance context layer expanding GPU memory across the rack using data processing units to handle traffic between GPUs and flash storage
4
.Related Stories
While high-performance SSDs capture attention for inference workloads, storage for agentic AI systems requires a tiered approach. Jason Feist, senior vice president at Seagate, emphasizes that "organizations need to balance performance with scalability, economics and resilience as AI datasets continue to grow"
5
.For enterprises operating at scale, HDDs remain the backbone of AI data storage architecture. Despite persistent narratives around all-flash environments, 87% of exabytes in large data centers are stored on hard drives
5
. On a properly measured cost-per-terabyte basis, flash can clock in at up to 20 times more expensive than HDD1
.The cost gap between flash and HDD was once expected to narrow, but instead has widened. "At production scale, only a small proportion of data requires high-speed access," notes Western Digital's Frapard. "The majority, such as logs, historical outputs, and training artifacts, should be stored reliably, accessed predictably, and retained economically over long periods. This is where HDDs becomes essential"
1
.Modern AI infrastructure demands software-defined, globally distributed object storage as the persistent system of record, while compute scales independently based on workload demands
5
. This architectural shift reflects AI's dependence on data being continuously stored, shared, and reused across multiple applications and teams.Emerging tools like context graphs—accumulated structures of decision traces woven among entities and time—are gaining recognition as trillion-dollar opportunities for making precedent searchable
4
. Companies like Neo4j are providing tools for generating full-stack applications with AI agents backed by graph databases for contextual memory.
Source: TechRadar
Industry experts speaking at RAISE Summit emphasized that petabyte-scale storage requirements will only intensify as long-term context windows expand and agentic sessions multiply across enterprise workforces
3
. Organizations that optimize AI infrastructure by treating storage as a strategic layer rather than commodity plumbing will be better positioned to maximize token generation efficiency and infrastructure ROI.Summarized by
Navi
[1]
[3]
[4]
19 Mar 2025•Technology

27 Sept 2024

17 Jan 2025•Technology

1
Technology

2
Policy and Regulation

3
Technology
