Weka launches NeuralMesh 6 to cache 100% of tokens and solve GPU memory bottlenecks in AI inference

Weka's new storage platform caches 100% of a model's pre-calculated tokens, so it never has to redo the work

GPU memory is the most expensive resource in production AI, and it's also the one running out fastest. Long context windows and multi-turn conversations force AI models to repeatedly recompute information they've already processed, consuming GPU memory and compute that could otherwise serve additional users or generate new responses. Instead of treating GPU memory as the limiting resource, why not extend it with much cheaper storage technologies? Weka, for one, believes that cheap flash storage can close that gap. The company's NeuralMesh 6 software platform, launching alongside its first self-designed hardware line, Wekapod 3, extends what Weka calls Augmented Memory Grid, an approach that aggregates NAND flash to behave like GPU memory at a fraction of the cost. This is an active and increasingly crowded category. Dell, NetApp, Pure Storage and VAST have all repositioned toward AI infrastructure over the past two years and Weka is one of several vendors arguing it's built for this specific moment rather than adapting to it. "What we're seeing now with customers is they're chasing availability of compute, and once they get new allocation from anyone, they want to be able to grab it and start running right away," Weka co-founder and CEO Liran Zvibel, told VentureBeat. The potential payoff is straightforward: better utilization of existing GPU investments, lower inference costs and faster deployment of new AI workloads without waiting months for additional GPU capacity. Inside Weka's NeuralMesh 6 NeuralMesh 6 adds four capabilities aimed directly at a functionality gap Zvibel says has been costing Weka deals in competitive evaluations. Composable and virtual multi-tenancy. Composable clusters give anchor tenants full hardware-level isolation, dedicated CPU, memory, and storage. Virtual multi-tenancy runs through Weka's RDMA fabric, delivering network-level isolation that scales past 1,000 tenants per cluster, with provisioning in under 30 minutes. Combined, a single cluster running 50 composable clusters can support up to 50,000 tenants. Unified file and object storage. Most storage systems keep two separate paths: a file-based path (the standard way servers and applications read and write files, used heavily in training and fine-tuning pipelines) and an object-based path (S3, the format inference and cloud-native tools typically expect). Normally a gateway translates between the two, meaning the data effectively exists twice. Weka's claim is that the same physical data on disk is directly readable through either path at once, no translation layer, no second copy. Zvibel is targeting non-AWS GPU clouds specifically, naming Lambda, Nebius, G42, and CoreWeave, with what he described as roughly two orders of magnitude higher performance than conventional S3 and a capacity-based pricing model instead of per-API charges. Metadata-first replication. Destination environments become browsable before a full data copy arrives, with data hydrating only when accessed. "They had to wait for all of that to make it to the other side, and this takes days or weeks, in extreme cases a month," Zvibel said. "We now allow our customers to grab some allocation of new GPUs and get up and running within an hour." AlloyFlash and Always-On data reduction. TLC and QLC are two types of NAND flash memory. TLC is faster and more durable but costs more per terabyte, while QLC is cheaper and holds more data per chip but is slower. AlloyFlash mixes both within a single cluster, automatically routing latency-sensitive work to TLC while running bulk-capacity workloads on QLC, cutting cost per terabyte without a performance penalty on the work that needs speed. Data reduction now runs by default rather than as an option. Solving AI's context problem Multi-tenancy and object storage solve how enterprises and neo clouds operate the platform day to day. A harder problem sits underneath: as context windows and multi-turn interactions grow, so does the GPU compute wasted recalculating work a model has already done. Augmented Memory Grid, a NeuralMesh 6 feature built specifically for this, is Weka's answer. Every prompt triggers two stages. Prefill calculates attention, the core mechanism behind how large language models process input, and it's computationally expensive. Decode converts that calculation into output and is comparatively lightweight. The cost shows up hardest in multi-turn sessions like chat or coding, where each new turn re-triggers prefill for everything that came before it, unless that work has been cached. "If you have 10 turns, you may overcalculate 100 times because you're redoing all of them. If you have 20, you'll overcalculate 400 times," Zvibel said. "You can put two orders of magnitude more NAND than you could afford in shared memory, and we can cache 100% of the pre-calculated tokens, so you never need to redo it." Where Weka sits competitively Storage vendors have spent the past year and a half repositioning around AI, and separating genuine capability from repositioned messaging is now a real evaluation problem for buyers. "The storage world is shifting its focus from serving bits to enterprise workloads to managing data at the speed of AI. We've seen that most clearly over the past 18 months from Dell, NetApp, and Pure," Steve McDowell, chief analyst at NAND Research, told VentureBeat. "The interesting thing is that companies like Weka, and VAST, are the true AI-native data companies, solving these problems since day one." McDowell singled out Augmented Memory Grid as Weka's clearest technical lead. "Weka continues to have the most technically capable KV cache implementation on the market with its Augmented Memory Grid," he said. " They were early with this technology, and continue to innovate. This is critical for AI inference, as it enables a level of GPU efficiency that, without question, saves money on GPUs and memory. That's key for today's memory and GPU constrained market." He also flagged Weka's contractual guarantee on its data reduction claims as underappreciated. "One flying a little under the radar: Weka is putting its money where its mouth is with its contractual guarantees for its data reduction promises," he said. McDowell's advice to buyers evaluating competing claims from Weka, VAST, Pure and NetApp alike was pointed suggesting that enterprise buyers should look hard at what vendors are promising versus what they're actually delivering. "A smart buyer will look at how competing vendors are solving real-world problems today," McDowell said. " They do this by talking to organizations running similar workloads at similar scale. If a vendor can't point to that, then it should be a warning sign."

WekaIO revamps its AI data storage platform and unveils its first hardware for agentic workloads

WekaIO Inc. today announced what it says is its most significant product refresh to date. The distributed big-data management platform provider unveiled a massive overhaul of its flagship software together with the release of its first custom-designed storage hardware that's built specifically for artificial intelligence inference. The two-pronged announcement saw the company launch WEKA NeuralMesh 6, the latest iteration of its software that's designed to eliminate the storage and memory bottlenecks holding back AI inference workloads. It's designed to run on the new WEKApod Nitro, WEKApod Prime and WEKApod Prime Max AI appliances that promise to deliver the perfect combination of capacity and performance for running interference workloads at scale with a reduced footprint and energy consumption. According to WekaIO, it really could not have timed the release of its new software and systems any better. It says enterprise AI is reaching a critical inflection point as more organizations shift away from training AI models to running them in production inference environments. The startup believes that the AI training-focused storage infrastructures of yesterday are quite unsuitable for the long-context reasoning, agentic workflows and retrieval-augmented generation workloads that are now the main focus of most organization's AI efforts. Simply put, most businesses have already trained their AI models and now they want to see what they can do, and that requires a major overhaul of the underlying architecture that powers them. Organizations are also facing some major physical constraints. The available space in physical data centers is shrinking fast, and energy grids are struggling to keep up with the power demands of these facilities. There's no easy way to get around hardware shortages, other than to try to maximize the resources that you do have, and that's what WekaIO is trying to do. With NeuralMesh 6, WekaIO is introducing its powerful new augmented memory grid technology, which can extend the memory of graphics processing units by ensuring that the critical key-value cache is located within the NVMe storage. In this way, it's able to bypass the recalculation overheads that silently drain GPUs running inference workloads. The company pointed to some nice benchmark results, saying that NeuralMesh 6 with NVMe delivered 10 times higher token throughout and served 10 times more concurrent users on Oracle Cloud Infrastructure than standard dynamic random-access memory. The software also brings native hyperscale multitenancy, a unified file-and-object S3 protocol stack running directly on NVMe, and always-on data reduction with performance guarantees. Chief Product Officer Ajay Singh said most data centers that are running production AI workloads today were built in a chaotic way, mixing and matching different platforms, chips and networking technologies from multiple vendors. Basically, they just grabbed whatever hardware was available. But the result is that you end up with separate stacks for file and object storage, which means using manual processes to move data between them. "NeuralMesh 6 delivers what they've actually needed all along: a single platform that handles the high-performance file layer and the high-capacity object layer on the same blocks, with native multitenancy, intelligent data mobility and always-on data efficiency built in from the start," Singh said. "This is what production inference infrastructure looks like when it's designed for the workload, not retrofitted for it." On the hardware side, WekaIO is moving away from general-purpose servers and engineering its own platforms from the ground up. The new WEKApod systems are built with a PCIe Gen 6 internal fabric and feature a software-management thermal architecture to enhance both performance and capacity. The new lineup is led by the WEKApod Prime and WEKApod PRIME Max arrays, which come with up to 245 terabytes of ultra-dense solid-state drives and utilize NeuralMesh's proprietary data reduction techniques to create as much as 1.1 exabytes of effective capacity in a single 56-unit rack. There's also a lower-cost system in WEKApod Nitro, which is aimed at high-concurrency environments where storage bandwidth determines GPU utilization. Co-founder and Chief Executive Liran Zvibel insisted that WekaIO is, and will always be a software company first, adding that NeuralMesh is still its flagship product. "Building our own hardware was not the original plan," he said. "Our customers' inference economics made it a necessity. We built WEKApod because the alternative was letting someone else's hardware define the limits of what our software could do."