Nvidia Vera CPU targets AI agent workloads with focus on single-threaded performance

Nvidia touts Vera CPU's single-threaded performance as its agentic AI advantage, reveals next-gen 'Rigel' Arm CPU cores -- frames chip as a 'max single-threaded CPU at scale,' not a parallel monster

AI agents, like humans, demand high single-threaded performance Only a little while back, Phoronix got the chance to test-drive one of Nvidia's upcoming Arm-based Vera CPUs. In certain approved workloads, the chip put up an impressive showing, nipping at the heels of its Xeon and Epyc x86 competitors. In specific single-threaded scenarios, Vera "absolutely dusted the competition" (our words). But AMD had some things to say about the Phoronix test, firing back with its own metrics of a 3.3x performance gain over Vera for the projected output of a 100 kW rack of its hardware. And Nvidia is already thinking about this future. It revealed that its next-gen Rigel Arm v9.2 CPU core, shipping as part of its Rosa CPU, will deliver even higher per-core performance than Vera's Olympus core within the same silicon footprint via "better instruction delivery," more L2 cache, and better memory handling. Now, Nvidia is reasserting Vera's advantage for AI work by describing it with a new product category: a "max single-threaded CPU at scale" rather than a parallel-processing beast. Instead of simply maximizing the core count per socket, Nvidia says Vera's monolithic 88-core design is meant to provide strong performance per core under load, enough memory bandwidth per core to keep active cores supplied with data, and predictable latency. Nvidia describes AI inference workloads as being bound by single-thread speed. For example, a reasoning AI will run the model for one step, and will run the model again as many times as it takes until the answer is generated. Since each step needs the output from the previous one, no amount of parallelism will help -- the speed at which one thread can run is most important. The situation is similar in agentic workloads, as agent B can't get its work started without knowing what happened with agent A. Vera's design, then, appears to be one aimed at both having and eating the proverbial cake: high single-thread speed with a large number of available threads. Vera is an 88-core design with SMT support for 176 total threads. And to supply each of those cores with adequate bandwidth, Nvidia says Vera talks to LPDDR5X RAM at 1.2 TB/s, and that its monolithic compute die keeps cores well fed and avoids bottlenecks thanks to 3.4 TB/s of core-to-core bandwidth. The company says the latter figure is 3x that of "any other data center CPU." There are many ways to measure inter-core bandwidth, so direct comparisons are tricky at best, but given the bespoke design of Vera for AI inference tasks, the claim is at least plausible. The company's latest blog post about the new silicon reiterates this point, claiming its new silicon delivers 1.8x higher performance versus its x86 competition in "loaded CPU workloads that represent agentic execution," 1.5x higher perf in coding workflows, and 3x faster work in database analytics. The numbers Nvidia touts purportedly come from real-world scenarios, starting with those from Perplexity, whose usage of Vera in coding agent work delivered a claimed 1.5x performance increase over x86, and a 1.9x speedup running concurrent sandboxes. The claimed speed increases are wider still in database workloads, with Starburst (federated database firm) clocking a 3x uplift in large-scale SQL analytics, while Redpanda's real-time analytics saw a claimed 6x latency drop. According to Nvidia, all this purported performance is delivered by Vera's particular architecture, one that aims to deliver maximal single-thread performance with high thread counts. We should note that vendor-approved benchmarks should always be taken with a bucket of salt, particularly those for hardware in a field that can shuffle trillions of dollars in a single day. The company doesn't say which precise x86 chips it tested Vera against, but it's a fair guess that they're mid- to high-end Intel Xeon and AMD Epyc models. Nevertheless, in the blog post, Nvidia describes a conundrum that's familiar to most any server administrator: big-iron server chips can pack obscene amounts of cores, making them ideal for processing many tasks at once. However, the more cores you add, the slower they need to be to keep thermal performance and power draw in check. But that scale is an obstacle for tasks that need to be done now, parallelization be darned. And the architectural decisions involved in using chiplets to scale to high core counts aren't free, either. Nvidia calls this "chiplet tax", and it says that scaling using chiplets creates memory access and performance inconsistencies that Vera's monolithic design is specifically meant to avoid. We've long emphasized the importance of high single-threaded performance for a fast and responsive experience for client PCs, and it seems like AI agents are going to end up placing similar demands on hardware as they do their thing. If that's how the agentic AI future plays out, Nvidia's particular design optimizations for Vera make greater sense than prioritizing core count above all, as it might be for a general-purpose server chip meant to satisfy different economic and customer demands. We'll have to see if Intel and AMD respond with "max single-threaded CPUs at scale" of their own. Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.

Perplexity says it plans to use Nvidia's new CPU

SAN FRANCISCO, July 7 (Reuters) - AI startup Perplexity on Tuesday confirmed it plans to use Nvidia's (NVDA.O), opens new tab new central processing units, as the chip giant works to broaden its market and take on entrenched players such as Intel (INTC.O), opens new tab and Advanced Micro Devices (AMD.O), opens new tab. Nvidia has said it expects to generate $20 billion in sales from its "Vera" CPU, a more generic computing chip than its AI-specific offerings, by the end of this fiscal year. ⁠The Vera chips are part of Nvidia's efforts to diversify sales as artificial intelligence companies such as OpenAI and DeepSeek make their own AI chips. Nvidia is entering a crowded market for CPUs long dominated by Intel and AMD, who supply CPUs for everything from laptops to web servers. But many of those chips were designed before the rise of what are known as AI "agents" that can carry out ⁠complex tasks on their own after receiving instructions from their human users. Unlike human users of CPUs, who take breaks between tasks, AI agents do not. Perplexity Vice President for Computer Enterprise and Infrastructure Nate Kupp ⁠said Nvidia's CPU carried out AI agent coding tasks about 1.5 times faster than traditional CPUs. "Vera really stood out to us as just like ⁠a dead-on fit for a lot of the core workloads that we have," Kupp said in an interview. Perplexity declined to disclose ⁠how many Nvidia CPUs it plans to buy. Nvidia has previously disclosed that OpenAI, Anthropic and Oracle plan to use its CPUs. Reporting by Stephen Nellis in San Francisco Editing by Bill Berkrot Our Standards: The Thomson Reuters Trust Principles., opens new tab

AI Innovators Adopt NVIDIA Vera -- Why Max Single-Threaded CPU at Scale Matters

Your browser doesn't support HTML5 video. Here is a link to the video instead. Max single-threaded CPUs at scale are a new category of CPUs built for the agentic AI era. Across the creation and deployment of an agentic system, the CPU is on the critical path for reasoning, response time and learning. CPUs are the processor which executes the work the AI model commands: the tool calling, code execution, data processing, KV-cache and result analysis. For agents in AI factories, speed matters. The faster the CPU can run the tool, the faster the agent can perform the task at hand. For the AI factory, the utilization of GPU is the most valuable resource in the data center so any time waiting for a task to complete constrains the revenue of an AI factory -- or worse, impacts the GPU utilization waiting for the CPU to finish its task. AI factories need a CPU with max single-threaded performance to maximize AI factory revenue and agent performance. Today's data center CPUs are not designed for speed at scale. While the world has fast CPUs for PCs and workstations, data center CPUs have been evolving in directions away from single-threaded performance. The advent of the cloud has pushed CPU makers to build higher core-count CPUs while minimizing cost at the expense of performance. Building CPUs that optimize costs per rentable core increased the number of cores per chip while taking away silicon area from what makes those cores run fast -- like high-performance memory fabrics and faster instruction processing per core. The move to chiplet architectures further reduced cost but created a "chiplet tax" where each CPU's cores can no longer can get access to the full memory performance of the chip. AI agents need a CPU designed for max single-threaded performance at scale. A max single-threaded CPU at scale keeps each agent step fast while the system is fully loaded. Every core completes the agent task at full performance without other cores slowing it down. Max single-threaded CPUs at scale are designed differently to deliver: * Strong performance per core under load * Enough memory bandwidth per core to keep active cores supplied with data * Predictable latency Every core can finish its task without any other core slowing it down, delivering excellent throughput and, more importantly, the fastest possible single-core task performance possible. NVIDIA Vera exemplifies this new class of CPU design. How Max Single-Threaded CPUs at Scale Are Built to Run the Agentic Loop An AI agent doesn't stop running after a single request. It acts in a loop. The model reasons about the next step. The CPU executes the work around the model. The result comes back. The model decides what to do next. Then the loop runs again. That pattern creates a demand profile for which conventional CPUs were not optimized. Traditional CPU work is intermittent and user-driven, made up of short interactions triggered by people. Agentic work is persistent and parallel: swarms of agents running continuously, each advancing through a chain of steps where each step depends on the result of the one before it. More cores in a CPU means more agent tasks per CPU, and data center CPUs need lots of cores to maximize throughput of tasks. However, adding more cores to a CPU cannot shorten the time for each step inside a single agent loop. More cores can't make any one task run faster. In fact, CPUs designed to maximize core count can even slow down the performance of each core as they contend for resources. Individual per-core performance matters to drive the speed of each step's completion. The throughput of additional cores is useful but insufficient. And since each action is dependent on the previous result, per-core speed determines how fast the loop advances. In the end, the best agentic CPU needs the best single-threaded performance per core, and every core needs to deliver that performance without compromise. The world counts in seconds. Agents count in nanoseconds. NVIDIA Vera is built for this new category -- and speed -- of work. NVIDIA Vera Is the Max Single-Threaded CPU at Scale for Agents NVIDIA Vera is a max single-threaded CPU at scale, designed from the ground up for the agent loop: the work that happens between model calls as agents use tools, process data, run code and check results. At the core of Vera is Olympus, NVIDIA's custom CPU core, which delivers 50% higher instructions per cycle than NVIDIA Grace. That matters because many agent steps are sequential. A tool call, code execution, test run or data-processing step must finish before the next model call can use the result. Faster cores move each loop forward faster. Vera pairs those faster cores with up to 1.2TB/s of LPDDR5X memory bandwidth at less than 40 watts of memory power, plus a monolithic compute die that helps active cores stay fed and keeps data movement predictable with 3.4TB/s of core-to-core bandwidth, 3x greater than any other data center CPU. This enables all 88 cores with the full memory performance of the CPU without creating bottlenecks that slows down every core. The result is faster agent loops. In loaded CPU workloads that represent agentic execution, Vera delivers 1.8x the sustained per-core performance of x86. Those gains compound across tool calls, code executions, data-processing steps and verification passes, helping AI factories complete more agent work with the GPUs they already operate. Perplexity tested Vera on the agentic work it runs every day. Running a real coding workflow -- cloning a repository and running its test suite in sandboxes -- Vera completed the job about 1.5x faster than x86, and started concurrent sandboxes up to 1.9x faster. Perplexity is now looking to deploy Vera in its upcoming production system. Agents also depend on data. They query, retrieve, filter and move information constantly, and Vera runs those CPU-side data workloads faster. Partners have measured 3x faster large-scale SQL analytics with Starburst and up to 6x lower latency on real-time streaming with Redpanda, both against leading x86 server CPUs. Agent work isn't one workload. An agent runs tools and sandboxes, processes data, serves requests and trains the next model with reinforcement learning -- and all of it leans on the same strengths. One Vera handles the whole range, rather than requiring a different CPU for each kind of work. And because Vera is the same CPU that hosts the GPUs in NVIDIA Vera Rubin and powers the NVIDIA BlueField-4 STX storage processor, the whole AI factory runs on one architecture and one toolchain. And NVIDIA's not done. NVIDIA's next-generation Rosa CPU with the Rigel core will continue the company's CPU roadmap for the agentic AI era. Rigel is NVIDIA's next-generation Arm v9.2 CPU core, delivering higher per-core performance than Olympus while keeping the same silicon footprint. Key improvements include better instruction delivery, a larger L2 cache and more efficient memory handling. Built for the Speed of Agents In the agentic AI era, there will be billions of agents, and every one of them will turn to a CPU to act, check, retrieve, execute and verify. In this new market, completed agent work is the product. Faster agent loops help every GPU spend more time generating revenue producing work and less time waiting. NVIDIA Vera is the CPU built for that future. Learn more about the NVIDIA Vera CPU.

Perplexity Bets on NVIDIA's Vera CPU, Calling The Max Single-Threaded Chip a "Dead-On" Fit After It Ran 1.5x Faster in Agentic Coding

NVIDIA's Vera CPUs are seeing increased demand as their single-threaded & inference-optimized design makes them ideal for firms such as Perplexity. Perplexity Bets on NVIDIA's Vera As The Chip Promises A Fully Inference-Optimized Architecture, Designed Purely For AI The Vera CPU is an ambitious chip for NVIDIA as it claims to generate $20 billion worth of revenue while becoming the leading CPU supplier this year. The chip is already in mass production & has landed at various firms such as OpenAI, xAI, Oracle and Anthropic. This increased demand for CPUs is driven by Agentic AI workloads, which are more reliant on CPUs than GPUs. Even the competition is focusing more on enhancing the inference capabilities of their upcoming chips, which is the trend going forward. AI firms are also developing their own custom CPUs to address in-house needs. "Vera really stood out to us as just like ⁠a dead-on fit for a lot of the core workloads that we have," Kupp said in an interview. Nate Kupp - VP at Perplexity (via Reuters) According to Reuters, Perplexity is the latest AI firm that has placed its bets on NVIDIA's Vera CPU. VP Nate Kupp says that NVIDIA's Vera CPUs were 1.5 times faster than traditional CPUs in Agentic AI coding tasks, making it a "dead-on fit" for the majority of their core workloads. Although the AI firm hasn't disclosed the number of CPUs or their value, it should be noted that Vera is landing at every major AI firm, which shows its successes already. NVIDIA Pitches Vera As The Only "Max Single-Threaded" CPU At Scale In its latest blog post, NVIDIA has also described Vera as the world's only Max Single-Threaded CPU at scale. The term "Max-Single-Threaded" CPUs delivers on three key fundamentals: * Strong performance per core under load * Enough memory bandwidth per core to keep active cores supplied with data * Predictable latency Agentic AI workloads run in a loop: as the CPU executes work, the results come back, and the model decides what to do next, and the same process repeats over and over. NVIDIA says that this pattern is something that conventional CPUs aren't optimized for. You can add more cores, but those extra cores cannot shorten the time for each step inside a single agent loop. CPUs with more cores can also hamper performance as each core contends for resources. This is why NVIDIA has bet on a smaller number of cores than the competition. Vera offers per-core optimizations for each core, as the throughput of additional cores, while useful, can be insufficient. In the end, the best agentic CPU needs the best single-threaded performance per core, and every core needs to deliver that performance without compromise. The world counts in seconds. Agents count in nanoseconds. NVIDIA Vera is built for this new category -- and speed -- of work. NVIDIA Vera brings 50% higher IPC with its custom Olympus cores than Grace. It is paired with 1.2 TB/s of LPDDR5X memory bandwidth at less than 40W memory power while its monolithic compute die keeps the active cores fed with 3.4 TB/s of core-to-core bandwidth, over 3x faster than any other CPU on the market. Vera pairs those faster cores with up to 1.2TB/s of LPDDR5X memory bandwidth at less than 40 watts of memory power, plus a monolithic compute die that helps active cores stay fed and keeps data movement predictable with 3.4TB/s of core-to-core bandwidth, 3x greater than any other data center CPU. This enables all 88 cores with the full memory performance of the CPU without creating bottlenecks that slows down every core. NVIDIA We have already mentioned how Perplexity was able to achieve a 50% uplift in its Agentic AI workflows versus traditional x86 CPUs. In concurrent sandboxes, this performance is up by 90%. Furthermore, NVIDIA partners are measuring 3x faster performance in large-scale SQL analytics with Starburst and up to 6x lower latency on real-time streaming with Redpanda, against x86 CPU offerings. With NVIDIA Vera aiming for success, the company is already highlighting its next-gen Data Center CPU, codenamed Rosa, featuring the updated Rigel core architecture. Rosa looks very impressive as a follow-up to Vera, and NVIDIA is on the path to tackle x86 competitors with a big adoption rate. Follow Wccftech on Google to get more of our news coverage in your feeds.

Perplexity to use Nvidia's new CPUs for AI agent tasks By Investing.com

Investing.com -- AI startup Perplexity said Tuesday it will use Nvidia's new central processing units as the chip maker expands into a market dominated by Intel and Advanced Micro Devices. Nvidia said it expects to generate $20 billion in sales from its Vera CPU by the end of this fiscal year. The Vera chips represent a shift toward more generic computing products as artificial intelligence companies like OpenAI and DeepSeek develop their own AI-specific chips. The CPU market has long been controlled by Intel and AMD, which supply processors for devices ranging from laptops to web servers. Many existing chips were designed before AI agents emerged that can complete complex tasks independently after receiving instructions from users. AI agents operate continuously without breaks between tasks, unlike human users. Nate Kupp, Perplexity's vice president for computer enterprise and infrastructure, said Nvidia's CPU performs AI agent coding tasks about 1.5 times faster than traditional CPUs. Perplexity did not say how many Nvidia CPUs it plans to purchase. Nvidia previously announced that OpenAI, Anthropic and Oracle will use its CPUs. This article was generated with the support of AI and reviewed by an editor. For more information see our T&C.