OpenAI Jalapeño Chip Delivers 1.9x More Work Per Watt Than Nvidia GB300, Benchmarks Show

OpenAI's Jalapeño chip is built for fast inference at scale, benchmarks show

At the Hot Chips conference on Tuesday, OpenAI shared a more detailed look at Jalapeño, including the first batch of benchmark results for the new system. Tested on Semianalysis's InferenceX benchmark, Jalapeño registered both more tokens per user and more throughput per kilowatt than the currently available state-of-the-art inference processors. "The bottom line is that the results show a very, very significant performance advance over state of the art," said Richard Ho, OpenAI's head of hardware, in a press call. "Jalapeño can serve more AI work per unit of power, while also returning responses more quickly. It's very efficient to serve a lot of customers, but it can also be very low latency." Notably, that comparison is against an Nvidia Blackwell system -- but by the time Jalapeño reaches full deployment, the competition may have advanced significantly. Ho estimated that Jalapeño would deploy at the end of 2026 "in very small volumes," with more significant deployment coming in 2027. First announced last October, Jalapeño was developed by OpenAI in close collaboration with Broadcom, with OpenAI's own models assisting in the development process. The company plans to make Jalapeño a multigenerational platform, allowing AI products, models, chips and memory all developed in concert. Because of that full-stack approach, OpenAI was able to address specific phases in the inference process that often cause friction during inference processing. In particular, Jalapeño is designed to minimize delays during the prefill and communication phases of processing, which OpenAI says often act as bottlenecks. "We designed Jalapeño to minimize data movement and communication delays," the company said in a blog post presenting the results. "This means that model state, including the KV cache used while generating a response, can be explicitly placed and kept local while the system activates the right combination of compute, memory, and networking for each inference phase."

OpenAI says its Jalapeño chip can power faster AI responses than the competition

OpenAI says its new AI chip, Jalapeño, completes tasks more efficiently and returns responses faster than other AI systems, according to a blog post published on Tuesday. During a briefing with reporters, OpenAI hardware vice president Richard Ho said Jalapeño offers the "best of both worlds" with lower latency and higher throughput, as AI systems typically "have to make a trade-off between the two." First introduced in June, Jalapeño is an Application-Specific Integrated Circuit (ASIC) made in partnership with Broadcom. It's designed for AI inference -- the process of running a trained AI model to complete a task or deploy an agent. To measure Jalapeño's performance, OpenAI used InferenceX, a benchmarking platform that shows how well AI systems handle inference. The test compared Jalapeño's performance against the best results recorded at the time, which were with Nvidia's GB200 or GB300 superchips. OpenAI says Jalapeño delivered 1.5 to 1.9 times more AI work per watt across GPT-OSS 120B, DeepSeek R1, and Kimi K2.5 1T than the comparison systems, while offering 1.7 to 3.6 times lower end-to-end latency across the three models. That means the chip can provide users with "faster responses, more responsive agents, and more reliable access as the demand grows," according to Ho. OpenAI plans to deploy Jalapeño in "small volumes" by the end of this year, but will begin to "ramp the volume up" into 2027, Ho added. The company doesn't say how many chips it plans to deploy next year, however. Even with these performance improvements, Ho said OpenAI doesn't expect to replace its entire chip lineup with Jalapeño, saying its overall compute strategy includes "very good partners," like Nvidia. OpenAI will continue developing the second and third generations of the new chip.

OpenAI's upcoming Jalapeño chip looks like it'll be an inference beast

OpenAI offered its closest look yet at its spicy new Jalapeño AI accelerator at the annual Hot Chips semiconductor development conference at Stanford on Tuesday. The chips, first teased earlier this year, were developed in collaboration with Broadcom, and are the first in a series of custom silicon from OpenAI, designed (in part) by AI, for AI. Compared to contemporary GPU systems from Nvidia, OpenAI says the parts will deliver both higher throughput and lower latency when they start trickling out later this year and reach volume production in 2027. To be clear, Jalapeño won't replace OpenAI's long-time hardware partners, which also happen to be some of its most important investors. OpenAI still needs compute for training, and the highly programmable nature of GPUs means that OpenAI is likely to deploy on AMD and Nvidia first and then transition to in-house silicon later. It's also worth noting that AMD's MI455X and Nvidia's Rubin GPUs, also expected to ramp production in early 2027, are very different kinds of chips optimized for a mix of training and inference, whereas OpenAI's custom silicon only needs to excel at one job: inference. Memory bandwidth is king When it comes to inference, compute is key but memory bandwidth is king. And based on early benchmarks OpenAI shared with the press before its Hot Chips presentation Tuesday, the chip is shaping up to be an inference beast. Testing on SemiAnalysis' InferenceX benchmark suite -- presumably this is an unofficial test -- shows OpenAI's Jalapeño-based systems delivering between 1.5x and 1.9x more "AI work" at peak throughput, and 1.7x to 3.6x lower end-to-end latency than the competition across GPT-OSS-120B, DeepSeek R1, and Kimi K2.5. If the latter two seem like weird models for OpenAI to be testing against, it's not that OpenAI plans to use these chips to run competitors' models, it's just the models InferenceX uses. In any case, the test shows that Jalapeño isn't some model-specific architecture designed for maximum performance at the expense of programmability... cough, cough Taalas. Meanwhile, for ultra-low-latency inference, which has become the hot new segment for AI infrastructure providers, OpenAI says its chips are 2.1x to 4.1x faster. At a system level -- we're starting here because the frontier models OpenAI trains rarely run on a single chip any more -- each Jalapeño system with its 128 accelerators packs 1.7 exaFLOPS of 4-bit compute, 27.5 TB of HBM4 and just shy of 2 petabytes a second of memory bandwidth. By comparison, AMD and Nvidia's latest rack systems are faster, delivering 1.46x to 2x more compute and up to 12 percent more memory on Helios, but just 85 percent the memory bandwidth of OpenAI's rack. As of writing, OpenAI hasn't shared system-level power consumption, but based on what we know about the accelerators we'd wager each rack will use between 40 and 60 percent of the power of competing GPU systems. As usual, take all of these claims with a grain of salt. The competitive systems in question are Nvidia's GB200 NVL72 and GB300 NVL72 rack systems, which launched in 2024 and 2025 respectively. OpenAI's point of comparison also excluded speculative decoding, which uses a small draft model to boost inference performance by predicting the outputs of another. While it can be argued this makes for a cleaner apples-to-apples comparison, when you're measuring inference performance, you're not just comparing hardware but the platform as a whole. Joining the rack-scale revolution Jalapeño's system design won't surprise anyone. The chip is designed around a rack scale architecture similar to Nvidia's NVL72 or AMD Helios. Each rack is equipped with 128 Jalapeño accelerators on board, each of which, we're told, is capable of churning out 13.4 petaFLOPS at MXFP4. That compute is fed by 216 GB of HBM4 memory (we assume six 12-high stacks) good for 15.4 TB/s of memory bandwidth. OpenAI is expected to shed more light on the system and shipping during its Hot Chips presentation on Tuesday, but from what we gather, the chip also appears to feature a large SRAM cache. According to Richard Ho, VP of hardware at OpenAI, the chip was designed to minimize data movement and keep intermediate operations and things like key value caches, which track state, on the chip. "We designed Jalapeño to minimize data movement and communication delays. This means that model state, including the KV cache used while generating a response, can be explicitly placed and kept local while the system activates the right combination of compute, memory, and networking for each inference phase," the company explained in a blog post. A large SRAM cache would be the most intuitive way to go about this, and it's what we've seen the likes of SambaNova, dMatrix, and others do in their own AI XPU designs. However, unlike Nvidia's Groq LPX racks, Jalapeño isn't a one-trick pony. Instead, we're told, it's optimized for both the compute-heavy prefill operations where prompts are processed, and the memory-bandwidth-intensive decode phase where output tokens are generated. "A system that excels at one phase can lose that advantage while waiting for data or moving model state between different resources," the company explained. Designed with AI for AI Perhaps the least surprising detail in all of this is that OpenAI used AI to help design, architect, and optimize the chip for inference, cutting time from inception to tape out (the point at which a semiconductor design is finished and ready to be sent off to the foundry for fabrication) to just nine months. This included using its models to optimize its inference-serving engines and to write custom kernels as new models were introduced. But, as impressive as that might sound, AMD and Nvidia have been doing essentially the same thing internally for a while now. In fact, at Advancing AI last month, the House of Zen opened its ROCm.AI offering to the public. Moreover, building a chip that can scale to hundreds of thousands of accelerators isn't a trivial task, and AMD and Nvidia likely have the edge with regard to programmability, what with GPUs being general purpose accelerators. In any case, Jalapeño won't be the last chip we see from OpenAI, assuming of course the bubble doesn't pop before its second-gen chip tapes out. ®

OpenAI's Jalapeno chip outperformed the GB300 on power and speed, according to OpenAI

Jalapeno leads the GB300 on work per watt and response speed at 700 watts, though it was not tested against Vera Rubin and cannot train models at all OpenAI says its Jalapeno inference chip, developed with Broadcom, outperformed Nvidia's GB300 on AI work per unit of power and on response speed. It was not tested against Nvidia's newer Vera Rubin hardware and is not designed for training. OpenAI has published numbers for the chip it built to need Nvidia less. It says Jalapeno, developed with Broadcom, beat Nvidia's GB300 on two measures, the amount of AI work handled per unit of power and the speed of returning an answer. The company supplied its own caveats. Jalapeno was not tested against Vera Rubin, Nvidia's newer generation which has just started shipping, and it cannot train models at all. TNW flagged the obvious problem when the chip was unveiled. We wrote in June that vendor benchmarks deserve a raised eyebrow until independent numbers land, and these are still OpenAI's numbers. The power figure carries most of the weight. Jalapeno reaches its results at 700 watts, and electricity is the dominant running cost in any data centre. Chip chief Richard Ho says it does two jobs usually split between parts. High throughput to serve many customers cheaply, and low latency for those who care about response time. What it ran on is worth noting. OpenAI tested a small open model of its own plus third-party models from DeepSeek and Moonshot, with the widest advantage on Kimi, the largest one it tried. The roadmap is already moving. A second version tapes out in the coming months and a third generation exists in concept. Ho was careful about who is listening. "Nvidia is a really good partner, and we continue to need a lot of Nvidia," he said. Every large buyer is now doing this. Anthropic is designing its own silicon and Google has been talking to Marvell about custom inference chips. Europe is buying the merchant product. The EU has committed around €20bn to AI gigafactories that will purchase hardware on the open market, weeks after Nvidia told customers server prices are rising more than 15%. Its answer to this is smaller than the problem. Europe's most prominent AI chip company is Axelera, whose Samsung-backed accelerator targets edge devices rather than the data centres the gigafactory money is building.

OpenAI chip beats Nvidia systems with 1.9x more work per watt

OpenAI says its first custom inference chip, Jalapeño, can deliver more AI work per watt while also reducing response times, pointing to a hardware design aimed at handling increasingly demanding model workloads more efficiently. The company tested Jalapeño across three public models: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. Across the tests, the chip delivered 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison systems. For highly interactive workloads, OpenAI said Jalapeño achieved 2.1 to 4.1 times higher performance. The company argues that combining throughput and low latency in one architecture could help reduce the hardware and power needed to serve AI models. The chip is rated at 700 watts, although OpenAI said its measured sustained power remained at or below 550 watts during the tested workloads. The company compared Jalapeño with commercially available accelerator systems using the public InferenceX benchmark from SemiAnalysis. A major focus of the design is avoiding the usual tradeoff between throughput and latency. AI inference has different demands depending on what the system is doing. Processing a user's prompt, known as prefill, is generally compute-intensive, while generating the response token by token, known as decode, depends more heavily on memory bandwidth. Jalapeño was designed to handle both phases within the same architecture. OpenAI said the chip keeps model state, including the key-value cache used during generation, closer to the processing resources that need it. Its networking system is also integrated into the architecture to reduce the amount of data that needs to move between chips. That is intended to limit communication delays that can leave computing resources waiting for data. The approach becomes particularly relevant for AI agents, which may perform many inference steps in sequence. A small delay in each step can add up to a much longer overall task. OpenAI also used its own models during the chip's development. The company said AI helped engineers explore implementations, shorten design and verification cycles, and optimize arithmetic circuits. The team moved from initial design to tapeout in nine months. OpenAI then used Codex with GPT-Astra to bring three open-weight models that were not part of the original production plan to high performance on Jalapeño within two months. For selected GPT-OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5 to 1.8 times faster than existing implementations written by human experts. OpenAI stressed that these results applied to individual blocks rather than the complete models. The company also tested Jalapeño against large models including Kimi K2.5 1T. On that model, it reported about 1.5 times higher peak performance per watt and 3.4 times lower end-to-end latency than the comparison system. OpenAI plans to begin deploying Jalapeño in its compute infrastructure by the end of 2026. The company said the chip is the first generation of a multigenerational roadmap, with second- and third-generation designs already in development. The broader goal is to make inference faster and more power-efficient as OpenAI expands its computing infrastructure and serves increasingly capable models.

OpenAI says its Jalapeño chip bests Nvidia, others

Why it matters: It's part of a trend of AI model creators developing their own chips to cut costs, meet demand and reduce dependence on industry giant Nvidia. State of play: OpenAI said that the benchmarks show Jalapeño outperforming chips from Nvidia and others when running DeepSeek R1, the 1-trillion-parameter version of Kimi K2.5 and OpenAI's latest open source model. * OpenAI says it expects to have a limited number of Jalapeño-based systems in place this year, with greater capacity next year. Meanwhile, it says it is deep in development on a second-generation chip with plans starting to take shape on a third version. Yes, but: OpenAI's chip isn't designed for training new models, meaning the company is still highly dependent on Nvidia, among others. * "We're going to need a lot of compute," OpenAI VP of hardware Richard Ho said in an interview. "And Jalapeño is part of that. Cerberus is part of that. Nvidia is part of that. AMD is part of that." The big picture: Most of OpenAI's rivals either have or are said to be developing their own chip designs. Google has its TPU line, including chips for both training and inference, while Amazon and Microsoft also have their own AI chip designs * Anthropic confirmed earlier this month it is looking to build its own in-house chip design effort. Between the lines: Although OpenAI demonstrated Jalapeño's performance on open source models to compare it to rival chips, the company says it has no plans to offer the chip to others. * "We have so much need for it that we can't imagine when we would be able to [sell them to anyone else]," Ho said. What we're watching: How much of OpenAI's overall compute ends up coming from Jalapeño and its successors in the coming years.

OpenAI's First-Gen Jalapeno ASIC Blows Competition Out Of The Park, Performs 1.5x to 1.9x More Work Per Kilowatt Than NVIDIA's Blackwell Chips, While Threatening The CUDA Moat

OpenAI seems to have used its current spate of models that run on NVIDIA GPUs to design the Jalapeno chip, and paired it with its own Gluon kernel programming language, thereby daring to threaten NVIDIA's legendary CUDA moat. How the tables have turned! The architecture of OpenAI's Jalapeno chip SemiAnalysis has just provided a detailed breakdown of OpenAI's Jalapeno ASIC, describing a somewhat novel architecture that delivers substantial economies of scale. As such, the Jalapeno pairs a single reticle-sized compute die - built on TSMC's N3P node - with an N3E I/O chiplet and HBM4 memory (likely from Samsung), delivering 15.4 TB/s of memory bandwidth per package. The cores and memory are divided into matching "slices," giving each core slice a low-latency local view of its own assigned HBM slice, linked through a high-bandwidth collective network. Now, AI models do not typically process data line-by-line. Instead, they process billions of numbers organized in grids or matrices. As such, the matrix engine is the "heavy lifter" built solely to accelerate these grid-based math problems. Of course, traditional AI math uses standard 16-bit or 8-bit numbers. Jalapeno, however, uses MXFP numerical formats. For the benefit of those who might not be aware, MX formats further compress AI math data - down to 4-bit in MXFP4. This compression is done by grouping numbers together and making them share a single scaling factor (exponent), thereby slashing the amount of memory needed to store and move data. The Jalapeno's matrix engines feature a weight-stationary systolic array. In a standard CPU, the chip has to constantly read data from memory, perform a calculation, and write the result back to memory. This creates a bottleneck. A systolic array, on the other hand, works like a human heart that pumps blood: data flows into a grid of tightly connected processing cells, passing directly from one cell to the next one without constantly stopping to read/write to external memory. AI calculations typically involve multiplying incoming data (like your prompt) by fixed internal values (the model's weights). However, the Jalapeno's matrix engines are weight-stationary, which means that the chip loads the model weights into the processing grid once and and then locks them in place (stationary). So, by freezing the heavy AI weights in place (weight stationary), pumping data smoothly through the grid (systolic array), and compressing the numbers (MXFP), Jalapeño eliminates data-travel bottlenecks. Apart from matrix engines, OpenAI's Jalapeno chip also sports 64-bit scalar cores to run the control code, manage memory, and orchestrate the whole operation. These scalar cores bear Out-of-Order (OoO) execution pipelines, and come paired with an L1 cache. This prevents these cores from stalling while waiting for data. As such, the out-of-order scalar cores work ahead of the math engines: they stream data into specialized hardware queues so that when the rigid, in-order matrix and vector cores are ready, the numbers are already lined up waiting for them. Finally, the chip boasts of FP32/INT32 vector cores, which can apply one single instruction to an entire row of numbers simultaneously, stepping in for instances that require high-precision math, and where compression from matrix engines would ruin the results. This is useful for finalizing the probabilities of the next word (softmax layers) or normalizing data layers. Of course, OpenAI is not just building the Jalapeno chip but also a complete rack-scale solution that consists of: OpenAI's Jalapeno chip blows away competition As per the results shared by SemiAnalysis, the Jalapeno is by far the most efficient chip for inference workloads, managing to complete 1.5x to 1.9x more work per kilowatt than NVIDIA's Blackwell chips. Critically, it is not specifically optimized for one single family of models, and can seemingly run any model with ease. According to SemiAnalysis, the Jalapeno "delivers 13.4 PFLOPs of MXFP4 on a single reticle-sized compute die that is manufactured on TSMC's N3P. This compares to 17.5 PFLOPs of dense Rubin NVFP4 for a single Rubin compute die that is similar size and on the same node." What's more, while OpenAI has rated Jalapeno at a TDP of 700W, it's sustained power consumption typically stays at or below 550W during active AI workloads. Critically, the Jalapeno achieves this feat while using a Single-Token Prediction (STP) architecture, where the model takes a prompt, processes it, and predicts exactly one token (a word or word piece) at a time. This means that the ASIC did not rely on architectural tricks like multi-token prediction (MTP) or speculative decoding to inflate its performance numbers. Do note that speculative decoding adds around a 3x improvement on cost per token. As a final caveat, do note that the Jalapeno sports HBM4 while the GB200 and GB300 systems sport HBM3E. This does stack the deck slightly in favor of OpenAI's ASIC. Follow Wccftech on Google to get more of our news coverage in your feeds.