Google DeepMind's DiffusionGemma delivers 4x faster text generation with parallel processing

Google's latest DiffusionGemma open AI model comes with a 4x speed boost

Another day, another AI model from Google. This time, Google DeepMind has released a new member of the Gemma 4 open model family, but it's fundamentally different from the rest of the lineup. DiffusionGemma doesn't generate outputs linearly like most AI models. Instead, it can produce an entire block of text in parallel. Google says this makes it faster and more efficient when running on local hardware like an Nvidia DGX or a humble gaming GPU. Most AI models are designed to be autoregressive -- they generate text left to right one token at a time. DiffusionGemma has more in common with image generation models, which start with static and then denoise it to create the desired content. This model takes a field of placeholder tokens running over the canvas multiple times to generate likely tokens and using those to improve estimation of others. At the end of the process, the model finalizes its token outputs in one large block -- the "denoised" text canvas. DiffusionGemma is fairly large in the realm of Google's open models. It's a Mixture of Experts (MoE) model with a total of 26 billion parameters, but only 3.8 billion are activated during inference. That means it should fit in the 18GB ram allotment of a high-end GPU. In testing with an RTX 5090, DiffusionGemma spits out around 700 tokens per second. With a single Nvidia H100 AI accelerator, DiffusionGemma can produce 1,000+ tokens per second. That's about four times the output of the similarly sized autoregressive Gemma models. This approach to text generation shifts the bottleneck from memory bandwidth to compute, generating up to 256 tokens in parallel. Google says this offers a measurable boost in non-linear tasks like in-line editing, molecular sequencing, and mathematical graphing. The animation above shows how DiffusionGemma was tuned to solve Sudoku puzzles, which is a notoriously challenging task for standard autoregressive AI models because each token depends on future tokens. DiffusionGemma's ability to continuously self-correct large sets of tokens makes that easier. Multiple paths to local efficiency If diffusion is so much faster, why isn't Google using it in big cloud-based Gemini models? Google has experimented with this, but there are a few drawbacks to text diffusion, including a higher error rate. In image diffusion models, a single badly predicted pixel doesn't make the image useless, but language is discreet. An equivalent error in text can make a block of tokens meaningless and force you to start over to get a better output. Diffusion models also waste resources when the desired output is only a few tokens long. They have to do a lot more parallel work to whittle down to a few tokens that an autoregressive model does from beginning to end in just five steps. The efficiency gain for local processing makes this an appealing avenue of experimentation, though. In the cloud, autoregressive models can batch large numbers of compute jobs from multiple users so they're always churning out tokens, and the high bandwidth memory (HBM) used in these systems can move data around much more efficiently. Conversely, local AI encounters wasted compute cycles due to lower memory bandwidth and idle time. Diffusion models can make more efficient use of available compute, but this isn't the only way. Google also recently began implementing Multi-Token Prediction (MTP) drafters, which use otherwise wasted compute cycles to predict possible tokens to increase speed. But diffusion is even faster than the MTP versions of Gemma. Google stresses that DiffusionGemma is experimental, but it's available under the same Apache 2.0 license as all the other fourth-generation Gemma models. You can download the model weights today from Hugging Face. Google says it worked with Nvidia to ensure DiffusionGemma was optimized for a variety of setups, including high-end RTX GPUs (quantized) and enterprise systems like the H100 or DGX Spark platform.

NVIDIA Accelerates Google DeepMind's DiffusionGemma for Local AI

The new Diffusion Gemma open model generates text in parallel -- not one token at a time -- and is optimized to run on the NVIDIA RTX PRO platform, NVIDIA DGX Spark systems and GeForce RTX GPUs. Today, Google DeepMind released DiffusionGemma -- an experimental open model built for exceptionally fast text generation. NVIDIA has optimized DiffusionGemma to run even faster across NVIDIA GeForce RTX GPUs, the NVIDIA RTX PRO platform and NVIDIA DGX Spark systems, from local PCs to the cloud. Rather than generating text one word at a time, DiffusionGemma generates multiple words in parallel to output whole blocks of text, opening a new, low-latency frontier for the kind of single-user workloads that developers, researchers and AI enthusiasts run every day. Features of the new model include: * Parallel generation: DiffusionGemma denoises up to 256 tokens per step instead of predicting one at a time. * Built on Gemma 4: DiffusionGemma is built on Gemma 4, a 26-billion-parameter mixture-of-experts model that activates just 3.8 billion parameters per step, pairing a diffusion head with Google's Gemma 4 architecture. * Up to 4x faster performance: The boost means fast text generation, where single-user generation usually stalls -- at batch size 1, on local hardware. * Open and local: DiffusionGemma is open weights under a permissive Apache 2.0 license and runs entirely on RTX and DGX Spark -- no cloud, no per-token cost -- with day-zero support in Hugging Face Transformers, vLLM and Unsloth. A Different Way to Generate Text Almost every large language model (LLM) in wide use today is autoregressive -- meaning it generates text one token at a time, with each new word depending on the one before it. That sequential process is what makes interactive AI feel like it's typing. DiffusionGemma takes a different path. Built on the Gemma 4 26B mixture-of-experts architecture, it generates text the way diffusion models generate images: by starting from noise and refining a whole block of text at once. Each step denoises up to 256 tokens in parallel rather than emitting a single token and waiting to compute the next. The result is a model that thinks in blocks instead of sequentially. For latency-sensitive, single-user work -- such as interactive chat, agentic loops or on-device assistants that plan and act -- that parallelism translates into responses fast enough to keep pace with how developers think and iterate. DiffusionGemma Flies on NVIDIA GPUs Generating one token at a time is fundamentally a memory-bound problem -- at batch size 1, a traditional LLM spends most of its time waiting on memory bandwidth, not doing math, which leaves a lot of compute on the table. Diffusion flips the equation. Pulling a full 256-token block through the transformer in parallel is a compute-bound workload -- exactly what NVIDIA GPUs are built for. NVIDIA Tensor Cores accelerate the dense parallel math, and the CUDA software stack lets the model run efficiently from day one without bespoke tuning. In short, the model's design plays directly to the GPU''s strengths. That shows up in the numbers. DiffusionGemma delivers 1,000 tokens/sec at batch size 1 on a single NVIDIA H100 Tensor Core GPU, 150 tokens/sec on NVIDIA DGX Spark and fastest local inference on NVIDIA DGX Station -- roughly 4x faster than an equivalent autoregressive model running in the same single-user regime. That advantage holds across NVIDIA's full lineup, running: * Locally on the NVIDIA DGX Spark deskside personal AI supercomputer -- powered by the NVIDIA GB10 Grace Blackwell Superchip with 128GB of unified memory -- with the preinstalled NVIDIA AI software stack ready for prototyping, fine-tuning and fully local agent workflows. * On NVIDIA RTX PRO 6000 workstations, providing developers, researchers and AI professionals with the headroom to run local low-latency generation and agentic loops as part of a professional workflow. * On DGX Station, delivering best-in-class, high-speed inference at up to 800 tokens/sec for low-latency text generation and agentic loops with 748GB of coherent memory. * On GeForce RTX GPUs, with llama.cpp support coming soon. Get Started Locally The fastest way to start testing and prototyping the model is through Hugging Face Transformers, which runs DiffusionGemma on a GeForce RTX 5090 or DGX Spark out of the box. For higher-throughput inference, vLLM provides day-zero serving support. For adapting the model to a specific task or domain, fine-tuning is available through Unsloth and NVIDIA NeMo framework, with ready-made DGX Spark playbooks to get a local environment running quickly. Check out the vLLM playbooks for DGX Spark , RTX PRO and DGX Station. Try Diffusion Gemma on Hugging Face or test it for free using NVIDIA-hosted application programming interfaces at build.nvidia.com. Go deeper on the architecture and local deployment by reading the NVIDIA technical blog and the Google DeepMind announcement. #ICYMI: The Latest From RTX AI Garage 🎬 NVIDIA researchers released SANA-WM, an open source world model that turns a single image and a camera path into a minute-long, 720p video with precise 6-DoF control. At just 2.6 billion parameters, its distilled version generates a full 60-second clip in 34 seconds on a single NVIDIA GeForce RTX 5090 GPU using the NVFP4 format -- delivering up to 36x higher throughput than comparable open models while running on one GPU. Read the paper. 🛠️ Building Windows agents just got a full toolset -- NVIDIA and Microsoft rolled out turnkey agent sandboxing on native Windows -- Microsoft eXecution Containers plus the NVIDIA OpenShell runtime -- alongside up to 2x faster agentic inference and native Windows support for Hermes Agent. 🤖DGX Spark goes from unboxing to a running agent in minutes -- A streamlined NVIDIA NemoClaw install gets developers to a working local agent fast, with Qwen3.6-35B running up to 2.6x faster on vLLM. And the new cluster assistant in NVIDIA Sync links up to four DGX Spark units into one 512GB pool -- enough for ~400-billion-parameter models. Plug in to RTX Spark on Facebook, Instagram, TikTok and X -- and stay informed by subscribing to the RTX Spark newsletter.

Google's new DiffusionGemma model speeds up text generation by 4x

Google has unveiled DiffusionGemma, a new experimental AI model that generates text using diffusion rather than the autoregressive approach used by most large language models today. The company says the model can deliver up to four times faster text generation on dedicated GPUs while running on consumer hardware. The model builds on Google's Gemma 4 family and Gemini Diffusion research. Unlike traditional language models that generate text one token at a time from left to right, DiffusionGemma creates and refines blocks of text in parallel. According to Google, the approach enables output speeds exceeding 1,000 tokens per second on an NVIDIA H100 GPU and more than 700 tokens per second on an NVIDIA GeForce RTX 5090. The company says DiffusionGemma is aimed at developers working on speed-sensitive applications such as interactive editing, rapid content iteration, code infilling, and other workflows where low latency is more important than maximum output quality. Most large language models generate text sequentially, predicting one token after another. While effective, this process can leave local hardware underutilized when serving a single user. DiffusionGemma takes a different approach. Instead of generating text word by word, it creates a 256-token block at once and then repeatedly refines it through multiple passes. Google compares the difference to moving from a typewriter to a printing press. Rather than waiting for each token to be generated before producing the next one, the model processes an entire section of text simultaneously. The company says this shifts the bottleneck from memory bandwidth to compute performance, allowing modern GPUs to operate more efficiently during local inference. Another key feature is bi-directional attention. Since the model generates text in parallel, every token can attend to every other token during generation. This makes it better suited for tasks where future context matters, such as code completion, in-line editing, mathematical structures, and biological sequences. Google highlighted a demonstration in which DiffusionGemma was fine-tuned to solve Sudoku puzzles, a task that can be challenging for conventional autoregressive models because later tokens influence earlier decisions. The model uses a 26-billion-parameter mixture-of-experts architecture but activates only 3.8 billion parameters during inference. According to Google, this allows the model to fit within roughly 18 GB of VRAM when quantized, making it accessible on high-end consumer GPUs. DiffusionGemma also includes an iterative self-correction mechanism. Because it evaluates an entire text block during refinement, it can identify and fix mistakes as generation progresses. However, Google acknowledged that the model prioritizes speed over quality. The company said standard Gemma 4 models remain the preferred choice for production environments where output quality is the primary concern. The speed advantage is also most apparent in local deployments and low-concurrency environments. In cloud settings serving large numbers of users simultaneously, conventional autoregressive models can often utilize hardware efficiently through batching, reducing the benefits of diffusion-based generation. Google has released DiffusionGemma under an Apache 2.0 license through Hugging Face and is supporting deployment through tools including MLX, vLLM, Hugging Face Transformers, NVIDIA NeMo, and Unsloth.

Google's DiffusionGemma AI Hits 1,000 Tokens Per Second -- And It's Free

On NVIDIA NIM, the model arrived preconfigured at 8,192 tokens of context -- below the 64,000-token floor that agentic frameworks like Hermes Agent require -- meaning autonomous workflows won't run without manual reconfiguration. Google dropped DiffusionGemma today, an open model AI that generates text the way image generators create pictures: start with noise, refine until it makes sense. It hits 1,000 tokens per second on an NVIDIA H100. (Tokens are the basic unit of information that an AI model handles.) That means it's four times faster than regular Gemma. It's also free, Apache 2.0, with weights on Hugging Face. The catch, as always, is in the fine print. Per Google's announcement, the model hits "700+ tokens per second on NVIDIA GeForce RTX 5090." It also trails standard Gemma 4 on output quality. Google says so themselves. This is a speed model, not a quality upgrade. What this actually does Every LLM you've used is a typewriter. One token at a time with each word dependent on the last. That's how autoregressive architectures work. DiffusionGemma doesn't do that. Instead of generating tokens sequentially, it starts with refined chunks of garbled text in parallel. Per Google's developer guide, it "starts with a canvas of random placeholder tokens" and iteratively locks in confident tokens until the whole block snaps into focus. Two hundred fifty-six tokens per forward pass. The GPU stays busy. The side effect is bidirectional attention -- every token can see every other token while being generated, which is impossible in autoregressive models (they cannot see the future, what is going to be encoded). That makes it unusually good at tasks where the end of the answer constrains the beginning: code infilling, structured output, constraint-heavy problems, etc. Google fine-tuned a version to solve Sudoku as a demo. The base model got roughly 0% of puzzles right. The fine-tuned version hit 80%. Text diffusion has been a research project for years. MDLM, SEDD, LLaDA, Dream -- academic models that proved the approach worked at small scales and mostly stayed as proof of concepts. Inception Labs shipped Mercury 2 in February 2026 as the first commercial diffusion reasoning model, claiming speeds five times faster than speed-optimized competitors. But none of that was open-weight, and none of it came with day-zero support in vLLM, Hugging Face Transformers, and Unsloth. DiffusionGemma is the first major open release from a tier-one lab. There's also a historical irony worth noting. Image generators started as diffusion models (hence the name Stable Diffusion) and are now moving toward autoregressive architectures for better quality. Language models started as autoregressive and are now experimenting with diffusion for speed. Why it's a pain to run... for now Running DiffusionGemma efficiently requires a drafter -- a lightweight module that proposes token blocks in parallel, which the main model then verifies in one forward pass. This is called speculative decoding. DFlash is a framework published in early 2026 that uses a small diffusion model as the drafter, enabling over 6x speedup on some tasks. It's the engine that makes this class of model practical. The problem: DiffusionGemma needs a specific drafter to run locally via MLX -- Apple's machine learning framework for Apple Silicon. That module doesn't exist in any public version of mlx-lm, in any open pull request, or in LM Studio's bundled runtime. We tried running DiffusionGemma with Hermes through NVIDIA NIM. The model loaded, but then: "agent init failed: Model google/diffusiongemma-26b-a4b-it has a context window of 8,192 tokens, which is below the minimum 64,000 required by Hermes Agent." To be precise: DiffusionGemma's actual context window is 256K tokens. The 8,192 figure was Nvidia messing things up by default, not the model's architectural limit. In practice, getting it configured correctly for agentic use requires manual work that most everyday users haven't figured out yet, and Hermes Agent simply won't initialize without it. Parallel speed means nothing if the agent can't boot. Hopefully, in the next few days, the community will produce better resources to run these models. Who this is actually for Developers with NVIDIA RTX 4090 or 5090 hardware building real-time tools -- inline editors, autocomplete, code infilling, structured generation. That's the target. As Decrypt covered in May, Google has been on a steady push to make local inference faster without new hardware. For researchers, bidirectional generation opens territory that autoregressive models simply can't reach -- protein sequences, mathematical graphs, anything where position N depends on position N+50. That's not a small thing. Google launched Gemma 4 under Apache 2.0 in April, and DiffusionGemma continues that strategy. There's already a draft llama.cpp PR open as of today. When the toolchain catches up, this reaches a much wider audience. On a machine with a capable discrete GPU, 1,000 tokens per second is real.

DiffusionGemma: 4x faster text generation

You can improve DiffusionGemma's performance on specific tasks through fine-tuning. In the example below, Unsloth fine-tuned DiffusionGemma to play Sudoku -- a task autoregressive models struggle with because each token depends on future tokens. DiffusionGemma's bi-directional attention makes this much easier. While the AI research community has explored diffusion-based text generation for years, applying it to large models has remained a challenge. DiffusionGemma changes this by shifting how models use hardware. The trade-off with traditional models Most language models act like a typewriter, generating one token at a time from left to right. In the cloud, this is efficient because servers can batch thousands of user requests together to share the hardware load. But when run locally for a single user, this word-by-word process leaves your dedicated GPU or TPU underutilized -- it spends most of its time simply waiting for the next "keystroke." DiffusionGemma reverses this inefficiency. Instead of predicting words sequentially, it drafts an entire 256-token paragraph simultaneously. By giving the computer's processor a larger chunk of work at once, DiffusionGemma utilizes your hardware to its full potential. It upgrades your model inference from a single, sequential typewriter to a massive printing press that stamps the entire block of text simultaneously.

Google open-sources speedy DiffusionGemma text diffusion model

Google open-sources speedy DiffusionGemma text diffusion model Google LLC today released DiffusionGemma, a large language model based on an emerging machine learning approach known as text diffusion. The company says that the algorithm can generate text four times faster than traditional LLMs. Furthermore, DiffusionGemma does so using less RAM. The model's memory efficiency enables it to run on high-end consumer graphics cards that usually struggle to support LLMs. DiffusionGemma's text diffusion architecture is derived from a method that AI models use to generate images. The image generation workflow begins with a blurry photo that contains a type of error called Gaussian noise. An AI model removes a small portion of the noise, analyzes the enhanced photo and uses its findings to restore another batch of pixels. It then repeats the process until arriving at a usable image. When DiffusionGemma receives a prompt, it generates a placeholder response that comprises random words. It then replaces a subset of the random text with words that will form part of its answer to the user's prompt. DiffusionGemma reviews the edits, generates a few more words and repeats the process until its prompt response is ready. AI models usually generate prompt responses one token at a time. DiffusionGemma's text diffusion architecture, by contrast, enables it to produce 256 tokens at once. That parallelization is what makes the model faster than standard LLMs. Google says that DiffusionGemma can generate more than 1,000 tokens per second when running on a single H100, a server-grade GPU that Nvidia Corp. launched in 2022. The model can generate over 700 tokens per second on the chipmaker's desktop-grade GeForce RTX 5090 chip. One reason DiffusionGemma can run on consumer GPUs is that it's based on a mixture of experts architecture. The model includes 26 billion parameters but activates only 3.8 billion of them to answer the prompt, which lowers memory usage. DiffusionGemma further lowers RAM consumption by keeping information in a lightweight data format called NVFP4. DiffusionGemma is based on an LLM called Gemma 4 26B A4B that Google released in April. To facilitate text diffusion, the search giant replaced the latter model's attention mechanism, the software module it uses to interpret prompts. The original mechanism inferred the meaning of each word in a prompt by analyzing the preceding text. The new attention module also reviews the text that follows a given word. "While the AI research community has explored diffusion-based text generation for years, applying it to large models has remained a challenge," Google research scientists Brendan O'Donoghue and Sebastian Flennerhag wrote in a blog post today. "DiffusionGemma changes this by shifting how models use hardware." DiffusionGemma is available on Hugging Face under an open-source license.

NVIDIA Delivers Day-1 Support For DeepMind's DiffusionGemma Open Model Across RTX & DGX Platforms, 150 Tokens/s With DGX Spark

NVIDIA's entire RTX/DGX lineups are getting full support for Google DeepMind's DiffusionGemma Open AI model. Google Intros Its Newest Open AI Model: DiffusionGemma - NVIDIA Offers Full Support Across Its DGX & RTX Families The DiffusionGemma model is an open model designed to offer speedy text generation, and with its launch, NVIDIA is announcing support across its RTX and DGX lineups. What's even better is that while DiffusionGemma is fast, NVIDIA's optimizations for the model and its hardware make it even faster. The following are the main highlights of the model: * Parallel generation: DiffusionGemma denoises up to 256 tokens per step instead of predicting one at a time. * Built on Gemma 4: DiffusionGemma is built on Gemma 4, a 26-billion-parameter mixture-of-experts model that activates just 3.8 billion parameters per step, pairing a diffusion head with Google's Gemma 4 architecture. * Up to 4x faster performance: The boost means fast text generation, where single-user generation usually stalls -- on local hardware. * Open and local: DiffusionGemma is open-weight under a permissive Apache 2.0 license and runs entirely on RTX and DGX Spark -- no cloud, no per-token cost -- with day-zero support in Hugging Face Transformers, vLLM and Unsloth. On NVIDIA's side, they are offering day-1 support across GeForce RTX GPUs, RTX PRO Platforms, and DGX systems ranging from Spark Mini PCs to workstations powered by their datacenter-grade chips. NVIDIA is utilizing its tensor core architecture and the CUDA software stack, offering robust support that requires no additional tuning. NVIDIA has shared some stats too. The company states that its H100 Tensor Core GPUs on DGX Stations offer 1000 tokens/s (single GPU), DGX Spark systems offer 150 tokens/s, and DGX Station offers the fastest in-class local inference. The solutions offer roughly 4 times faster performance than an equivalent autoregressive model. * Locally on the NVIDIA DGX Spark deskside personal AI supercomputer -- powered by the NVIDIA GB10 Grace Blackwell Superchip with 128GB of unified memory -- with the preinstalled NVIDIA AI software stack ready for prototyping, fine-tuning and fully local agent workflows. * On NVIDIA RTX PRO 6000 workstations, providing developers, researchers, and AI professionals with the headroom to run local low-latency generation and agentic loops as part of a professional workflow. * On DGX Station, delivering best-in-class, high-speed inference at up to 800 tokens/sec for low-latency text generation and agentic loops with 748GB of coherent memory. * On GeForce RTX GPUs, with llama.cpp support coming soon. Users who want to try out the DiffusionGemma model out of the box can do so right now on an RTX 5090 or DGX Spark system. NVIDIA offers a full-stack and ready-to-use framework to try out the model right now. Follow Wccftech on Google to get more of our news coverage in your feeds.