Google DeepMind releases DiffusionGemma for 4x faster local AI

Google DeepMind has released DiffusionGemma, a new member of the Gemma 4 open model family that generates text in parallel rather than one token at a time.

DiffusionGemma uses a parallel generation approach borrowed from image models. Unlike autoregressive models that produce text left to right one token at a time, DiffusionGemma starts with a field of placeholder tokens and runs over the canvas multiple times to generate likely tokens. It uses those to improve estimation of others before finalizing outputs in one large block of denoised text. Google says this makes the model faster and more efficient on local hardware such as an Nvidia DGX or a gaming GPU.

The model is a 26-billion-parameter Mixture of Experts design. Only 3.8 billion parameters activate during inference, allowing it to fit in the 18GB RAM of a high-end GPU. In testing on an RTX 5090, DiffusionGemma produces around 700 tokens per second. On a single Nvidia H100, it reaches 1,000-plus tokens per second, roughly four times the output of similarly sized autoregressive Gemma models.

Parallel generation shifts the performance bottleneck from memory to compute. The model can generate up to 256 tokens at once. Google reports measurable gains on non-linear tasks including in-line editing, molecular sequencing and mathematical graphing. An animation in the release demonstrates how DiffusionGemma solves Sudoku puzzles by continuously self-correcting large sets of tokens, a task that challenges standard autoregressive models because each token depends on future ones.

Text diffusion carries trade-offs that limit its use in cloud models. Google has experimented with the technique for its Gemini models but notes a higher error rate. In language, a single bad token can render an entire block meaningless and require restarting, unlike image generation where one flawed pixel rarely ruins the result. Diffusion models also waste resources on short outputs that autoregressive models can complete in just a few steps.

Local hardware benefits more from diffusion than cloud systems do. Cloud autoregressive models batch jobs across users and leverage high-bandwidth memory to stay efficient. Local AI often faces idle cycles and lower memory bandwidth. Diffusion makes better use of available compute, outperforming even Google's Multi-Token Prediction drafters that also target wasted cycles. Google describes DiffusionGemma as experimental.