𝗚𝗼𝗼𝗴𝗹𝗲 AI 𝗷𝘂𝘀𝘁 𝗿𝗲𝗹𝗲𝗮𝘀𝗲𝗱 𝗗𝗶𝗳𝗳𝘂𝘀𝗶𝗼𝗻𝗚𝗲𝗺𝗺𝗮 — 𝗮𝗻 𝗼𝗽𝗲𝗻 𝗺𝗼𝗱𝗲𝗹 𝘁𝗵𝗮𝘁 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗲𝘀 𝘁𝗲𝘅𝘁 𝗶𝗻 𝗽𝗮𝗿𝗮𝗹𝗹𝗲𝗹, 𝗻𝗼𝘁 𝘁𝗼𝗸𝗲𝗻-𝗯𝘆-𝘁𝗼𝗸𝗲𝗻.

Most LLMs today are autoregressive — one token at a time, left to right. DiffusionGemma takes a different path, it replaces token-by-token autoregression with discrete diffusion. Here is how it works:

𝟭. 𝗠𝗼𝗱𝗲𝗹 → 26B Mixture-of-Experts on the Gemma 4 backbone (25.2B total, 3.8B active). → 8 active experts of 128, plus 1 shared. 30 layers, 256K context.

𝟮. 𝗗𝗲𝗰𝗼𝗱𝗶𝗻𝗴 → It denoises a 256-token canvas in parallel, not one token at a time. → Roughly 15–20 tokens are finalized per forward pass. → Google calls the mechanism Uniform State Diffusion.

𝟯. 𝗔𝘁𝘁𝗲𝗻𝘁𝗶𝗼𝗻 → Prefill uses causal attention to ingest the prompt and write the KV cache. → Denoising uses bidirectional attention, so every canvas token attends to all others.

𝟰. 𝗟𝗼𝗻𝗴 𝘀𝗲𝗾𝘂𝗲𝗻𝗰𝗲𝘀 → Block Autoregressive Diffusion commits a finished 256-token block to the KV cache. → A fresh canvas then initializes, conditioned on prior history.

𝟱. 𝗦𝗮𝗺𝗽𝗹𝗶𝗻𝗴 → Entropy-Bounded Denoising with adaptive stopping, max 48 denoising steps. → Low-confidence tokens are re-noised and refined — a self-correction path autoregressive models lack.

𝟲. 𝗣𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝗮𝗻𝗱 𝗳𝗼𝗼𝘁𝗽𝗿𝗶𝗻𝘁 → Up to 4x faster on dedicated GPUs: 1000+ tokens/sec on H100, 700+ on RTX 5090. → Fits in 18GB VRAM when quantized. Native NVFP4 support.

𝟳. 𝗟𝗶𝗺𝗶𝘁𝗮𝘁𝗶𝗼𝗻𝘀 → Output quality is below standard Gemma 4; Google recommends Gemma 4 for production. → The speedup applies to local, low-concurrency inference, not high-QPS cloud serving.

Full breakdown with the comparison table: https://www.marktechpost.com/2026/06/10/google-ai-releases-diffusiongemma-a-26b-moe-open-model-using-text-diffusion-for-up-to-4x-faster-generation/

Model weight on HF: https://huggingface.co/google/diffusiongemma-26B-A4B-it

Technical details: https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/