DiffusionGemma - Google's open-source experimental text diffusion model

What is DiffusionGemma?

DiffusionGemma is an experimental open-source text diffusion model released by Google DeepMind. Built upon the Gemma 4 architecture and Gemini Diffusion research, it employs a 26-bit MoE design and achieves a generation speed of over 1000 tokens per second on a single NVIDIA H100 GPU by denoising 256-token text blocks in parallel, approximately four times faster than traditional autoregressive models. The model supports bidirectional attention and real-time self-correction, and after quantization, it requires only 18GB of VRAM to run locally on consumer-grade GPUs.

DiffusionGemma's main functions

• Parallel text generationEach forward propagation simultaneously denoises 256 tokens, enabling parallel generation of the entire text block rather than word-by-word output.
• Two-way contextual reasoningDuring the generation process, each token can follow all other tokens within the same block, supporting non-linear text tasks such as code filling and inline editing.
• Real-time self-correctionWhen the confidence of a token decreases, the sampler can re-noise it and replace the correction in subsequent steps.
• Multimodal input processingSupports interleaved input of text, images, and videos, generates text output, and has a context window of up to 256K tokens.
• Long sequence block-level generationThe algorithm uses a block autoregressive diffusion mechanism to submit the denoised 256-token block to the KV cache and then continue to generate the next block, balancing parallel speed and sequence stability.

The technical principles of DiffusionGemma

• Uniform State DiffusionDiffusionGemma's core mechanism borrows from image diffusion models: starting with a 256-token canvas filled with random placeholders, it gradually locks high-confidence tokens through multiple denoising iterations. These locked tokens are then used as contextual clues to parse adjacent positions, ultimately converging the entire sequence into coherent text. Each forward propagation solidifies approximately 15–20 tokens, and parallel processing significantly improves throughput.
• Hardware bottleneck shiftTraditional autoregressive models are limited by memory bandwidth during local inference in single-user environments, resulting in significant idle GPU computing units. DiffusionGemma shifts the bottleneck from memory bandwidth to the computation side: by providing large-scale parallel denoising workloads for tensor cores, it enables GPUs to maintain high utilization in local inference scenarios, achieving 1000+ tokens/second on a single H100 and 700+ tokens/second on an RTX 5090.
• Hybrid attention architectureThe inference process alternates between two attention modes: the Prefill phase uses causal attention to process input cues and write them to a key-value cache; the Denoising phase switches to bidirectional attention, allowing each token in the canvas to simultaneously attend to all other tokens. This architecture enables the model to handle constraint-intensive tasks.

How to use DiffusionGemma

• vLLM local deploymentAfter launching the model using vLLM's OpenAI-compatible server and configuring the diffusion sampler and 256-token canvas parameters, inference services can be provided externally.
• Hugging Face download weightAccess the Hugging Face repository to obtain the model weight files, which are open-source under the Apache 2.0 license.
• Mainstream inference frameworks runningThe model natively supports direct loading and inference from frameworks such as Hugging Face Transformers, SGLang, and MLX.
• Hackable Diffusion Fine-tuningUsing Google's open-source JAX research toolkit and official training recipes, we can quickly experiment and fine-tune for specific tasks.

DiffusionGemma's core advantages

• Extreme local inference speedIn a single-user local scenario, it is about 4 times faster than an autoregressive model of the same specification. The H100 test results show 1000+ tokens/second, significantly reducing the latency of interactive development tools.
• Consumer-grade hardware can runAfter quantization, it only requires 18GB of VRAM, supports consumer-grade graphics cards such as RTX 4090/5090 and DGX Spark desktop workstations, and does not require a server-level cluster.
• Zero-day support for open source ecosystemIt is compatible with mainstream toolchains such as vLLM, Hugging Face Transformers, MLX, and Unsloth from the outset, reducing the cost of integration.
• Non-linear text task expertiseBidirectional attention enables it to outperform traditional autoregressive models on tasks requiring global constraints, such as code filling, structured output, amino acid sequences, and mathematical graphs.