Google retrofits Gemma 4 into a text diffusion model on under 10% of the original token budget
Google DeepMind converted Gemma-4-26B-A4B into DiffusionGemma, which refines 256-token blocks in parallel at roughly 1,500 tokens per second on an H100, using less than a tenth of the original training tokens. The retrofit path — not the model — is the result: Apache 2.0 weights on Hugging Face make diffusion experiments cheap for anyone with an existing checkpoint, though quality trails autoregressive Gemma 4 and the throughput edge vanishes near 32 concurrent requests.
Source: the-decoder.com ↗
DiffusionGemma develops the answer and reasoning in parallel, so it can fix mistakes before the output is finalized.
Google DeepMind technical report
Why this matters
- → Diffusion models can retrofit existing checkpoints, avoiding billion-token retraining.
- → Bidirectional refinement fixes reasoning errors before output, unlike autoregressive one-pass models.
- → Speed advantage evaporates under load—single-user gains don't scale to production workloads.
Retrofit, not retrain