Masked diffusion method cuts LLM generation time to 3.4 seconds on H200
T-Tech and Higher School of Economics researchers presented a masked diffusion method at ICML that lets language models revise early-step errors without hundreds of refinement steps, cutting generation time from 9.2 to 3.4 seconds.
A masked diffusion method from T-Tech and Higher School of Economics researchers fixes a long-standing problem: once a token unmasks in standard masked diffusion LLMs, it freezes, and early mistakes propagate through the rest of the sequence. Existing re-masking approaches let the model reconsider its choices, but they require hundreds of steps and lose the speed advantage diffusion is supposed to deliver.
The new technique, presented at ICML, makes each noised state depend only on the clean text prediction, not on prior states. At every step the model predicts a full clean sequence, then the next state is sampled fresh from that prediction—any token can be re-masked and rewritten without additional training. The method can't generate from scratch this way because early-step re-masking destroys incomplete context, so the first ~70 percent of steps write a draft conventionally, then the revision mode kicks in. A lightweight error predictor, trained on the model's own generation mistakes, points out where to revise. The main limitation: no token insertion or deletion.
At 128 steps the approach beats the previous best 512-step method on text quality and diversity, generating nearly three times faster—3.4 seconds versus 9.2 seconds on an H200. The error predictor trained on web text transfers to code, math, and news domains without retraining.
