FORS sampler breaks polynomial error barrier, wins ICML 2026 Outstanding Paper

First-Order Rejection Sampling (FORS), a training-free method for high-accuracy sampling in diffusion models, won an ICML 2026 Outstanding Paper Award. The technique, detailed in a preprint by Fan Chen, Sinho Chewi, Constantinos Daskalakis, and Alexander Rakhlin, delivers samples with guaranteed error δ in polylog(1/δ) steps—exponentially faster than the poly(1/δ) scaling of conventional discrete SDE samplers.

FORS frames the accept-reject step as a Bernoulli factory problem and solves it through curvilinear integrals, sidestepping the need for density evaluations entirely. Standard diffusion samplers rely on score-function estimates (gradients of the log-density) but accumulate discretization error that scales polynomially with the target accuracy. FORS breaks that barrier by using only first-order score estimates under standard L₂ accuracy assumptions, proving that exponential convergence is achievable without architectural changes or access to the underlying density.

How it works

The paper applies to both generative diffusion models and log-concave distributions. In the diffusion setting, FORS operates on the reverse-time SDE that underpins models like score-based generative networks. For log-concave targets—distributions whose log-density is concave—the method provides the first score-based sampler with polylogarithmic step complexity. The authors' analysis shows that curvilinear integration over the score field yields an unbiased Bernoulli coin flip for the rejection step, eliminating the need to compute or approximate the density ratio that classical rejection sampling requires.

The work is theoretical; no code or pretrained checkpoints accompany the arXiv preprint. Practitioners looking to implement FORS will need to translate the curvilinear-integral machinery into a numerical routine compatible with existing score networks. The payoff is a sampler that, in principle, can be bolted onto any diffusion model without retraining and will converge to the target distribution exponentially faster than current discrete-time methods as the error tolerance tightens.