Flow Matching steers FLUX.2-klein through reference images, no retraining required

Reference-Guided Flow Matching is a control technique for flow-based diffusion models that steers generation by swapping reference images instead of retraining weights. Researchers from the University of Amsterdam and Northeastern University demonstrate the method on FLUX.2-klein (4B parameters), where it adjusts color, identity, style, and structure while keeping the prompt, random seed, and model weights fixed.

The core insight is that deterministic flow matching models compute a velocity field from a conditional endpoint mean. Shifting that mean shifts the entire flow. The authors instantiate this in two forms: Reference-Mean Guidance, which is training-free and applies a closed-form correction from a reference bank to a frozen model, and Semi-Parametric Guidance, which learns a residual refiner on top of an explicit mean anchor and can swap reference sets at inference time. The semi-parametric variant matches unconditional DiT-B/4 quality on the AFHQv2 dataset.

Training-free control on FLUX.2-klein. Reference-Mean Guidance computes a closed-form endpoint-mean correction from a reference bank and applies it to the frozen 4B-parameter model. The prompt, seed, and weights stay fixed; only the reference set changes.

Swap references at inference time. Semi-Parametric Guidance amortizes the mean-shift idea through an explicit mean anchor and a learned residual refiner. The reference set can be replaced at inference without retraining the refiner, enabling dynamic control over style, identity, or structure.

Matches DiT-B/4 quality on AFHQv2. The semi-parametric variant achieves unconditional DiT-B/4 baseline quality on the AFHQv2 dataset while preserving the ability to steer generation through reference examples.

No auxiliary networks or test-time search. Existing controllable generation methods typically rely on fine-tuning, ControlNet-style auxiliary networks, or iterative test-time optimization. Reference-Guided Flow Matching sidesteps all three by exploiting the mathematical structure of flow matching itself.

The paper frames a broader direction: generative models that adapt by changing the data they follow rather than updating weights. This could simplify deployment for tasks where control requirements shift frequently or where labeled data for fine-tuning is scarce.