Energy-based model solves inverse imaging without per-task retraining

A preprint posted to arXiv this week introduces an energy-based model that learns normalized priors for inverse imaging problems—inpainting, deblurring, and similar tasks—without requiring retraining for each new degradation operator. The model, trained with a covariance-based regularization term that enforces consistency across measurement conditions, computes explicit posterior densities and preserves the sampling capabilities of diffusion models while adding capabilities those models lack: energy-guided adaptive sampling that adjusts schedules on the fly, unbiased Metropolis-Hastings correction steps, and blind estimation of the degradation operator via Bayes rule.

Existing diffusion priors for inverse problems represent the prior density implicitly and rely on likelihood approximations that introduce sampling bias. The new approach sidesteps both limitations by training an energy function that can be queried directly. The authors validate the method on ImageNet, CelebA, and AFHQ across inpainting and deblurring tasks, reporting competitive or superior performance to established baselines. Because the model computes normalized densities, it can plug into standard MCMC samplers and adjust sampling trajectories mid-run based on the energy landscape—something diffusion models cannot do without expensive likelihood approximations.

The covariance regularization is the key architectural choice. By enforcing that the learned energy function remains consistent under different linear measurement operators during training, the model generalizes to new inverse problems at inference time without additional fine-tuning. The paper demonstrates blind operator estimation—recovering the degradation matrix from noisy measurements alone—by treating the operator as a latent variable and marginalizing over it via Bayes rule, a straightforward calculation when the posterior is normalized.

The preprint does not yet release code or weights, and the paper does not specify GPU requirements or inference speed. The open question is whether the method scales to higher-resolution datasets and whether the covariance regularization holds up under more complex, nonlinear degradations. Those tests will determine whether this approach becomes a standard tool for practitioners working on ill-posed imaging tasks.