Gray-Scott parameter recovery blocked by flat plateaus in loss landscape
Direct PDE backpropagation reveals pathological loss geometry with flat plateaus and bifurcation cliffs; ablating PINN components shows residual loss alone smooths the landscape by encoding full PDE dynamics.
Attempting to recover parameters of reaction-diffusion systems by backpropagating directly through the PDE—without surrogate models or neural networks—reveals a fundamental obstacle: the loss landscape is geometrically pathological, with flat plateaus offering no gradient signal and sharp cliffs aligned to bifurcation boundaries.
A preprint posted to arXiv on June 12, 2026 pursues this direct route as a diagnostic probe, unrolling Gray-Scott simulation and backpropagating a steady-state loss to recover parameters. Optimization fails to converge. Plotting the loss surface directly shows why: flat regions with vanishing gradients, bounded by sharp discontinuities that correspond to bifurcation points in the underlying dynamical system. This pathological geometry recurs across multiple loss functions and persists regardless of how gradients are routed to the parameters.
The authors treat this minimal setup as an ablation of physics-informed neural network (PINN) architecture, isolating what each component actually does. When the neural network is held fixed, the residual loss—the term measuring how well a candidate solution satisfies the PDE—becomes quadratic in the parameters and yields a smooth, well-behaved landscape. The residual loss alone avoids the plateau-and-cliff pathology because it implicitly encodes the full PDE dynamics across all initial conditions. The neural network, by contrast, cannot repair an ill-posed parameter subspace; it serves only to complete sparse or noisy observed data.
This division of labor has not been made explicit in prior PINN literature. When the parameter space itself is geometrically pathological, adding a neural network does not solve the inversion problem. Only the residual loss term can smooth the landscape by embedding the physics. The work carries broader implications for when added dimensions—whether neural networks or other augmentations—actually help versus when they mask a fundamental ill-posedness in the underlying problem.
