Tadpole autoencoder foundation model scales 3D PDE learning to terabyte-scale training

Researchers at TUM Physics-Based Simulation introduced Tadpole, a foundation model for three-dimensional partial differential equations that pre-trains as an autoencoder on synthetic data generated on the fly. The model learns representations across physical systems with different numbers of state variables and spatial resolutions by autoencoding single-channel spatial crops rather than full simulation volumes. The online data-generation framework scales training to the equivalent of hundreds of terabytes without storage or I/O bottlenecks—a key advantage for systems where disk capacity typically limits dataset size.

The autoencoder architecture enables multiple downstream tasks beyond reconstruction. For dynamics learning, the authors propose a parameter-efficient fine-tuning strategy that combines low-rank adaptation, latent-space transformations, and reintroduced skip connections. This approach achieves accurate temporal modeling while keeping the number of trainable parameters low. The model also supports generative modeling tasks, demonstrating versatility across the PDE learning pipeline.

Tadpole's single-channel crop strategy addresses a core transferability problem in 3D physics modeling: systems with heterogeneous state variables and resolutions typically require separate models. By learning spatial patterns at the crop level, the encoder produces representations that transfer across different physical regimes. Pre-training data is synthetic and generated during training rather than stored, which removes the dataset size ceiling imposed by disk capacity. Source code and pre-trained weights are available on GitHub. The next test will be whether the crop-based representations generalize to real-world PDE datasets with noise, irregular grids, and boundary conditions not seen in synthetic training.