Project
Machine Learning Surrogates for Partial Differential Equations and Microstructures Simulations
Machine Learning (ML) is increasingly used as a surrogate model for numerical solvers of Partial Differential Equations (PDEs) and for digital twins. Two obstacles, however, limit its reliability. First, the spectral bias of neural networks toward low frequencies hampers the modelling of multi-scale structure — the mid- and high-frequency content that dominates turbulence phenomena and fine-grained microstructures. Second, the data-hungry nature of these models makes training cost-prohibitive, since the labelled trajectories must themselves be produced by the expensive solvers the surrogate is meant to replace, particularly in parametric and out-of-domain settings. This project addresses both: more data-efficient training of neural PDE solvers through active learning, and neural-operator architectures designed to capture local, high-frequency features.
Project Details
Project term
February 1, 2025–May 9, 2026
Affiliations
University of Stuttgart
Institute
Computer Science, Electrical Engineering and Information Technology
Principal Investigator
Methods
The project advances two complementary methodologies. On the data side, we developed AL4PDE, a modular pool-based active-learning framework in which a numerical solver is queried only for the most informative initial conditions and PDE parameters. Candidate inputs are scored either by ensemble disagreement (query-by-committee uncertainty) or by latent-feature representations, and batches are assembled using selection strategies including SBAL (Stochastic Batch Active Learning), LCMD (Largest Cluster Maximum Distance), Core-Set, BAIT (Batch Active Learning via Information maTrices), and Top-K, benchmarked against random sampling and Latin Hypercube Sampling. The framework couples parametric PDEs of increasing spatial dimensionality (1D–3D) with multiple surrogate architectures (U-Net, FNO, SineNet) in a solver-in-the-loop cycle, retraining after every acquisition round.
On the modelling or representation side, we designed LOGLO-FNO, an extension of the popular Fourier Neural Operator model that directly targets its truncation of high-frequency modes. Two components are added in parallel to the global Fourier branch: a local spectral-convolution branch operating on non-overlapping patches to capture small-scale and local structures, and a high-frequency propagation module acting as a high-pass filter, as well as a vectorized implementation of a frequency-aware loss based on radially binned spectral-energy errors that explicitly penalises mid- and high-band deviations. Models are trained under both one-step (teacher-forced) and fully autoregressive schedules, and evaluated with spatial, spectral, and energy-spectra metrics against a broad suite of state-of-the-art neural-operator baselines.
Results
LOGLO-FNO consistently lowers both spatial and spectral errors relative to base FNO and to recent state-of-the-art baselines (U-FNO, F-FNO, LSM, Transolver, NO-LIDK) across the tested PDE problems — turbulent Kolmogorov flow, compressible Navier–Stokes, the Wave-Gauss equation, the compressible Euler four-quadrant Riemann problem, 2D Diffusion–Reaction, and a 3D turbulent radiative layer. Error reductions fall in the 20–50% range, are largest in the mid- and high-frequency bands, and translate into markedly improved stability over long autoregressive rollouts. Crucially, the local branch lets the LOGLO-FNO model match baseline FNO accuracy with up to 50% fewer trainable parameters.
For active learning, the strongest acquisition strategies — SBAL, LCMD, and BAIT — reduce average prediction error by up to 71% over random sampling, and reach equivalent or lower error using only about a quarter of the training data on complex physical systems. Greedy strategies (Top-K, Core-Set) and static Latin Hypercube Sampling provide no reliable benefit. Most valuably, active learning compresses the upper error quantiles, improving worst-case reliability rather than only the mean.
Discussion
Together, these results address the two bottlenecks raised at the outset: LOGLO-FNO recovers the high-frequency, local content that standard operators discard, while AL4PDE removes much of the data cost of training surrogates and improves their behaviour in the hardest dynamical regimes. The two lines are naturally synergistic — the architecture supplies a more stable, high-frequency-capable base model precisely where active learning’s gains were weakest (turbulent compressible Navier–Stokes), suggesting that placing a LOGLO-FNO-style operator inside the active-learning loop, with acquisition steered toward the spectral bands the model actually fails on, is a promising next step.
Reaching these conclusions was only possible with the granted compute resources at NHR4CES, RWTH Aachen. The central claims concern reliability and statistical consistency, not single demonstrations: each active-learning experiment retrains surrogates repeatedly within the loop, across six parametric PDEs spanning 1D to 3D, replicated over up to ten random seeds to report proper confidence intervals. In particular, the 3D compressible Navier-Stokes runs relied on the large-memory H100 GPUs of CLAIX’23, and the operator-architecture study required training and spectrally analysing our model against an extensive suite of competing baselines and PDE problems. The compute grant was therefore instrumental not merely in scale but in scientific validity.
Additional Project Information
DFG classification: 409-05 Interactive and Intelligent Systems, Image and Language Processing, Computer Graphics and Visualisation
Software: PyTorch, Python3, NumPy, JAX
Cluster: CLAIX
Publications
Daniel Musekamp, Marimuthu Kalimuthu, David Holzmüller, Makoto Takamoto, Mathias Niepert,
Active Learning for Neural PDE Solvers,
Peer-reviewed publication, December 2024
Marimuthu Kalimuthu, David Holzmüller, Mathias Niepert,
LOGLO-FNO: Efficient Learning of Local and Global Features in Fourier Neural Operators,
Peer-reviewed publication, April 2025
Daniel Musekamp, Marimuthu Kalimuthu, David Holzmüller, Makoto Takamoto, Mathias Niepert,
Active Learning for Neural PDE Solvers,
Peer-reviewed publication, April 2025
Marimuthu Kalimuthu, David Holzmüller, Mathias Niepert,
LOGLO-FNO: Efficient Learning of Local and Global Features in Fourier Neural Operators,
Peer-reviewed publication, December 2025