Neural Emulator Superiority: When Machine Learning for PDEs Surpasses its Training Data
Felix Koehler, Nils Thuerey
Neural operators or emulators for PDEs trained on data from numerical solversare conventionally assumed to be limited by their training data's fidelity. Wechallenge this assumption by identifying "emulator superiority," where neuralnetworks trained purely on low-fidelity solver data can achieve higher accuracythan those solvers when evaluated against a higher-fidelity reference. Ourtheoretical analysis reveals how the interplay between emulator inductivebiases, training objectives, and numerical error characteristics enablessuperior performance during multi-step rollouts. We empirically validate thisfinding across different PDEs using standard neural architectures, demonstratingthat emulators can implicitly learn dynamics that are more regularized orexhibit more favorable error accumulation properties than their training data,potentially surpassing training data limitations and mitigating numericalartifacts. This work prompts a re-evaluation of emulator benchmarking,suggesting neural emulators might achieve greater physical fidelity than theirtraining source within specific operational regimes.