In-Context Learning of Stochastic Differential Equations with Foundation Inference Models
Patrick Seifner, Kostadin Cvejoski, David Berghaus, César Ali Ojeda Marin, Ramsés J. Sánchez
Stochastic differential equations (SDEs) describe dynamical systems where deterministic flows, governed by a drift function, are superimposed with random fluctuations, dictated by a diffusion function. The accurate estimation (*or discovery*) of these functions from data is a central problem in machine learning, with wide application across the natural and social sciences. Yet, current solutions either heavily rely on prior knowledge of the dynamics or involve intricate training procedures. We introduce FIM-SDE (Foundation Inference Model for SDEs), a pretrained recognition model capable of performing accurate *in-context* (or zero-shot) estimation of the drift and diffusion functions of *low-dimensional* SDEs from noisy time series data. Leveraging concepts from amortized inference and neural operators, we train FIM-SDE in a supervised fashion to map a large set of noisy, discretely observed SDE paths onto the space of drift and diffusion functions.  We demonstrate that *one and the same* FIM-SDE model achieves robust in-context function estimation (*i.e.* without any parameter fine-tuning) across a wide range of synthetic and real-world processes --- from canonical SDE systems (*e.g*. double-well dynamics or weakly perturbed Lorentz attractors) to stock price recordings and oil-price and wind-speed fluctuations.Our pretrained model, repository and tutorials are available online (see link in .pdf file)