Self-Supervised Learning with Lie Symmetries for Partial Differential
  Equations
Machine learning for differential equations paves the way for computationally
efficient alternatives to numerical solvers, with potentially broad impacts in
science and engineering. Though current algorithms typically require simulated
training data tailored to a given setting, one may instead wish to learn useful
information from heterogeneous sources, or from real dynamical systems
observations that are messy or incomplete. In this work, we learn
general-purpose representations of PDEs from heterogeneous data by implementing
joint embedding methods for self-supervised learning (SSL), a framework for
unsupervised representation learning that has had notable success in computer
vision. Our representation outperforms baseline approaches to invariant tasks,
such as regressing the coefficients of a PDE, while also improving the
time-stepping performance of neural solvers. We hope that our proposed
methodology will prove useful in the eventual development of general-purpose
foundation models for PDEs.
64ae1e6eb575c5e2722159c1
Grégoire Mialon
claimed_verified
2023-07-12T13:28:39.442Z
6458c976f1f5263c2ef0514c
/avatars/0cab7449293be91a4fa08c23533b1b1a.svg
Grégoire Mialon
user
gregmialz
64ae1e6eb575c5e2722159c2
Quentin Garrido
extracted_pending
2025-05-12T16:14:44.734Z
63049022412a1b9d381b9dcb
/avatars/cac2a540fda40c5ae558968c50385f46.svg
Quentin Garrido
user
garridoq
64ae1e6eb575c5e2722159c3
Hannah Lawrence
64ae1e6eb575c5e2722159c4
Danyal Rehman
64ae1e6eb575c5e2722159c5
Yann LeCun
admin_assigned
2023-11-23T11:52:22.496Z
64ed0b8c2203a126eb1a5b9a
/avatars/9156dc406ed3f9ee62b73657ac20f5ed.svg
Yann LeCun
user
ylecun
64ae1e6eb575c5e2722159c6
Bobak T. Kiani
claimed_verified
2023-07-13T08:32:27.702Z
64aec711749f04f4f4c7cb4b
/avatars/0449729bdbae87f27ae6663a36430bf3.svg
Kiani
user
Bobak
Self-Supervised Learning with Lie Symmetries for Partial Differential
  Equations
Machine learning for differential equations paves the way for computationally
efficient alternatives to numerical solvers, with potentially broad impacts in
science and engineering. Though current algorithms typically require simulated
training data tailored to a given setting, one may instead wish to learn useful
information from heterogeneous sources, or from real dynamical systems
observations that are messy or incomplete. In this work, we learn
general-purpose representations of PDEs from heterogeneous data by implementing
joint embedding methods for self-supervised learning (SSL), a framework for
unsupervised representation learning that has had notable success in computer
vision. Our representation outperforms baseline approaches to invariant tasks,
such as regressing the coefficients of a PDE, while also improving the
time-stepping performance of neural solvers. We hope that our proposed
methodology will prove useful in the eventual development of general-purpose
foundation models for PDEs.
Joint embedding methods for self-supervised learning are applied to learn general-purpose representations of PDEs from heterogeneous data, outperforming baselines and enhancing neural solvers.
joint embedding methods self-supervised learning SSL unsupervised representation learning PDEs neural solvers