What Has a Foundation Model Found? Using Inductive Bias to Probe for
  World Models
Foundation models are premised on the idea that sequence prediction can
uncover deeper domain understanding, much like how Kepler's predictions of
planetary motion later led to the discovery of Newtonian mechanics. However,
evaluating whether these models truly capture deeper structure remains a
challenge. We develop a technique for evaluating foundation models that
examines how they adapt to synthetic datasets generated from some postulated
world model. Our technique measures whether the foundation model's inductive
bias aligns with the world model, and so we refer to it as an inductive bias
probe. Across multiple domains, we find that foundation models can excel at
their training tasks yet fail to develop inductive biases towards the
underlying world model when adapted to new tasks. We particularly find that
foundation models trained on orbital trajectories consistently fail to apply
Newtonian mechanics when adapted to new physics tasks. Further analysis reveals
that these models behave as if they develop task-specific heuristics that fail
to generalize.
68749c03257d4f043537033e
Keyon Vafa
68749c03257d4f043537033f
Peter G. Chang
68749c03257d4f0435370340
Ashesh Rambachan
68749c03257d4f0435370341
Sendhil Mullainathan
What Has a Foundation Model Found? Using Inductive Bias to Probe for
  World Models
Foundation models are premised on the idea that sequence prediction can
uncover deeper domain understanding, much like how Kepler's predictions of
planetary motion later led to the discovery of Newtonian mechanics. However,
evaluating whether these models truly capture deeper structure remains a
challenge. We develop a technique for evaluating foundation models that
examines how they adapt to synthetic datasets generated from some postulated
world model. Our technique measures whether the foundation model's inductive
bias aligns with the world model, and so we refer to it as an inductive bias
probe. Across multiple domains, we find that foundation models can excel at
their training tasks yet fail to develop inductive biases towards the
underlying world model when adapted to new tasks. We particularly find that
foundation models trained on orbital trajectories consistently fail to apply
Newtonian mechanics when adapted to new physics tasks. Further analysis reveals
that these models behave as if they develop task-specific heuristics that fail
to generalize.
Foundation models can perform well on training tasks but often fail to generalize to new tasks, as revealed by an inductive bias probe technique that assesses their alignment with underlying world models.
foundation models sequence prediction inductive bias synthetic datasets world model inductive bias probe Newtonian mechanics task-specific heuristics generalization