Do generative video models learn physical principles from watching
  videos?
AI video generation is undergoing a revolution, with quality and realism
advancing rapidly. These advances have led to a passionate scientific debate:
Do video models learn ``world models'' that discover laws of physics -- or,
alternatively, are they merely sophisticated pixel predictors that achieve
visual realism without understanding the physical principles of reality? We
address this question by developing Physics-IQ, a comprehensive benchmark
dataset that can only be solved by acquiring a deep understanding of various
physical principles, like fluid dynamics, optics, solid mechanics, magnetism
and thermodynamics. We find that across a range of current models (Sora,
Runway, Pika, Lumiere, Stable Video Diffusion, and VideoPoet), physical
understanding is severely limited, and unrelated to visual realism. At the same
time, some test cases can already be successfully solved. This indicates that
acquiring certain physical principles from observation alone may be possible,
but significant challenges remain. While we expect rapid advances ahead, our
work demonstrates that visual realism does not imply physical understanding.
Our project page is at https://physics-iq.github.io; code at
https://github.com/google-deepmind/physics-IQ-benchmark.
678a20d55a84f1087bd61c82
Saman Motamed
admin_assigned
2025-01-17T10:51:43.421Z
6475c37b04c82116f9bb2356
/avatars/6ec34eb3cfd091a38454ac3de72aaddc.svg
saman motamed
user
sam-motamed
678a20d55a84f1087bd61c83
Laura Culp
678a20d55a84f1087bd61c84
Kevin Swersky
admin_assigned
2025-01-17T10:51:57.327Z
630e6ef664f1f8d0c771b758
/avatars/1df2bfadb2b6fdf8307189936efc6ef0.svg
Kevin Swersky
user
kswersky
678a20d55a84f1087bd61c85
Priyank Jaini
678a20d55a84f1087bd61c86
Robert Geirhos
admin_assigned
2025-01-17T10:52:10.657Z
673bbe0d7dfcdedd52619ec2
/avatars/531a44f05d0c738bbe3e028c76c2e948.svg
Robert Geirhos
user
rgeirhos
Do generative video models learn physical principles from watching
  videos?
AI video generation is undergoing a revolution, with quality and realism
advancing rapidly. These advances have led to a passionate scientific debate:
Do video models learn ``world models'' that discover laws of physics -- or,
alternatively, are they merely sophisticated pixel predictors that achieve
visual realism without understanding the physical principles of reality? We
address this question by developing Physics-IQ, a comprehensive benchmark
dataset that can only be solved by acquiring a deep understanding of various
physical principles, like fluid dynamics, optics, solid mechanics, magnetism
and thermodynamics. We find that across a range of current models (Sora,
Runway, Pika, Lumiere, Stable Video Diffusion, and VideoPoet), physical
understanding is severely limited, and unrelated to visual realism. At the same
time, some test cases can already be successfully solved. This indicates that
acquiring certain physical principles from observation alone may be possible,
but significant challenges remain. While we expect rapid advances ahead, our
work demonstrates that visual realism does not imply physical understanding.
Our project page is at https://physics-iq.github.io; code at
https://github.com/google-deepmind/physics-IQ-benchmark.
A new benchmark dataset, Physics-IQ, shows that current AI video models achieve visual realism without understanding underlying physical principles, highlighting the need for further advancements.
Physics-IQ Sora Runway Pika Lumiere Stable Video Diffusion VideoPoet fluid dynamics optics solid mechanics magnetism thermodynamics