VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion
  Generation in Video Models
Despite tremendous recent progress, generative video models still struggle to
capture real-world motion, dynamics, and physics. We show that this limitation
arises from the conventional pixel reconstruction objective, which biases
models toward appearance fidelity at the expense of motion coherence. To
address this, we introduce VideoJAM, a novel framework that instills an
effective motion prior to video generators, by encouraging the model to learn a
joint appearance-motion representation. VideoJAM is composed of two
complementary units. During training, we extend the objective to predict both
the generated pixels and their corresponding motion from a single learned
representation. During inference, we introduce Inner-Guidance, a mechanism that
steers the generation toward coherent motion by leveraging the model's own
evolving motion prediction as a dynamic guidance signal. Notably, our framework
can be applied to any video model with minimal adaptations, requiring no
modifications to the training data or scaling of the model. VideoJAM achieves
state-of-the-art performance in motion coherence, surpassing highly competitive
proprietary models while also enhancing the perceived visual quality of the
generations. These findings emphasize that appearance and motion can be
complementary and, when effectively integrated, enhance both the visual quality
and the coherence of video generation. Project website:
https://hila-chefer.github.io/videojam-paper.github.io/
67a2ec904ea0e3138ac966f2
Hila Chefer
extracted_confirmed
2025-06-04T04:29:22.790Z
6181c72cdcc1df2c9de8a4d8
https://cdn-avatars.huggingface.co/v1/production/uploads/1655248010394-6181c72cdcc1df2c9de8a4d8.jpeg
Hila Chefer
user
Hila
67a2ec904ea0e3138ac966f3
Uriel Singer
admin_assigned
2025-02-05T16:53:54.046Z
6345b71843f4f2d2ed113355
/avatars/a497669a4c53a724c4f6ea615d1dda59.svg
Uriel Singer
user
urielsinger
67a2ec904ea0e3138ac966f4
Amit Zohar
67a2ec904ea0e3138ac966f5
Yuval Kirstain
67a2ec904ea0e3138ac966f6
Adam Polyak
67a2ec904ea0e3138ac966f7
Yaniv Taigman
67a2ec904ea0e3138ac966f8
Lior Wolf
67a2ec904ea0e3138ac966f9
Shelly Sheynin
VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion
  Generation in Video Models
Despite tremendous recent progress, generative video models still struggle to
capture real-world motion, dynamics, and physics. We show that this limitation
arises from the conventional pixel reconstruction objective, which biases
models toward appearance fidelity at the expense of motion coherence. To
address this, we introduce VideoJAM, a novel framework that instills an
effective motion prior to video generators, by encouraging the model to learn a
joint appearance-motion representation. VideoJAM is composed of two
complementary units. During training, we extend the objective to predict both
the generated pixels and their corresponding motion from a single learned
representation. During inference, we introduce Inner-Guidance, a mechanism that
steers the generation toward coherent motion by leveraging the model's own
evolving motion prediction as a dynamic guidance signal. Notably, our framework
can be applied to any video model with minimal adaptations, requiring no
modifications to the training data or scaling of the model. VideoJAM achieves
state-of-the-art performance in motion coherence, surpassing highly competitive
proprietary models while also enhancing the perceived visual quality of the
generations. These findings emphasize that appearance and motion can be
complementary and, when effectively integrated, enhance both the visual quality
and the coherence of video generation. Project website:
https://hila-chefer.github.io/videojam-paper.github.io/
VideoJAM enhances generative video models by integrating motion coherence with appearance fidelity, achieving top performance in motion coherence and visual quality.
generative video models motion coherence dynamics physics pixel reconstruction VideoJAM joint appearance-motion representation Inner-Guidance dynamic guidance signal