PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video
  Generation
Existing video generation models excel at producing photo-realistic videos
from text or images, but often lack physical plausibility and 3D
controllability. To overcome these limitations, we introduce PhysCtrl, a novel
framework for physics-grounded image-to-video generation with physical
parameters and force control. At its core is a generative physics network that
learns the distribution of physical dynamics across four materials (elastic,
sand, plasticine, and rigid) via a diffusion model conditioned on physics
parameters and applied forces. We represent physical dynamics as 3D point
trajectories and train on a large-scale synthetic dataset of 550K animations
generated by physics simulators. We enhance the diffusion model with a novel
spatiotemporal attention block that emulates particle interactions and
incorporates physics-based constraints during training to enforce physical
plausibility. Experiments show that PhysCtrl generates realistic,
physics-grounded motion trajectories which, when used to drive image-to-video
models, yield high-fidelity, controllable videos that outperform existing
methods in both visual quality and physical plausibility. Project Page:
https://cwchenwang.github.io/physctrl
68d4a00136950a9dff156875
Chen Wang
claimed_verified
2025-09-25T07:11:07.843Z
62e9d5af6687b60b9c01240a
/avatars/3983be5662aa28aefcb18deaf08d7cb1.svg
Chen Wang
user
chenwang
68d4a00136950a9dff156876
Chuhao Chen
admin_assigned
2025-09-25T11:59:52.439Z
65ffa278871b36bf842153df
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/X8NiJoFvHCd-IWRii-mRh.jpeg
Chuhao Chen
user
MorPhLingXD
68d4a00136950a9dff156877
Yiming Huang
68d4a00136950a9dff156878
Zhiyang Dou
admin_assigned
2025-09-25T12:00:12.191Z
645223fb01d7bd9555ea399a
https://cdn-avatars.huggingface.co/v1/production/uploads/645223fb01d7bd9555ea399a/fVR7XmGg6pMSRKx9sEdvT.png
Zhiyang Dou
user
frankzydou
68d4a00136950a9dff156879
Yuan Liu
68d4a00136950a9dff15687a
Jiatao Gu
admin_assigned
2025-09-25T12:00:19.569Z
6164e72d73996c363c52e66d
https://cdn-avatars.huggingface.co/v1/production/uploads/1634002684894-noauth.png
Jiatao Gu
user
thomagram
68d4a00136950a9dff15687b
Lingjie Liu
PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video
  Generation
Existing video generation models excel at producing photo-realistic videos
from text or images, but often lack physical plausibility and 3D
controllability. To overcome these limitations, we introduce PhysCtrl, a novel
framework for physics-grounded image-to-video generation with physical
parameters and force control. At its core is a generative physics network that
learns the distribution of physical dynamics across four materials (elastic,
sand, plasticine, and rigid) via a diffusion model conditioned on physics
parameters and applied forces. We represent physical dynamics as 3D point
trajectories and train on a large-scale synthetic dataset of 550K animations
generated by physics simulators. We enhance the diffusion model with a novel
spatiotemporal attention block that emulates particle interactions and
incorporates physics-based constraints during training to enforce physical
plausibility. Experiments show that PhysCtrl generates realistic,
physics-grounded motion trajectories which, when used to drive image-to-video
models, yield high-fidelity, controllable videos that outperform existing
methods in both visual quality and physical plausibility. Project Page:
https://cwchenwang.github.io/physctrl
PhysCtrl is a physics-grounded framework for generating realistic, controllable videos from images using a diffusion model with spatiotemporal attention and physics-based constraints.
generative physics network diffusion model physical parameters force control 3D point trajectories spatiotemporal attention block physics-based constraints physical plausibility image-to-video models high-fidelity videos