ORV: 4D Occupancy-centric Robot Video Generation
Acquiring real-world robotic simulation data through teleoperation is
notoriously time-consuming and labor-intensive. Recently, action-driven
generative models have gained widespread adoption in robot learning and
simulation, as they eliminate safety concerns and reduce maintenance efforts.
However, the action sequences used in these methods often result in limited
control precision and poor generalization due to their globally coarse
alignment. To address these limitations, we propose ORV, an Occupancy-centric
Robot Video generation framework, which utilizes 4D semantic occupancy
sequences as a fine-grained representation to provide more accurate semantic
and geometric guidance for video generation. By leveraging occupancy-based
representations, ORV enables seamless translation of simulation data into
photorealistic robot videos, while ensuring high temporal consistency and
precise controllability. Furthermore, our framework supports the simultaneous
generation of multi-view videos of robot gripping operations - an important
capability for downstream robotic learning tasks. Extensive experimental
results demonstrate that ORV consistently outperforms existing baseline methods
across various datasets and sub-tasks. Demo, Code and Model:
https://orangesodahub.github.io/ORV
683fee0a179d710da07d4352
Xiuyu Yang
claimed_verified
2025-06-04T08:54:13.216Z
634aab35dcf125e4dafc87b1
/avatars/aa2db84fd423e9eefe3ef3167c9d3999.svg
YangXiuyu
user
gzzyyxy
683fee0a179d710da07d4353
Bohan Li
683fee0a179d710da07d4354
Shaocong Xu
683fee0a179d710da07d4355
Nan Wang
683fee0a179d710da07d4356
Chongjie Ye
683fee0a179d710da07d4357
Zhaoxi Chen
683fee0a179d710da07d4358
Minghan Qin
claimed_verified
2025-07-29T12:51:35.788Z
642002b51ccd411979d72b18
https://cdn-avatars.huggingface.co/v1/production/uploads/642002b51ccd411979d72b18/JO9e0o8fAFNC-eYFbx_JW.png
Minghan Qin
user
Qmh
683fee0a179d710da07d4359
Yikang Ding
683fee0a179d710da07d435a
Xin Jin
683fee0a179d710da07d435b
Hang Zhao
683fee0a179d710da07d435c
Hao Zhao
ORV: 4D Occupancy-centric Robot Video Generation
Acquiring real-world robotic simulation data through teleoperation is
notoriously time-consuming and labor-intensive. Recently, action-driven
generative models have gained widespread adoption in robot learning and
simulation, as they eliminate safety concerns and reduce maintenance efforts.
However, the action sequences used in these methods often result in limited
control precision and poor generalization due to their globally coarse
alignment. To address these limitations, we propose ORV, an Occupancy-centric
Robot Video generation framework, which utilizes 4D semantic occupancy
sequences as a fine-grained representation to provide more accurate semantic
and geometric guidance for video generation. By leveraging occupancy-based
representations, ORV enables seamless translation of simulation data into
photorealistic robot videos, while ensuring high temporal consistency and
precise controllability. Furthermore, our framework supports the simultaneous
generation of multi-view videos of robot gripping operations - an important
capability for downstream robotic learning tasks. Extensive experimental
results demonstrate that ORV consistently outperforms existing baseline methods
across various datasets and sub-tasks. Demo, Code and Model:
https://orangesodahub.github.io/ORV
ORV, an Occupancy-centric Robot Video generation framework, uses 4D semantic occupancy sequences to produce photorealistic, temporally consistent, and precisely controllable robot videos, enhancing existing methods.
action-driven generative models occupancy-centric 4D semantic occupancy sequences video generation photorealistic robot videos temporal consistency precise controllability multi-view videos