EnerVerse-AC: Envisioning Embodied Environments with Action Condition
Robotic imitation learning has advanced from solving static tasks to
addressing dynamic interaction scenarios, but testing and evaluation remain
costly and challenging due to the need for real-time interaction with dynamic
environments. We propose EnerVerse-AC (EVAC), an action-conditional world model
that generates future visual observations based on an agent's predicted
actions, enabling realistic and controllable robotic inference. Building on
prior architectures, EVAC introduces a multi-level action-conditioning
mechanism and ray map encoding for dynamic multi-view image generation while
expanding training data with diverse failure trajectories to improve
generalization. As both a data engine and evaluator, EVAC augments
human-collected trajectories into diverse datasets and generates realistic,
action-conditioned video observations for policy testing, eliminating the need
for physical robots or complex simulations. This approach significantly reduces
costs while maintaining high fidelity in robotic manipulation evaluation.
Extensive experiments validate the effectiveness of our method. Code,
checkpoints, and datasets can be found at
<https://annaj2178.github.io/EnerverseAC.github.io>.
6826b00c251d26fc0cd035cc
Yuxin Jiang
admin_assigned
2025-05-16T08:10:30.068Z
63c20105726f62e411fbe882
https://cdn-avatars.huggingface.co/v1/production/uploads/63c20105726f62e411fbe882/2UsU9O2psbDjJzz-sAmGH.jpeg
Yuxin Jiang
user
YuxinJiang
6826b00c251d26fc0cd035cd
Shengcong Chen
admin_assigned
2025-05-16T08:10:37.358Z
6575f9aeca03b6c514fe6e5c
/avatars/a6e9d428beaa124ee989d702b9bf4f85.svg
Shengcong Chen
user
Shengcong
6826b00c251d26fc0cd035ce
Siyuan Huang
admin_assigned
2025-05-16T08:10:50.217Z
63c7a33121bd95f80ed74652
/avatars/7dd59afea785a2bff0ec2b757abd474e.svg
Siyuan Huang
user
thuhsy
6826b00c251d26fc0cd035cf
Liliang Chen
admin_assigned
2025-05-16T08:10:55.980Z
640b00555a9c21b95c6449b3
/avatars/5fa43b956f3acc671f033e31b7ca76c5.svg
Liliang Chen
user
pathcn
6826b00c251d26fc0cd035d0
Pengfei Zhou
6826b00c251d26fc0cd035d1
Yue Liao
6826b00c251d26fc0cd035d2
Xindong He
6826b00c251d26fc0cd035d3
Chiming Liu
6826b00c251d26fc0cd035d4
Hongsheng Li
admin_assigned
2025-05-16T08:11:29.857Z
65c04e9c27a5fdca81abcbd9
/avatars/12a155683c824fa23da4a9e2bed4f64e.svg
Hongsheng LI
user
hsli-cuhk
6826b00c251d26fc0cd035d5
Maoqing Yao
admin_assigned
2025-05-16T08:11:36.940Z
67739bfa64e8b7438ae68eb4
/avatars/15193bfbce487b2de4ce8c86bd18885a.svg
Maoqing Yao
user
AutobotZero
6826b00c251d26fc0cd035d6
Guanghui Ren
admin_assigned
2025-05-16T08:11:44.407Z
646ec9b135f55eb49e405faa
/avatars/a17194be585d20e2a021e77a5a20e213.svg
Guanghui Ren
user
sundrops
EnerVerse-AC: Envisioning Embodied Environments with Action Condition
Robotic imitation learning has advanced from solving static tasks to
addressing dynamic interaction scenarios, but testing and evaluation remain
costly and challenging due to the need for real-time interaction with dynamic
environments. We propose EnerVerse-AC (EVAC), an action-conditional world model
that generates future visual observations based on an agent's predicted
actions, enabling realistic and controllable robotic inference. Building on
prior architectures, EVAC introduces a multi-level action-conditioning
mechanism and ray map encoding for dynamic multi-view image generation while
expanding training data with diverse failure trajectories to improve
generalization. As both a data engine and evaluator, EVAC augments
human-collected trajectories into diverse datasets and generates realistic,
action-conditioned video observations for policy testing, eliminating the need
for physical robots or complex simulations. This approach significantly reduces
costs while maintaining high fidelity in robotic manipulation evaluation.
Extensive experiments validate the effectiveness of our method. Code,
checkpoints, and datasets can be found at
<https://annaj2178.github.io/EnerverseAC.github.io>.
EnerVerse-AC, an action-conditional world model, enables realistic robotic inference and testing by simulating future actions and observations, thereby reducing costs and improving generalization in dynamic settings.
action-conditional world model robotic imitation learning dynamic multi-view image generation ray map encoding failure trajectories policy testing