DreamActor-M1: Holistic, Expressive and Robust Human Image Animation
  with Hybrid Guidance
While recent image-based human animation methods achieve realistic body and
facial motion synthesis, critical gaps remain in fine-grained holistic
controllability, multi-scale adaptability, and long-term temporal coherence,
which leads to their lower expressiveness and robustness. We propose a
diffusion transformer (DiT) based framework, DreamActor-M1, with hybrid
guidance to overcome these limitations. For motion guidance, our hybrid control
signals that integrate implicit facial representations, 3D head spheres, and 3D
body skeletons achieve robust control of facial expressions and body movements,
while producing expressive and identity-preserving animations. For scale
adaptation, to handle various body poses and image scales ranging from
portraits to full-body views, we employ a progressive training strategy using
data with varying resolutions and scales. For appearance guidance, we integrate
motion patterns from sequential frames with complementary visual references,
ensuring long-term temporal coherence for unseen regions during complex
movements. Experiments demonstrate that our method outperforms the
state-of-the-art works, delivering expressive results for portraits,
upper-body, and full-body generation with robust long-term consistency. Project
Page: https://grisoon.github.io/DreamActor-M1/.
67edf7b6d277de0ec2aa5b6b
Yuxuan Luo
67edf7b6d277de0ec2aa5b6c
Zhengkun Rong
67edf7b6d277de0ec2aa5b6d
Lizhen Wang
admin_assigned
2025-04-03T13:47:42.725Z
63451d2dfeba4bdba59cf9b1
/avatars/ff5dd2f3b502c54e7c3e2512c3e98b28.svg
Lizhen Wang
user
wanglz14
67edf7b6d277de0ec2aa5b6e
Longhao Zhang
admin_assigned
2025-04-03T13:47:48.593Z
602e78801f993496bc14d9c9
/avatars/eff73f60e6652d107cebeda3b2b87fe1.svg
Longhao Zhang
user
zhanglonghao
67edf7b6d277de0ec2aa5b6f
Tianshu Hu
67edf7b6d277de0ec2aa5b70
Yongming Zhu
admin_assigned
2025-04-03T13:48:03.944Z
668f66c56575435392165c25
/avatars/4ebe10abc29229e7862ada6891d1ccbe.svg
Zhu
user
YongmingZhu
DreamActor-M1: Holistic, Expressive and Robust Human Image Animation
  with Hybrid Guidance
While recent image-based human animation methods achieve realistic body and
facial motion synthesis, critical gaps remain in fine-grained holistic
controllability, multi-scale adaptability, and long-term temporal coherence,
which leads to their lower expressiveness and robustness. We propose a
diffusion transformer (DiT) based framework, DreamActor-M1, with hybrid
guidance to overcome these limitations. For motion guidance, our hybrid control
signals that integrate implicit facial representations, 3D head spheres, and 3D
body skeletons achieve robust control of facial expressions and body movements,
while producing expressive and identity-preserving animations. For scale
adaptation, to handle various body poses and image scales ranging from
portraits to full-body views, we employ a progressive training strategy using
data with varying resolutions and scales. For appearance guidance, we integrate
motion patterns from sequential frames with complementary visual references,
ensuring long-term temporal coherence for unseen regions during complex
movements. Experiments demonstrate that our method outperforms the
state-of-the-art works, delivering expressive results for portraits,
upper-body, and full-body generation with robust long-term consistency. Project
Page: https://grisoon.github.io/DreamActor-M1/.
A diffusion transformer framework, DreamActor-M1, enhances image-based human animation through hybrid guidance, achieving robust control, multi-scale adaptability, and long-term temporal coherence.
diffusion transformer DreamActor-M1 hybrid guidance implicit facial representations 3D head spheres 3D body skeletons robust control body movements progressive training strategy varying resolutions sequential frames visual references long-term temporal coherence expressive animations identity-preserving animations