X-Dyna: Expressive Dynamic Human Image Animation
We introduce X-Dyna, a novel zero-shot, diffusion-based pipeline for
animating a single human image using facial expressions and body movements
derived from a driving video, that generates realistic, context-aware dynamics
for both the subject and the surrounding environment. Building on prior
approaches centered on human pose control, X-Dyna addresses key shortcomings
causing the loss of dynamic details, enhancing the lifelike qualities of human
video animations. At the core of our approach is the Dynamics-Adapter, a
lightweight module that effectively integrates reference appearance context
into the spatial attentions of the diffusion backbone while preserving the
capacity of motion modules in synthesizing fluid and intricate dynamic details.
Beyond body pose control, we connect a local control module with our model to
capture identity-disentangled facial expressions, facilitating accurate
expression transfer for enhanced realism in animated scenes. Together, these
components form a unified framework capable of learning physical human motion
and natural scene dynamics from a diverse blend of human and scene videos.
Comprehensive qualitative and quantitative evaluations demonstrate that X-Dyna
outperforms state-of-the-art methods, creating highly lifelike and expressive
animations. The code is available at https://github.com/bytedance/X-Dyna.
678dc72ab3cda33f4a7e3b94
Di Chang
extracted_confirmed
2025-01-20T04:33:59.756Z
64a5d8219f3b568c202b3137
https://cdn-avatars.huggingface.co/v1/production/uploads/64a5d8219f3b568c202b3137/TI20Z1lWHzZpLsMkayDbU.png
Di Chang
user
Boese0601
678dc72ab3cda33f4a7e3b95
Hongyi Xu
678dc72ab3cda33f4a7e3b96
You Xie
admin_assigned
2025-01-20T14:20:36.757Z
6408dfd4b6a334f53e24023c
/avatars/b7e3fa4fbec6313e94ff3384b74dabfc.svg
You Xie
user
youxie
678dc72ab3cda33f4a7e3b97
Yipeng Gao
admin_assigned
2025-01-20T14:20:23.934Z
67312401d433c6b122c38202
/avatars/599523d1412ab01671aa7f86c14d508a.svg
Yipeng Gao
user
YipengGao
678dc72ab3cda33f4a7e3b98
Zhengfei Kuang
admin_assigned
2025-01-20T14:20:17.627Z
673460e68f847a342114a00d
/avatars/a8bed17291f166a6e1dc79413f6ce80a.svg
Zhengfei Kuang
user
zfkuang
678dc72ab3cda33f4a7e3b99
Shengqu Cai
admin_assigned
2025-01-20T14:20:11.259Z
66a1da7cc9e703d2af5ad742
/avatars/f9cd9ae3407e249ab4569479200feb1f.svg
Shengqu Cai
user
primecai
678dc72ab3cda33f4a7e3b9a
Chenxu Zhang
678dc72ab3cda33f4a7e3b9b
Guoxian Song
admin_assigned
2025-01-20T14:19:38.417Z
63086a237dc1b1a54cc6c24d
/avatars/477b94134edc4c18c8f769ecbb7d8091.svg
Song
user
guoxiansong
678dc72ab3cda33f4a7e3b9c
Chao Wang
678dc72ab3cda33f4a7e3b9d
Yichun Shi
admin_assigned
2025-01-20T14:19:30.964Z
6309ad81b105f8675bd5a796
/avatars/01c2186924486da4606e128e83709164.svg
Shi
user
Yichun
678dc72ab3cda33f4a7e3b9e
Zeyuan Chen
678dc72ab3cda33f4a7e3b9f
Shijie Zhou
admin_assigned
2025-01-20T14:18:48.920Z
642a276516d4d8293c9a47e8
/avatars/80e6db8bc2544f3486b11b57858a8692.svg
Shijie Zhou
user
shijiezhou
678dc72ab3cda33f4a7e3ba0
Linjie Luo
678dc72ab3cda33f4a7e3ba1
Gordon Wetzstein
admin_assigned
2025-01-20T14:17:49.662Z
6694e583ac96ca2c17131505
/avatars/6e7a31f257e36cf301da6f879dc0a122.svg
Gordon Wetzstein
user
wetzste1
678dc72ab3cda33f4a7e3ba2
Mohammad Soleymani
admin_assigned
2025-01-20T14:17:43.486Z
65fcb99d383d3f256c3a92d2
/avatars/b85d32f4d7a19816b8d499e05b173ad1.svg
Mohammad Soleymani
user
msoleymani
X-Dyna: Expressive Dynamic Human Image Animation
We introduce X-Dyna, a novel zero-shot, diffusion-based pipeline for
animating a single human image using facial expressions and body movements
derived from a driving video, that generates realistic, context-aware dynamics
for both the subject and the surrounding environment. Building on prior
approaches centered on human pose control, X-Dyna addresses key shortcomings
causing the loss of dynamic details, enhancing the lifelike qualities of human
video animations. At the core of our approach is the Dynamics-Adapter, a
lightweight module that effectively integrates reference appearance context
into the spatial attentions of the diffusion backbone while preserving the
capacity of motion modules in synthesizing fluid and intricate dynamic details.
Beyond body pose control, we connect a local control module with our model to
capture identity-disentangled facial expressions, facilitating accurate
expression transfer for enhanced realism in animated scenes. Together, these
components form a unified framework capable of learning physical human motion
and natural scene dynamics from a diverse blend of human and scene videos.
Comprehensive qualitative and quantitative evaluations demonstrate that X-Dyna
outperforms state-of-the-art methods, creating highly lifelike and expressive
animations. The code is available at https://github.com/bytedance/X-Dyna.
X-Dyna uses a diffusion-based pipeline and Dynamics-Adapter to animate human images by integrating facial expressions and body movements, resulting in realistic and expressive videos.
zero-shot diffusion-based pipeline facial expressions body movements dynamic details lifelike qualities Dynamics-Adapter spatial attentions diffusion backbone motion modules fluid and intricate dynamic details identity-disentangled facial expressions expression transfer natural scene dynamics physical human motion