Wan-Animate: Unified Character Animation and Replacement with Holistic
  Replication
We introduce Wan-Animate, a unified framework for character animation and
replacement. Given a character image and a reference video, Wan-Animate can
animate the character by precisely replicating the expressions and movements of
the character in the video to generate high-fidelity character videos.
Alternatively, it can integrate the animated character into the reference video
to replace the original character, replicating the scene's lighting and color
tone to achieve seamless environmental integration. Wan-Animate is built upon
the Wan model. To adapt it for character animation tasks, we employ a modified
input paradigm to differentiate between reference conditions and regions for
generation. This design unifies multiple tasks into a common symbolic
representation. We use spatially-aligned skeleton signals to replicate body
motion and implicit facial features extracted from source images to reenact
expressions, enabling the generation of character videos with high
controllability and expressiveness. Furthermore, to enhance environmental
integration during character replacement, we develop an auxiliary Relighting
LoRA. This module preserves the character's appearance consistency while
applying the appropriate environmental lighting and color tone. Experimental
results demonstrate that Wan-Animate achieves state-of-the-art performance. We
are committed to open-sourcing the model weights and its source code.
68cb68635a7803ff3be42daa
Gang Cheng
68cb68635a7803ff3be42dab
Xin Gao
68cb68635a7803ff3be42dac
Li Hu
68cb68635a7803ff3be42dad
Siqi Hu
68cb68635a7803ff3be42dae
Mingyang Huang
68cb68635a7803ff3be42daf
Chaonan Ji
68cb68635a7803ff3be42db0
Ju Li
68cb68635a7803ff3be42db1
Dechao Meng
68cb68635a7803ff3be42db2
Jinwei Qi
68cb68635a7803ff3be42db3
Penchong Qiao
68cb68635a7803ff3be42db4
Zhen Shen
68cb68635a7803ff3be42db5
Yafei Song
68cb68635a7803ff3be42db6
Ke Sun
68cb68635a7803ff3be42db7
Linrui Tian
68cb68635a7803ff3be42db8
Feng Wang
68cb68635a7803ff3be42db9
Guangyuan Wang
68cb68635a7803ff3be42dba
Qi Wang
68cb68635a7803ff3be42dbb
Zhongjian Wang
68cb68635a7803ff3be42dbc
Jiayu Xiao
68cb68635a7803ff3be42dbd
Sheng Xu
68cb68635a7803ff3be42dbe
Bang Zhang
68cb68635a7803ff3be42dbf
Peng Zhang
68cb68635a7803ff3be42dc0
Xindi Zhang
68cb68635a7803ff3be42dc1
Zhe Zhang
68cb68635a7803ff3be42dc2
Jingren Zhou
68cb68635a7803ff3be42dc3
Lian Zhuo
Wan-Animate: Unified Character Animation and Replacement with Holistic
  Replication
We introduce Wan-Animate, a unified framework for character animation and
replacement. Given a character image and a reference video, Wan-Animate can
animate the character by precisely replicating the expressions and movements of
the character in the video to generate high-fidelity character videos.
Alternatively, it can integrate the animated character into the reference video
to replace the original character, replicating the scene's lighting and color
tone to achieve seamless environmental integration. Wan-Animate is built upon
the Wan model. To adapt it for character animation tasks, we employ a modified
input paradigm to differentiate between reference conditions and regions for
generation. This design unifies multiple tasks into a common symbolic
representation. We use spatially-aligned skeleton signals to replicate body
motion and implicit facial features extracted from source images to reenact
expressions, enabling the generation of character videos with high
controllability and expressiveness. Furthermore, to enhance environmental
integration during character replacement, we develop an auxiliary Relighting
LoRA. This module preserves the character's appearance consistency while
applying the appropriate environmental lighting and color tone. Experimental
results demonstrate that Wan-Animate achieves state-of-the-art performance. We
are committed to open-sourcing the model weights and its source code.
Wan-Animate is a unified framework for character animation and replacement, using spatially-aligned skeleton signals and implicit facial features to generate high-fidelity character videos with seamless environmental integration.
character animation reference video high-fidelity character videos Wan model input paradigm spatially-aligned skeleton signals implicit facial features character replacement environmental integration Relighting LoRA appearance consistency