FantasyPortrait: Enhancing Multi-Character Portrait Animation with
  Expression-Augmented Diffusion Transformers
Producing expressive facial animations from static images is a challenging
task. Prior methods relying on explicit geometric priors (e.g., facial
landmarks or 3DMM) often suffer from artifacts in cross reenactment and
struggle to capture subtle emotions. Furthermore, existing approaches lack
support for multi-character animation, as driving features from different
individuals frequently interfere with one another, complicating the task. To
address these challenges, we propose FantasyPortrait, a diffusion transformer
based framework capable of generating high-fidelity and emotion-rich animations
for both single- and multi-character scenarios. Our method introduces an
expression-augmented learning strategy that utilizes implicit representations
to capture identity-agnostic facial dynamics, enhancing the model's ability to
render fine-grained emotions. For multi-character control, we design a masked
cross-attention mechanism that ensures independent yet coordinated expression
generation, effectively preventing feature interference. To advance research in
this area, we propose the Multi-Expr dataset and ExprBench, which are
specifically designed datasets and benchmarks for training and evaluating
multi-character portrait animations. Extensive experiments demonstrate that
FantasyPortrait significantly outperforms state-of-the-art methods in both
quantitative metrics and qualitative evaluations, excelling particularly in
challenging cross reenactment and multi-character contexts. Our project page is
https://fantasy-amap.github.io/fantasy-portrait/.
6879df6521b37e676c8e41cd
Qiang Wang
claimed_verified
2025-07-18T07:49:24.746Z
653b195c5f1703225b2fd571
https://cdn-avatars.huggingface.co/v1/production/uploads/653b195c5f1703225b2fd571/aCDRKL6PBkzhh8-nzDiMo.jpeg
wangqiang
user
wangqiang9
6879df6521b37e676c8e41ce
Mengchao Wang
6879df6521b37e676c8e41cf
Fan Jiang
6879df6521b37e676c8e41d0
Yaqi Fan
6879df6521b37e676c8e41d1
Yonggang Qi
6879df6521b37e676c8e41d2
Mu Xu
FantasyPortrait: Enhancing Multi-Character Portrait Animation with
  Expression-Augmented Diffusion Transformers
Producing expressive facial animations from static images is a challenging
task. Prior methods relying on explicit geometric priors (e.g., facial
landmarks or 3DMM) often suffer from artifacts in cross reenactment and
struggle to capture subtle emotions. Furthermore, existing approaches lack
support for multi-character animation, as driving features from different
individuals frequently interfere with one another, complicating the task. To
address these challenges, we propose FantasyPortrait, a diffusion transformer
based framework capable of generating high-fidelity and emotion-rich animations
for both single- and multi-character scenarios. Our method introduces an
expression-augmented learning strategy that utilizes implicit representations
to capture identity-agnostic facial dynamics, enhancing the model's ability to
render fine-grained emotions. For multi-character control, we design a masked
cross-attention mechanism that ensures independent yet coordinated expression
generation, effectively preventing feature interference. To advance research in
this area, we propose the Multi-Expr dataset and ExprBench, which are
specifically designed datasets and benchmarks for training and evaluating
multi-character portrait animations. Extensive experiments demonstrate that
FantasyPortrait significantly outperforms state-of-the-art methods in both
quantitative metrics and qualitative evaluations, excelling particularly in
challenging cross reenactment and multi-character contexts. Our project page is
https://fantasy-amap.github.io/fantasy-portrait/.
FantasyPortrait, a diffusion transformer framework, generates high-fidelity and emotion-rich facial animations for single and multi-character scenarios using implicit representations and a masked cross-attention mechanism.
diffusion transformer expression-augmented learning implicit representations masked cross-attention mechanism Multi-Expr dataset ExprBench