Mixture of Global and Local Experts with Diffusion Transformer for
  Controllable Face Generation
Controllable face generation poses critical challenges in generative modeling
due to the intricate balance required between semantic controllability and
photorealism. While existing approaches struggle with disentangling semantic
controls from generation pipelines, we revisit the architectural potential of
Diffusion Transformers (DiTs) through the lens of expert specialization. This
paper introduces Face-MoGLE, a novel framework featuring: (1)
Semantic-decoupled latent modeling through mask-conditioned space
factorization, enabling precise attribute manipulation; (2) A mixture of global
and local experts that captures holistic structure and region-level semantics
for fine-grained controllability; (3) A dynamic gating network producing
time-dependent coefficients that evolve with diffusion steps and spatial
locations. Face-MoGLE provides a powerful and flexible solution for
high-quality, controllable face generation, with strong potential in generative
modeling and security applications. Extensive experiments demonstrate its
effectiveness in multimodal and monomodal face generation settings and its
robust zero-shot generalization capability. Project page is available at
https://github.com/XavierJiezou/Face-MoGLE.
68b93229d43cadaf7a688bc6
Xuechao Zou
admin_assigned
2025-09-04T19:37:38.778Z
6617af2beab5eef6b1e8bb9e
/avatars/d939c02027916331d4c44119565f2ca6.svg
XavierJiezou
user
XavierJiezou
68b93229d43cadaf7a688bc7
Shun Zhang
68b93229d43cadaf7a688bc8
Xing Fu
68b93229d43cadaf7a688bc9
Yue Li
68b93229d43cadaf7a688bca
Kai Li
68b93229d43cadaf7a688bcb
Yushe Cao
68b93229d43cadaf7a688bcc
Congyan Lang
68b93229d43cadaf7a688bcd
Pin Tao
68b93229d43cadaf7a688bce
Junliang Xing
Mixture of Global and Local Experts with Diffusion Transformer for
  Controllable Face Generation
Controllable face generation poses critical challenges in generative modeling
due to the intricate balance required between semantic controllability and
photorealism. While existing approaches struggle with disentangling semantic
controls from generation pipelines, we revisit the architectural potential of
Diffusion Transformers (DiTs) through the lens of expert specialization. This
paper introduces Face-MoGLE, a novel framework featuring: (1)
Semantic-decoupled latent modeling through mask-conditioned space
factorization, enabling precise attribute manipulation; (2) A mixture of global
and local experts that captures holistic structure and region-level semantics
for fine-grained controllability; (3) A dynamic gating network producing
time-dependent coefficients that evolve with diffusion steps and spatial
locations. Face-MoGLE provides a powerful and flexible solution for
high-quality, controllable face generation, with strong potential in generative
modeling and security applications. Extensive experiments demonstrate its
effectiveness in multimodal and monomodal face generation settings and its
robust zero-shot generalization capability. Project page is available at
https://github.com/XavierJiezou/Face-MoGLE.
Face-MoGLE, a novel framework using Diffusion Transformers, achieves high-quality, controllable face generation through semantic-decoupled latent modeling, expert specialization, and dynamic gating.
Diffusion Transformers Semantic-decoupled latent modeling mask-conditioned space factorization global experts local experts dynamic gating network multimodal face generation monomodal face generation zero-shot generalization