MagicInfinite: Generating Infinite Talking Videos with Your Words and
  Voice
We present MagicInfinite, a novel diffusion Transformer (DiT) framework that
overcomes traditional portrait animation limitations, delivering high-fidelity
results across diverse character types-realistic humans, full-body figures, and
stylized anime characters. It supports varied facial poses, including
back-facing views, and animates single or multiple characters with input masks
for precise speaker designation in multi-character scenes. Our approach tackles
key challenges with three innovations: (1) 3D full-attention mechanisms with a
sliding window denoising strategy, enabling infinite video generation with
temporal coherence and visual quality across diverse character styles; (2) a
two-stage curriculum learning scheme, integrating audio for lip sync, text for
expressive dynamics, and reference images for identity preservation, enabling
flexible multi-modal control over long sequences; and (3) region-specific masks
with adaptive loss functions to balance global textual control and local audio
guidance, supporting speaker-specific animations. Efficiency is enhanced via
our innovative unified step and cfg distillation techniques, achieving a 20x
inference speed boost over the basemodel: generating a 10 second 540x540p video
in 10 seconds or 720x720p in 30 seconds on 8 H100 GPUs, without quality loss.
Evaluations on our new benchmark demonstrate MagicInfinite's superiority in
audio-lip synchronization, identity preservation, and motion naturalness across
diverse scenarios. It is publicly available at https://www.hedra.com/, with
examples at https://magicinfinite.github.io/.
67d129d732b4bbfb938321a1
Hongwei Yi
admin_assigned
2025-03-12T15:35:50.514Z
67cda39ea1aad70c919a900f
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/vyF20L4ng460PaAIDeO97.png
hwy
user
hongweiyi
67d129d732b4bbfb938321a2
Tian Ye
claimed_verified
2025-03-12T08:50:12.319Z
66015e8aa4d296af07de538e
/avatars/a1295c631cc2646282c545859975ce4c.svg
Owen
user
Owen777
67d129d732b4bbfb938321a3
Shitong Shao
claimed_verified
2025-03-19T09:46:40.444Z
64aa48245fc663fee24125be
https://cdn-avatars.huggingface.co/v1/production/uploads/64aa48245fc663fee24125be/8Me4rbY3XdY48p9QyEpaA.jpeg
ShanShan
user
sst12345
67d129d732b4bbfb938321a4
Xuancheng Yang
67d129d732b4bbfb938321a5
Jiantong Zhao
admin_assigned
2025-03-12T15:36:27.599Z
655359eee0a82177ae5faf50
/avatars/2792e0349eaf32d1bb52a2bae23f143f.svg
Jiantong Zhao
user
zjt000125
67d129d732b4bbfb938321a6
Hanzhong Guo
67d129d732b4bbfb938321a7
Terrance Wang
67d129d732b4bbfb938321a8
Qingyu Yin
admin_assigned
2025-03-12T15:36:56.627Z
6453cb22908e259483c0a061
https://cdn-avatars.huggingface.co/v1/production/uploads/6453cb22908e259483c0a061/hMgdwZUsUbgquGalzPGzV.jpeg
Qingyu_Yin
user
MikaStars39
67d129d732b4bbfb938321a9
Zeke Xie
claimed_verified
2025-03-16T21:17:48.716Z
67a5fb17baac478d94c59fec
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/gMtdNiD8ztyAjB_weexcK.png
Z.X.
user
xLeaF
67d129d732b4bbfb938321aa
Lei Zhu
67d129d732b4bbfb938321ab
Wei Li
67d129d732b4bbfb938321ac
Michael Lingelbach
admin_assigned
2025-03-12T15:37:38.347Z
6170a328dff0ef663e421ac0
/avatars/d1699613947cc147ea087d41608d6f46.svg
Michael Lingelbach
user
mjlbach
67d129d732b4bbfb938321ad
Daquan Zhou
admin_assigned
2025-03-12T15:37:21.244Z
63fe0b160c1bbe8e29d2dd32
/avatars/bc574036287170a77057893efaa48e2d.svg
Zhou
user
DaQuan21
MagicInfinite: Generating Infinite Talking Videos with Your Words and
  Voice
We present MagicInfinite, a novel diffusion Transformer (DiT) framework that
overcomes traditional portrait animation limitations, delivering high-fidelity
results across diverse character types-realistic humans, full-body figures, and
stylized anime characters. It supports varied facial poses, including
back-facing views, and animates single or multiple characters with input masks
for precise speaker designation in multi-character scenes. Our approach tackles
key challenges with three innovations: (1) 3D full-attention mechanisms with a
sliding window denoising strategy, enabling infinite video generation with
temporal coherence and visual quality across diverse character styles; (2) a
two-stage curriculum learning scheme, integrating audio for lip sync, text for
expressive dynamics, and reference images for identity preservation, enabling
flexible multi-modal control over long sequences; and (3) region-specific masks
with adaptive loss functions to balance global textual control and local audio
guidance, supporting speaker-specific animations. Efficiency is enhanced via
our innovative unified step and cfg distillation techniques, achieving a 20x
inference speed boost over the basemodel: generating a 10 second 540x540p video
in 10 seconds or 720x720p in 30 seconds on 8 H100 GPUs, without quality loss.
Evaluations on our new benchmark demonstrate MagicInfinite's superiority in
audio-lip synchronization, identity preservation, and motion naturalness across
diverse scenarios. It is publicly available at https://www.hedra.com/, with
examples at https://magicinfinite.github.io/.
MagicInfinite, a diffusion Transformer framework, provides high-fidelity portrait animations across various styles with techniques like 3D full-attention, curriculum learning, region-specific masks, and step and cfg distillation.
diffusion Transformer DiT 3D full-attention mechanisms sliding window denoising infinite video generation temporal coherence visual quality two-stage curriculum learning audio-lip synchronization expressive dynamics identity preservation flexible multi-modal control region-specific masks adaptive loss functions unified step and cfg distillation inference speed boost benchmark motion naturalness