DanceTogether! Identity-Preserving Multi-Person Interactive Video
  Generation
Controllable video generation (CVG) has advanced rapidly, yet current systems
falter when more than one actor must move, interact, and exchange positions
under noisy control signals. We address this gap with DanceTogether, the first
end-to-end diffusion framework that turns a single reference image plus
independent pose-mask streams into long, photorealistic videos while strictly
preserving every identity. A novel MaskPoseAdapter binds "who" and "how" at
every denoising step by fusing robust tracking masks with semantically rich-but
noisy-pose heat-maps, eliminating the identity drift and appearance bleeding
that plague frame-wise pipelines. To train and evaluate at scale, we introduce
(i) PairFS-4K, 26 hours of dual-skater footage with 7,000+ distinct IDs, (ii)
HumanRob-300, a one-hour humanoid-robot interaction set for rapid cross-domain
transfer, and (iii) TogetherVideoBench, a three-track benchmark centered on the
DanceTogEval-100 test suite covering dance, boxing, wrestling, yoga, and figure
skating. On TogetherVideoBench, DanceTogether outperforms the prior arts by a
significant margin. Moreover, we show that a one-hour fine-tune yields
convincing human-robot videos, underscoring broad generalization to embodied-AI
and HRI tasks. Extensive ablations confirm that persistent identity-action
binding is critical to these gains. Together, our model, datasets, and
benchmark lift CVG from single-subject choreography to compositionally
controllable, multi-actor interaction, opening new avenues for digital
production, simulation, and embodied intelligence. Our video demos and code are
available at https://DanceTog.github.io/.
683463235a49ee2881ade255
Junhao Chen
claimed_verified
2025-05-26T13:18:31.745Z
652b80d4703b3743c25c87d5
/avatars/3d9c0429fc6082cfb28e2dcd05845733.svg
Dream How Chen
user
yisuanwang
683463235a49ee2881ade256
Mingjin Chen
683463235a49ee2881ade257
Jianjin Xu
683463235a49ee2881ade258
Xiang Li
683463235a49ee2881ade259
Junting Dong
683463235a49ee2881ade25a
Mingze Sun
683463235a49ee2881ade25b
Puhua Jiang
683463235a49ee2881ade25c
Hongxiang Li
683463235a49ee2881ade25d
Yuhang Yang
683463235a49ee2881ade25e
Hao Zhao
683463235a49ee2881ade25f
Xiaoxiao Long
683463235a49ee2881ade260
Ruqi Huang
DanceTogether! Identity-Preserving Multi-Person Interactive Video
  Generation
Controllable video generation (CVG) has advanced rapidly, yet current systems
falter when more than one actor must move, interact, and exchange positions
under noisy control signals. We address this gap with DanceTogether, the first
end-to-end diffusion framework that turns a single reference image plus
independent pose-mask streams into long, photorealistic videos while strictly
preserving every identity. A novel MaskPoseAdapter binds "who" and "how" at
every denoising step by fusing robust tracking masks with semantically rich-but
noisy-pose heat-maps, eliminating the identity drift and appearance bleeding
that plague frame-wise pipelines. To train and evaluate at scale, we introduce
(i) PairFS-4K, 26 hours of dual-skater footage with 7,000+ distinct IDs, (ii)
HumanRob-300, a one-hour humanoid-robot interaction set for rapid cross-domain
transfer, and (iii) TogetherVideoBench, a three-track benchmark centered on the
DanceTogEval-100 test suite covering dance, boxing, wrestling, yoga, and figure
skating. On TogetherVideoBench, DanceTogether outperforms the prior arts by a
significant margin. Moreover, we show that a one-hour fine-tune yields
convincing human-robot videos, underscoring broad generalization to embodied-AI
and HRI tasks. Extensive ablations confirm that persistent identity-action
binding is critical to these gains. Together, our model, datasets, and
benchmark lift CVG from single-subject choreography to compositionally
controllable, multi-actor interaction, opening new avenues for digital
production, simulation, and embodied intelligence. Our video demos and code are
available at https://DanceTog.github.io/.
DanceTogether, an end-to-end diffusion framework, generates long, photorealistic multi-actor interaction videos from single reference images and pose-mask streams, outperforming existing systems.
diffusion framework MaskPoseAdapter denoising identity drift appearance bleeding PairFS-4K HumanRob-300 TogetherVideoBench DanceTogEval-100 embodied-AI HRI tasks persistent identity-action binding