MAGREF: Masked Guidance for Any-Reference Video Generation
Video generation has made substantial strides with the emergence of deep
generative models, especially diffusion-based approaches. However, video
generation based on multiple reference subjects still faces significant
challenges in maintaining multi-subject consistency and ensuring high
generation quality. In this paper, we propose MAGREF, a unified framework for
any-reference video generation that introduces masked guidance to enable
coherent multi-subject video synthesis conditioned on diverse reference images
and a textual prompt. Specifically, we propose (1) a region-aware dynamic
masking mechanism that enables a single model to flexibly handle various
subject inference, including humans, objects, and backgrounds, without
architectural changes, and (2) a pixel-wise channel concatenation mechanism
that operates on the channel dimension to better preserve appearance features.
Our model delivers state-of-the-art video generation quality, generalizing from
single-subject training to complex multi-subject scenarios with coherent
synthesis and precise control over individual subjects, outperforming existing
open-source and commercial baselines. To facilitate evaluation, we also
introduce a comprehensive multi-subject video benchmark. Extensive experiments
demonstrate the effectiveness of our approach, paving the way for scalable,
controllable, and high-fidelity multi-subject video synthesis. Code and model
can be found at: https://github.com/MAGREF-Video/MAGREF
683917d0ecf59de6ef345e76
Yufan Deng
683917d0ecf59de6ef345e77
Xun Guo
683917d0ecf59de6ef345e78
Yuanyang Yin
683917d0ecf59de6ef345e79
Jacob Zhiyuan Fang
683917d0ecf59de6ef345e7a
Yiding Yang
683917d0ecf59de6ef345e7b
Yizhi Wang
683917d0ecf59de6ef345e7c
Shenghai Yuan
claimed_verified
2025-05-30T06:53:49.399Z
63468720dd6d90d82ccf3450
https://cdn-avatars.huggingface.co/v1/production/uploads/63468720dd6d90d82ccf3450/tVBFlmZNz8FRMkOrDaDID.jpeg
YSH
user
BestWishYsh
683917d0ecf59de6ef345e7d
Angtian Wang
683917d0ecf59de6ef345e7e
Bo Liu
683917d0ecf59de6ef345e7f
Haibin Huang
683917d0ecf59de6ef345e80
Chongyang Ma
MAGREF: Masked Guidance for Any-Reference Video Generation
Video generation has made substantial strides with the emergence of deep
generative models, especially diffusion-based approaches. However, video
generation based on multiple reference subjects still faces significant
challenges in maintaining multi-subject consistency and ensuring high
generation quality. In this paper, we propose MAGREF, a unified framework for
any-reference video generation that introduces masked guidance to enable
coherent multi-subject video synthesis conditioned on diverse reference images
and a textual prompt. Specifically, we propose (1) a region-aware dynamic
masking mechanism that enables a single model to flexibly handle various
subject inference, including humans, objects, and backgrounds, without
architectural changes, and (2) a pixel-wise channel concatenation mechanism
that operates on the channel dimension to better preserve appearance features.
Our model delivers state-of-the-art video generation quality, generalizing from
single-subject training to complex multi-subject scenarios with coherent
synthesis and precise control over individual subjects, outperforming existing
open-source and commercial baselines. To facilitate evaluation, we also
introduce a comprehensive multi-subject video benchmark. Extensive experiments
demonstrate the effectiveness of our approach, paving the way for scalable,
controllable, and high-fidelity multi-subject video synthesis. Code and model
can be found at: https://github.com/MAGREF-Video/MAGREF
MAGREF is a unified framework for video generation that uses masked guidance and dynamic masking for coherent multi-subject synthesis from diverse references and text prompts.
diffusion-based approaches multi-subject consistency unified framework masked guidance region-aware dynamic masking pixel-wise channel concatenation appearance features multi-subject video benchmark scalable controllable high-fidelity video synthesis