BindWeave: Subject-Consistent Video Generation via Cross-Modal
  Integration
Diffusion Transformer has shown remarkable abilities in generating
high-fidelity videos, delivering visually coherent frames and rich details over
extended durations. However, existing video generation models still fall short
in subject-consistent video generation due to an inherent difficulty in parsing
prompts that specify complex spatial relationships, temporal logic, and
interactions among multiple subjects. To address this issue, we propose
BindWeave, a unified framework that handles a broad range of subject-to-video
scenarios from single-subject cases to complex multi-subject scenes with
heterogeneous entities. To bind complex prompt semantics to concrete visual
subjects, we introduce an MLLM-DiT framework in which a pretrained multimodal
large language model performs deep cross-modal reasoning to ground entities and
disentangle roles, attributes, and interactions, yielding subject-aware hidden
states that condition the diffusion transformer for high-fidelity
subject-consistent video generation. Experiments on the OpenS2V benchmark
demonstrate that our method achieves superior performance across subject
consistency, naturalness, and text relevance in generated videos, outperforming
existing open-source and commercial models.
68de6e6f70ada21878c75057
Zhaoyang Li
68de6e6f70ada21878c75058
Dongjun Qian
68de6e6f70ada21878c75059
Kai Su
68de6e6f70ada21878c7505a
Qishuai Diao
68de6e6f70ada21878c7505b
Xiangyang Xia
68de6e6f70ada21878c7505c
Chang Liu
68de6e6f70ada21878c7505d
Wenfei Yang
68de6e6f70ada21878c7505e
Tianzhu Zhang
68de6e6f70ada21878c7505f
Zehuan Yuan
BindWeave: Subject-Consistent Video Generation via Cross-Modal
  Integration
Diffusion Transformer has shown remarkable abilities in generating
high-fidelity videos, delivering visually coherent frames and rich details over
extended durations. However, existing video generation models still fall short
in subject-consistent video generation due to an inherent difficulty in parsing
prompts that specify complex spatial relationships, temporal logic, and
interactions among multiple subjects. To address this issue, we propose
BindWeave, a unified framework that handles a broad range of subject-to-video
scenarios from single-subject cases to complex multi-subject scenes with
heterogeneous entities. To bind complex prompt semantics to concrete visual
subjects, we introduce an MLLM-DiT framework in which a pretrained multimodal
large language model performs deep cross-modal reasoning to ground entities and
disentangle roles, attributes, and interactions, yielding subject-aware hidden
states that condition the diffusion transformer for high-fidelity
subject-consistent video generation. Experiments on the OpenS2V benchmark
demonstrate that our method achieves superior performance across subject
consistency, naturalness, and text relevance in generated videos, outperforming
existing open-source and commercial models.
BindWeave, a unified framework using MLLM-DiT, enhances subject-consistent video generation by integrating deep cross-modal reasoning with diffusion transformers, achieving superior performance on OpenS2V.
Diffusion Transformer BindWeave MLLM-DiT multimodal large language model deep cross-modal reasoning subject-aware hidden states subject consistency naturalness text relevance OpenS2V benchmark