Tora2: Motion and Appearance Customized Diffusion Transformer for
  Multi-Entity Video Generation
Recent advances in diffusion transformer models for motion-guided video
generation, such as Tora, have shown significant progress. In this paper, we
present Tora2, an enhanced version of Tora, which introduces several design
improvements to expand its capabilities in both appearance and motion
customization. Specifically, we introduce a decoupled personalization extractor
that generates comprehensive personalization embeddings for multiple open-set
entities, better preserving fine-grained visual details compared to previous
methods. Building on this, we design a gated self-attention mechanism to
integrate trajectory, textual description, and visual information for each
entity. This innovation significantly reduces misalignment in multimodal
conditioning during training. Moreover, we introduce a contrastive loss that
jointly optimizes trajectory dynamics and entity consistency through explicit
mapping between motion and personalization embeddings. Tora2 is, to our best
knowledge, the first method to achieve simultaneous multi-entity customization
of appearance and motion for video generation. Experimental results demonstrate
that Tora2 achieves competitive performance with state-of-the-art customization
methods while providing advanced motion control capabilities, which marks a
critical advancement in multi-condition video generation. Project page:
https://github.com/alibaba/Tora .
686dde62cb5725779c60b467
Zhenghao Zhang
686dde62cb5725779c60b468
Junchao Liao
claimed_verified
2025-07-21T08:30:38.055Z
659fa7cc430ffa77ac38247e
https://cdn-avatars.huggingface.co/v1/production/uploads/659fa7cc430ffa77ac38247e/p-MIAnIIp81TPcTFQWe4T.jpeg
Junchao Liao
user
Le0jc
686dde62cb5725779c60b469
Xiangyu Meng
686dde62cb5725779c60b46a
Long Qin
686dde62cb5725779c60b46b
Weizhi Wang
Tora2: Motion and Appearance Customized Diffusion Transformer for
  Multi-Entity Video Generation
Recent advances in diffusion transformer models for motion-guided video
generation, such as Tora, have shown significant progress. In this paper, we
present Tora2, an enhanced version of Tora, which introduces several design
improvements to expand its capabilities in both appearance and motion
customization. Specifically, we introduce a decoupled personalization extractor
that generates comprehensive personalization embeddings for multiple open-set
entities, better preserving fine-grained visual details compared to previous
methods. Building on this, we design a gated self-attention mechanism to
integrate trajectory, textual description, and visual information for each
entity. This innovation significantly reduces misalignment in multimodal
conditioning during training. Moreover, we introduce a contrastive loss that
jointly optimizes trajectory dynamics and entity consistency through explicit
mapping between motion and personalization embeddings. Tora2 is, to our best
knowledge, the first method to achieve simultaneous multi-entity customization
of appearance and motion for video generation. Experimental results demonstrate
that Tora2 achieves competitive performance with state-of-the-art customization
methods while providing advanced motion control capabilities, which marks a
critical advancement in multi-condition video generation. Project page:
https://github.com/alibaba/Tora .
Tora2 enhances motion-guided video generation by introducing a decoupled personalization extractor, gated self-attention mechanism, and contrastive loss for improved multimodal conditioning and entity consistency.
diffusion transformer models Tora decoupled personalization extractor personalization embeddings gated self-attention mechanism trajectory textual description visual information contrastive loss trajectory dynamics entity consistency multi-entity customization video generation