Phantom: Subject-consistent video generation via cross-modal alignment
The continuous development of foundational models for video generation is
evolving into various applications, with subject-consistent video generation
still in the exploratory stage. We refer to this as Subject-to-Video, which
extracts subject elements from reference images and generates
subject-consistent video through textual instructions. We believe that the
essence of subject-to-video lies in balancing the dual-modal prompts of text
and image, thereby deeply and simultaneously aligning both text and visual
content. To this end, we propose Phantom, a unified video generation framework
for both single and multi-subject references. Building on existing
text-to-video and image-to-video architectures, we redesign the joint
text-image injection model and drive it to learn cross-modal alignment via
text-image-video triplet data. In particular, we emphasize subject consistency
in human generation, covering existing ID-preserving video generation while
offering enhanced advantages. The project homepage is here
https://phantom-video.github.io/Phantom/.
67b40141ad717fe02e188c1a
Lijie Liu
claimed_verified
2025-02-18T09:31:42.570Z
63a950ac3453852ef5394178
/avatars/48a5e537b10e2247a17e63502e3201a6.svg
Lijie Liu
user
liulj13
67b40141ad717fe02e188c1b
Tianxiang Ma
admin_assigned
2025-02-19T09:45:00.117Z
657ab4705e1c941f4c2f7877
/avatars/c450f81f83dd0436ae120ab15616c4f7.svg
Tianxiang Ma
user
Grayson111
67b40141ad717fe02e188c1c
Bingchuan Li
admin_assigned
2025-02-19T09:47:57.441Z
63b415037af2e415f2599c18
/avatars/4afbe7d6d05a702f1beeed9c53e78153.svg
Bingchuan Li
user
lbc402
67b40141ad717fe02e188c1d
Zhuowei Chen
admin_assigned
2025-02-19T09:47:50.995Z
6304e2dabad6ce7fc0287d57
/avatars/3fd4a9a62b0ef98db2573411463a9247.svg
Zhuowei_Chen
user
ZhuoweiChen
67b40141ad717fe02e188c1e
Jiawei Liu
claimed_verified
2025-06-12T12:44:43.200Z
63049b95dae2eb7d083f1bf3
/avatars/5eaeadc1318724b54ea59873da11275f.svg
Jiawei Liu
user
jwliu-cc
67b40141ad717fe02e188c1f
Qian He
67b40141ad717fe02e188c20
Xinglong Wu
Phantom: Subject-consistent video generation via cross-modal alignment
The continuous development of foundational models for video generation is
evolving into various applications, with subject-consistent video generation
still in the exploratory stage. We refer to this as Subject-to-Video, which
extracts subject elements from reference images and generates
subject-consistent video through textual instructions. We believe that the
essence of subject-to-video lies in balancing the dual-modal prompts of text
and image, thereby deeply and simultaneously aligning both text and visual
content. To this end, we propose Phantom, a unified video generation framework
for both single and multi-subject references. Building on existing
text-to-video and image-to-video architectures, we redesign the joint
text-image injection model and drive it to learn cross-modal alignment via
text-image-video triplet data. In particular, we emphasize subject consistency
in human generation, covering existing ID-preserving video generation while
offering enhanced advantages. The project homepage is here
https://phantom-video.github.io/Phantom/.
A unified video generation framework called Phantom integrates text and image inputs to produce subject-consistent videos, focusing on human generation and leveraging cross-modal alignment.
text-to-video image-to-video joint text-image injection model cross-modal alignment text-image-video triplet data subject consistency ID-preserving video generation