T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video
  Generation
Text-to-video (T2V) generation models have advanced significantly, yet their
ability to compose different objects, attributes, actions, and motions into a
video remains unexplored. Previous text-to-video benchmarks also neglect this
important ability for evaluation. In this work, we conduct the first systematic
study on compositional text-to-video generation. We propose T2V-CompBench, the
first benchmark tailored for compositional text-to-video generation.
T2V-CompBench encompasses diverse aspects of compositionality, including
consistent attribute binding, dynamic attribute binding, spatial relationships,
motion binding, action binding, object interactions, and generative numeracy.
We further carefully design evaluation metrics of MLLM-based metrics,
detection-based metrics, and tracking-based metrics, which can better reflect
the compositional text-to-video generation quality of seven proposed categories
with 700 text prompts. The effectiveness of the proposed metrics is verified by
correlation with human evaluations. We also benchmark various text-to-video
generative models and conduct in-depth analysis across different models and
different compositional categories. We find that compositional text-to-video
generation is highly challenging for current models, and we hope that our
attempt will shed light on future research in this direction.
66a05d78cf52b98e1d286375
Kaiyue Sun
claimed_verified
2024-07-24T07:26:33.664Z
63640ce5ff4b318d1b7b6f5c
/avatars/9336d1dab1491f3c8e84b1ea287eb891.svg
Kaiyue Sun
user
Kaiyue
66a05d78cf52b98e1d286376
Kaiyi Huang
admin_assigned
2024-12-06T17:51:03.269Z
637cba13b8e573d75be96ea6
/avatars/5eca230e63d66947b2a05c1ff964a96c.svg
Nina
user
NinaKarine
66a05d78cf52b98e1d286377
Xian Liu
admin_assigned
2024-07-24T08:10:45.554Z
6423f10a3fa01ecba6fca861
/avatars/e4b4c7dddc5cf962ade45d8bdfa10b1a.svg
Xian Liu
user
alvinliu0
66a05d78cf52b98e1d286378
Yue Wu
66a05d78cf52b98e1d286379
Zihan Xu
66a05d78cf52b98e1d28637a
Zhenguo Li
admin_assigned
2024-07-24T08:13:36.623Z
639944bef7c4648254d124f0
/avatars/c5cbb87cd51cc1341844f67d42c55151.svg
lzg
user
lizhenguo
66a05d78cf52b98e1d28637b
Xihui Liu
admin_assigned
2024-07-24T08:13:43.112Z
65d5ec74cd05bc1eaa125040
/avatars/2de1b1539a86452c2c89570eeb02f5ab.svg
Xihui Liu
user
XihuiLiu
T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video
  Generation
Text-to-video (T2V) generation models have advanced significantly, yet their
ability to compose different objects, attributes, actions, and motions into a
video remains unexplored. Previous text-to-video benchmarks also neglect this
important ability for evaluation. In this work, we conduct the first systematic
study on compositional text-to-video generation. We propose T2V-CompBench, the
first benchmark tailored for compositional text-to-video generation.
T2V-CompBench encompasses diverse aspects of compositionality, including
consistent attribute binding, dynamic attribute binding, spatial relationships,
motion binding, action binding, object interactions, and generative numeracy.
We further carefully design evaluation metrics of MLLM-based metrics,
detection-based metrics, and tracking-based metrics, which can better reflect
the compositional text-to-video generation quality of seven proposed categories
with 700 text prompts. The effectiveness of the proposed metrics is verified by
correlation with human evaluations. We also benchmark various text-to-video
generative models and conduct in-depth analysis across different models and
different compositional categories. We find that compositional text-to-video
generation is highly challenging for current models, and we hope that our
attempt will shed light on future research in this direction.
This study introduces T2V-CompBench, a benchmark for evaluating the compositional capabilities of text-to-video models, including metrics for consistent and dynamic attributes, spatial relationships, motion and action binding, object interactions, and generative numeracy.
text-to-video T2V-CompBench compositional generation MLLM-based metrics detection-based metrics tracking-based metrics compositional categories