IVEBench: Modern Benchmark Suite for Instruction-Guided Video Editing
  Assessment
Instruction-guided video editing has emerged as a rapidly advancing research
direction, offering new opportunities for intuitive content transformation
while also posing significant challenges for systematic evaluation. Existing
video editing benchmarks fail to support the evaluation of instruction-guided
video editing adequately and further suffer from limited source diversity,
narrow task coverage and incomplete evaluation metrics. To address the above
limitations, we introduce IVEBench, a modern benchmark suite specifically
designed for instruction-guided video editing assessment. IVEBench comprises a
diverse database of 600 high-quality source videos, spanning seven semantic
dimensions, and covering video lengths ranging from 32 to 1,024 frames. It
further includes 8 categories of editing tasks with 35 subcategories, whose
prompts are generated and refined through large language models and expert
review. Crucially, IVEBench establishes a three-dimensional evaluation protocol
encompassing video quality, instruction compliance and video fidelity,
integrating both traditional metrics and multimodal large language model-based
assessments. Extensive experiments demonstrate the effectiveness of IVEBench in
benchmarking state-of-the-art instruction-guided video editing methods, showing
its ability to provide comprehensive and human-aligned evaluation outcomes.
68edc14ade1fee572713a891
Yinan Chen
68edc14ade1fee572713a892
Jiangning Zhang
68edc14ade1fee572713a893
Teng Hu
68edc14ade1fee572713a894
Yuxiang Zeng
68edc14ade1fee572713a895
Zhucun Xue
68edc14ade1fee572713a896
Qingdong He
68edc14ade1fee572713a897
Chengjie Wang
68edc14ade1fee572713a898
Yong Liu
68edc14ade1fee572713a899
Xiaobin Hu
68edc14ade1fee572713a89a
Shuicheng Yan
IVEBench: Modern Benchmark Suite for Instruction-Guided Video Editing
  Assessment
Instruction-guided video editing has emerged as a rapidly advancing research
direction, offering new opportunities for intuitive content transformation
while also posing significant challenges for systematic evaluation. Existing
video editing benchmarks fail to support the evaluation of instruction-guided
video editing adequately and further suffer from limited source diversity,
narrow task coverage and incomplete evaluation metrics. To address the above
limitations, we introduce IVEBench, a modern benchmark suite specifically
designed for instruction-guided video editing assessment. IVEBench comprises a
diverse database of 600 high-quality source videos, spanning seven semantic
dimensions, and covering video lengths ranging from 32 to 1,024 frames. It
further includes 8 categories of editing tasks with 35 subcategories, whose
prompts are generated and refined through large language models and expert
review. Crucially, IVEBench establishes a three-dimensional evaluation protocol
encompassing video quality, instruction compliance and video fidelity,
integrating both traditional metrics and multimodal large language model-based
assessments. Extensive experiments demonstrate the effectiveness of IVEBench in
benchmarking state-of-the-art instruction-guided video editing methods, showing
its ability to provide comprehensive and human-aligned evaluation outcomes.
IVEBench is a benchmark suite for instruction-guided video editing that addresses limitations in existing benchmarks through diverse video sources, comprehensive task coverage, and a multi-dimensional evaluation protocol.
instruction-guided video editing benchmark suite source diversity task coverage evaluation metrics large language models video quality instruction compliance video fidelity multimodal assessments