The Anatomy of Video Editing: A Dataset and Benchmark Suite for AI-Assisted Video Editing
Machine learning is transforming the video editing industry. Recent advances
in computer vision have leveled-up video editing tasks such as intelligent
reframing, rotoscoping, color grading, or applying digital makeups. However,
most of the solutions have focused on video manipulation and VFX. This work
introduces the Anatomy of Video Editing, a dataset, and benchmark, to foster
research in AI-assisted video editing. Our benchmark suite focuses on video
editing tasks, beyond visual effects, such as automatic footage organization
and assisted video assembling. To enable research on these fronts, we annotate
more than 1.5M tags, with relevant concepts to cinematography, from 196176
shots sampled from movie scenes. We establish competitive baseline methods and
detailed analyses for each of the tasks. We hope our work sparks innovative
research towards underexplored areas of AI-assisted video editing.