WikiVideo: Article Generation from Multiple Videos
We present the challenging task of automatically creating a high-level
Wikipedia-style article that aggregates information from multiple diverse
videos about real-world events, such as natural disasters or political
elections. Videos are intuitive sources for retrieval-augmented generation
(RAG), but most contemporary RAG workflows focus heavily on text and existing
methods for video-based summarization focus on low-level scene understanding
rather than high-level event semantics. To close this gap, we introduce
WikiVideo, a benchmark consisting of expert-written articles and densely
annotated videos that provide evidence for articles' claims, facilitating the
integration of video into RAG pipelines and enabling the creation of in-depth
content that is grounded in multimodal sources. We further propose
Collaborative Article Generation (CAG), a novel interactive method for article
creation from multiple videos. CAG leverages an iterative interaction between
an r1-style reasoning model and a VideoLLM to draw higher level inferences
about the target event than is possible with VideoLLMs alone, which fixate on
low-level visual features. We benchmark state-of-the-art VideoLLMs and CAG in
both oracle retrieval and RAG settings and find that CAG consistently
outperforms alternative methods, while suggesting intriguing avenues for future
work.
67efffab5e8b84102499ebfc
Alexander Martin
claimed_verified
2025-04-08T06:53:55.294Z
644ffb154f731658826945c8
/avatars/46fa3a15c2b974692bee5b6ed6468be0.svg
Alex Martin
user
alexmartin1722
67efffab5e8b84102499ebfd
Reno Kriz
67efffab5e8b84102499ebfe
William Gantt Walden
67efffab5e8b84102499ebff
Kate Sanders
67efffab5e8b84102499ec00
Hannah Recknor
67efffab5e8b84102499ec01
Eugene Yang
67efffab5e8b84102499ec02
Francis Ferraro
67efffab5e8b84102499ec03
Benjamin Van Durme
WikiVideo: Article Generation from Multiple Videos
We present the challenging task of automatically creating a high-level
Wikipedia-style article that aggregates information from multiple diverse
videos about real-world events, such as natural disasters or political
elections. Videos are intuitive sources for retrieval-augmented generation
(RAG), but most contemporary RAG workflows focus heavily on text and existing
methods for video-based summarization focus on low-level scene understanding
rather than high-level event semantics. To close this gap, we introduce
WikiVideo, a benchmark consisting of expert-written articles and densely
annotated videos that provide evidence for articles' claims, facilitating the
integration of video into RAG pipelines and enabling the creation of in-depth
content that is grounded in multimodal sources. We further propose
Collaborative Article Generation (CAG), a novel interactive method for article
creation from multiple videos. CAG leverages an iterative interaction between
an r1-style reasoning model and a VideoLLM to draw higher level inferences
about the target event than is possible with VideoLLMs alone, which fixate on
low-level visual features. We benchmark state-of-the-art VideoLLMs and CAG in
both oracle retrieval and RAG settings and find that CAG consistently
outperforms alternative methods, while suggesting intriguing avenues for future
work.
retrieval-augmented generation (RAG) WikiVideo Collaborative Article Generation (CAG) r1-style reasoning model VideoLLM