Paper2Video: Automatic Video Generation from Scientific Papers
Academic presentation videos have become an essential medium for research
communication, yet producing them remains highly labor-intensive, often
requiring hours of slide design, recording, and editing for a short 2 to 10
minutes video. Unlike natural video, presentation video generation involves
distinctive challenges: inputs from research papers, dense multi-modal
information (text, figures, tables), and the need to coordinate multiple
aligned channels such as slides, subtitles, speech, and human talker. To
address these challenges, we introduce PaperTalker, the first benchmark of 101
research papers paired with author-created presentation videos, slides, and
speaker metadata. We further design four tailored evaluation metrics--Meta
Similarity, PresentArena, PresentQuiz, and IP Memory--to measure how videos
convey the paper's information to the audience. Building on this foundation, we
propose PaperTalker, the first multi-agent framework for academic presentation
video generation. It integrates slide generation with effective layout
refinement by a novel effective tree search visual choice, cursor grounding,
subtitling, speech synthesis, and talking-head rendering, while parallelizing
slide-wise generation for efficiency. Experiments on Paper2Video demonstrate
that the presentation videos produced by our approach are more faithful and
informative than existing baselines, establishing a practical step toward
automated and ready-to-use academic video generation. Our dataset, agent, and
code are available at https://github.com/showlab/Paper2Video.
68e479bfe4e093a7044e4d04
Zeyu Zhu
claimed_verified
2025-10-07T12:27:10.799Z
66c45954ab8f09b10b7ab6a8
/avatars/f9946c775c4d70b8e044865ac34ef121.svg
Zhu
user
ZaynZhu
68e479bfe4e093a7044e4d05
Kevin Qinghong Lin
claimed_verified
2025-10-07T12:27:13.351Z
64440be5af034cdfd69ca3a7
https://cdn-avatars.huggingface.co/v1/production/uploads/64440be5af034cdfd69ca3a7/qmx24QiDFT29vleCxL9TX.jpeg
Qinghong (Kevin) Lin
user
KevinQHLin
68e479bfe4e093a7044e4d06
Mike Zheng Shou
Paper2Video: Automatic Video Generation from Scientific Papers
Academic presentation videos have become an essential medium for research
communication, yet producing them remains highly labor-intensive, often
requiring hours of slide design, recording, and editing for a short 2 to 10
minutes video. Unlike natural video, presentation video generation involves
distinctive challenges: inputs from research papers, dense multi-modal
information (text, figures, tables), and the need to coordinate multiple
aligned channels such as slides, subtitles, speech, and human talker. To
address these challenges, we introduce PaperTalker, the first benchmark of 101
research papers paired with author-created presentation videos, slides, and
speaker metadata. We further design four tailored evaluation metrics--Meta
Similarity, PresentArena, PresentQuiz, and IP Memory--to measure how videos
convey the paper's information to the audience. Building on this foundation, we
propose PaperTalker, the first multi-agent framework for academic presentation
video generation. It integrates slide generation with effective layout
refinement by a novel effective tree search visual choice, cursor grounding,
subtitling, speech synthesis, and talking-head rendering, while parallelizing
slide-wise generation for efficiency. Experiments on Paper2Video demonstrate
that the presentation videos produced by our approach are more faithful and
informative than existing baselines, establishing a practical step toward
automated and ready-to-use academic video generation. Our dataset, agent, and
code are available at https://github.com/showlab/Paper2Video.
PaperTalker is a multi-agent framework that automates academic presentation video generation by integrating slide generation, layout refinement, subtitling, speech synthesis, and talking-head rendering, outperforming existing methods.
multi-agent framework slide generation layout refinement tree search visual choice cursor grounding subtitling speech synthesis talking-head rendering parallelization slide-wise generation