SciVideoBench: Benchmarking Scientific Video Reasoning in Large
  Multimodal Models
Large Multimodal Models (LMMs) have achieved remarkable progress across
various capabilities; however, complex video reasoning in the scientific domain
remains a significant and challenging frontier. Current video benchmarks
predominantly target general scenarios where perception/recognition is heavily
relied on, while with relatively simple reasoning tasks, leading to saturation
and thus failing to effectively evaluate advanced multimodal cognitive skills.
To address this critical gap, we introduce SciVideoBench, a rigorous benchmark
specifically designed to assess advanced video reasoning in scientific
contexts. SciVideoBench consists of 1,000 carefully crafted multiple-choice
questions derived from cutting-edge scientific experimental videos spanning
over 25 specialized academic subjects and verified by a semi-automatic system.
Each question demands sophisticated domain-specific knowledge, precise
spatiotemporal perception, and intricate logical reasoning, effectively
challenging models' higher-order cognitive abilities. Our evaluation highlights
significant performance deficits in state-of-the-art proprietary and
open-source LMMs, including Gemini 2.5 Pro and Qwen2.5-VL, indicating
substantial room for advancement in video reasoning capabilities. Detailed
analyses of critical factors such as reasoning complexity and visual grounding
provide valuable insights and clear direction for future developments in LMMs,
driving the evolution of truly capable multimodal AI co-scientists. We hope
SciVideoBench could fit the interests of the community and help to push the
boundary of cutting-edge AI for border science.
68e86dbd95e8e6771df3890f
Andong Deng
claimed_verified
2025-10-10T09:12:33.201Z
64616f03455531c6be7cfc37
/avatars/a97207e7ec9482162f051624bb28fdfb.svg
Andong Deng
user
groundmore
68e86dbd95e8e6771df38910
Taojiannan Yang
68e86dbd95e8e6771df38911
Shoubin Yu
68e86dbd95e8e6771df38912
Lincoln Spencer
68e86dbd95e8e6771df38913
Mohit Bansal
68e86dbd95e8e6771df38914
Chen Chen
68e86dbd95e8e6771df38915
Serena Yeung-Levy
68e86dbd95e8e6771df38916
Xiaohan Wang
claimed_verified
2025-10-21T14:06:38.031Z
65703fab7f50602340d23704
/avatars/324c45f5fba9cd8c38a89b30427c06b4.svg
Xiaohan Wang
user
nicholswang
SciVideoBench: Benchmarking Scientific Video Reasoning in Large
  Multimodal Models
Large Multimodal Models (LMMs) have achieved remarkable progress across
various capabilities; however, complex video reasoning in the scientific domain
remains a significant and challenging frontier. Current video benchmarks
predominantly target general scenarios where perception/recognition is heavily
relied on, while with relatively simple reasoning tasks, leading to saturation
and thus failing to effectively evaluate advanced multimodal cognitive skills.
To address this critical gap, we introduce SciVideoBench, a rigorous benchmark
specifically designed to assess advanced video reasoning in scientific
contexts. SciVideoBench consists of 1,000 carefully crafted multiple-choice
questions derived from cutting-edge scientific experimental videos spanning
over 25 specialized academic subjects and verified by a semi-automatic system.
Each question demands sophisticated domain-specific knowledge, precise
spatiotemporal perception, and intricate logical reasoning, effectively
challenging models' higher-order cognitive abilities. Our evaluation highlights
significant performance deficits in state-of-the-art proprietary and
open-source LMMs, including Gemini 2.5 Pro and Qwen2.5-VL, indicating
substantial room for advancement in video reasoning capabilities. Detailed
analyses of critical factors such as reasoning complexity and visual grounding
provide valuable insights and clear direction for future developments in LMMs,
driving the evolution of truly capable multimodal AI co-scientists. We hope
SciVideoBench could fit the interests of the community and help to push the
boundary of cutting-edge AI for border science.
SciVideoBench is a benchmark designed to evaluate advanced video reasoning in scientific contexts, challenging models with sophisticated domain-specific knowledge and logical reasoning.
Large Multimodal Models LMMs video reasoning scientific domain video benchmarks perception recognition reasoning tasks multiple-choice questions spatiotemporal perception logical reasoning cognitive abilities reasoning complexity visual grounding multimodal AI co-scientists