OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni
  MLLMs
Recent advances in multimodal large language models (MLLMs) have demonstrated
substantial potential in video understanding. However, existing benchmarks fail
to comprehensively evaluate synergistic reasoning capabilities across audio and
visual modalities, often neglecting either one of the modalities or integrating
them in a logically inconsistent manner. To bridge this gap, we introduce
OmniVideoBench, a large-scale and rigorously designed benchmark dedicated to
assessing synergistic audio-visual understanding, with a strong emphasis on
modality complementarity and logical consistency. Specifically, OmniVideoBench
comprises 1000 high-quality question-answer(QA) pairs, each annotated with
step-by-step reasoning traces, derived from 628 diverse videos ranging from
several seconds to 30 minutes, and manually verified to guarantee complete
correctness and uniqueness. Moreover, OmniVideoBench encompasses 13 carefully
designed question types, covering temporal reasoning, spatial localization,
counting, causal inference, summarization, and beyond, thereby capturing the
essential challenges of video understanding. Evaluation of multiple MLLMs on
OmniVideoBench reveals a pronounced gap between model performance and human
reasoning, with open-source models lagging significantly behind their
closed-source counterparts, underscoring the inherent difficulty of genuine
audio-visual reasoning. We will release OmniVideoBench to foster the
development of MLLMs with stronger and more generalizable reasoning
capabilities.
68edc3cede1fee572713a8c7
Caorui Li
68edc3cede1fee572713a8c8
Yu Chen
68edc3cede1fee572713a8c9
Yiyan Ji
68edc3cede1fee572713a8ca
Jin Xu
68edc3cede1fee572713a8cb
Zhenyu Cui
68edc3cede1fee572713a8cc
Shihao Li
claimed_verified
2025-10-22T15:46:30.828Z
67f9d060395fb1a0d7e4ae21
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/GjpOfOuazN7IxcXBpVqRm.png
Shihao Li
user
Leexeo
68edc3cede1fee572713a8cd
Yuanxing Zhang
68edc3cede1fee572713a8ce
Jiafu Tang
68edc3cede1fee572713a8cf
Zhenghao Song
68edc3cede1fee572713a8d0
Dingling Zhang
68edc3cede1fee572713a8d1
Ying He
68edc3cede1fee572713a8d2
Haoxiang Liu
68edc3cede1fee572713a8d3
Yuxuan Wang
68edc3cede1fee572713a8d4
Qiufeng Wang
68edc3cede1fee572713a8d5
Zhenhe Wu
68edc3cede1fee572713a8d6
Jiehui Luo
68edc3cede1fee572713a8d7
Zhiyu Pan
68edc3cede1fee572713a8d8
Weihao Xie
68edc3cede1fee572713a8d9
Chenchen Zhang
claimed_verified
2025-10-14T07:30:15.133Z
64b74b906ab5d14ca7f289cd
/avatars/b131b7c4ce5216708ca4a678f35ead0a.svg
Chenchen Zhang
user
xxzcc
68edc3cede1fee572713a8da
Zhaohui Wang
68edc3cede1fee572713a8db
Jiayi Tian
68edc3cede1fee572713a8dc
Yanghai Wang
68edc3cede1fee572713a8dd
Zhe Cao
68edc3cede1fee572713a8de
Minxin Dai
68edc3cede1fee572713a8df
Ke Wang
68edc3cede1fee572713a8e0
Runzhe Wen
68edc3cede1fee572713a8e1
Yinghao Ma
68edc3cede1fee572713a8e2
Yaning Pan
claimed_verified
2025-10-14T14:24:17.947Z
66f0cf3ee3f6b40485241301
/avatars/75967726c0baa2228b13892fb8ef3d8c.svg
panyaning
user
panyaning
68edc3cede1fee572713a8e3
Sungkyun Chang
68edc3cede1fee572713a8e4
Termeh Taheri
68edc3cede1fee572713a8e5
Haiwen Xia
68edc3cede1fee572713a8e6
Christos Plachouras
68edc3cede1fee572713a8e7
Emmanouil Benetos
68edc3cede1fee572713a8e8
Yizhi Li
68edc3cede1fee572713a8e9
Ge Zhang
68edc3cede1fee572713a8ea
Jian Yang
68edc3cede1fee572713a8eb
Tianhao Peng
68edc3cede1fee572713a8ec
Zili Wang
68edc3cede1fee572713a8ed
Minghao Liu
claimed_verified
2025-10-24T16:14:39.587Z
6417d9ea8f689506e7148417
https://cdn-avatars.huggingface.co/v1/production/uploads/6417d9ea8f689506e7148417/bAYcruWNw4WvmuQcGgcwC.jpeg
minghao
user
Liam-Liu
68edc3cede1fee572713a8ee
Junran Peng
68edc3cede1fee572713a8ef
Zhaoxiang Zhang
68edc3cede1fee572713a8f0
Jiaheng Liu
OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni
  MLLMs
Recent advances in multimodal large language models (MLLMs) have demonstrated
substantial potential in video understanding. However, existing benchmarks fail
to comprehensively evaluate synergistic reasoning capabilities across audio and
visual modalities, often neglecting either one of the modalities or integrating
them in a logically inconsistent manner. To bridge this gap, we introduce
OmniVideoBench, a large-scale and rigorously designed benchmark dedicated to
assessing synergistic audio-visual understanding, with a strong emphasis on
modality complementarity and logical consistency. Specifically, OmniVideoBench
comprises 1000 high-quality question-answer(QA) pairs, each annotated with
step-by-step reasoning traces, derived from 628 diverse videos ranging from
several seconds to 30 minutes, and manually verified to guarantee complete
correctness and uniqueness. Moreover, OmniVideoBench encompasses 13 carefully
designed question types, covering temporal reasoning, spatial localization,
counting, causal inference, summarization, and beyond, thereby capturing the
essential challenges of video understanding. Evaluation of multiple MLLMs on
OmniVideoBench reveals a pronounced gap between model performance and human
reasoning, with open-source models lagging significantly behind their
closed-source counterparts, underscoring the inherent difficulty of genuine
audio-visual reasoning. We will release OmniVideoBench to foster the
development of MLLMs with stronger and more generalizable reasoning
capabilities.
OmniVideoBench is a comprehensive benchmark for evaluating audio-visual reasoning in multimodal large language models, addressing modality complementarity and logical consistency.
multimodal large language models video understanding audio-visual understanding modality complementarity logical consistency question-answer pairs reasoning traces temporal reasoning spatial localization counting causal inference summarization