Benchmarking Multimodal Mathematical Reasoning with Explicit Visual
  Dependency
Recent advancements in Large Vision-Language Models (LVLMs) have
significantly enhanced their ability to integrate visual and linguistic
information, achieving near-human proficiency in tasks like object recognition,
captioning, and visual question answering. However, current benchmarks
typically focus on knowledge-centric evaluations that assess domain-specific
expertise, often neglecting the core ability to reason about fundamental
mathematical elements and visual concepts. We identify a gap in evaluating
elementary-level math problems, which rely on explicit visual
dependencies-requiring models to discern, integrate, and reason across multiple
images while incorporating commonsense knowledge, all of which are crucial for
advancing toward broader AGI capabilities. To address this gap, we introduce
VCBENCH, a comprehensive benchmark for multimodal mathematical reasoning with
explicit visual dependencies. VCBENCH includes 1,720 problems across six
cognitive domains, featuring 6,697 images (averaging 3.9 per question) to
ensure multi-image reasoning. We evaluate 26 state-of-the-art LVLMs on VCBENCH,
revealing substantial performance disparities, with even the top models unable
to exceed 50% accuracy. Our findings highlight the ongoing challenges in
visual-mathematical integration and suggest avenues for future LVLM
advancements.
68106d0daa36fca4aeebb34a
Zhikai Wang
claimed_verified
2025-04-29T07:57:55.121Z
67132d16659c7cd704867365
/avatars/cab345716718965b87e29ac248fbc7e4.svg
zhikai wang
user
cloudcatcher2
68106d0daa36fca4aeebb34b
Jiashuo Sun
admin_assigned
2025-04-29T12:25:04.139Z
64527f701a57e1179c1c3693
/avatars/25b2632d7aa9ce26d5d4924ecb00c4f4.svg
Jiashuo Sun
user
gasolsun
68106d0daa36fca4aeebb34c
Wenqi Zhang
68106d0daa36fca4aeebb34d
Zhiqiang Hu
admin_assigned
2025-04-29T12:25:37.087Z
637f228152229c63921119c3
https://cdn-avatars.huggingface.co/v1/production/uploads/637f228152229c63921119c3/acwXorra1r9_7i3KlBFjS.jpeg
Zhiqiang Hu
user
Zhiqiang007
68106d0daa36fca4aeebb34e
Xin Li
68106d0daa36fca4aeebb34f
Fan Wang
68106d0daa36fca4aeebb350
Deli Zhao
Benchmarking Multimodal Mathematical Reasoning with Explicit Visual
  Dependency
Recent advancements in Large Vision-Language Models (LVLMs) have
significantly enhanced their ability to integrate visual and linguistic
information, achieving near-human proficiency in tasks like object recognition,
captioning, and visual question answering. However, current benchmarks
typically focus on knowledge-centric evaluations that assess domain-specific
expertise, often neglecting the core ability to reason about fundamental
mathematical elements and visual concepts. We identify a gap in evaluating
elementary-level math problems, which rely on explicit visual
dependencies-requiring models to discern, integrate, and reason across multiple
images while incorporating commonsense knowledge, all of which are crucial for
advancing toward broader AGI capabilities. To address this gap, we introduce
VCBENCH, a comprehensive benchmark for multimodal mathematical reasoning with
explicit visual dependencies. VCBENCH includes 1,720 problems across six
cognitive domains, featuring 6,697 images (averaging 3.9 per question) to
ensure multi-image reasoning. We evaluate 26 state-of-the-art LVLMs on VCBENCH,
revealing substantial performance disparities, with even the top models unable
to exceed 50% accuracy. Our findings highlight the ongoing challenges in
visual-mathematical integration and suggest avenues for future LVLM
advancements.
VCBENCH evaluates multimodal mathematical reasoning by assessing LVLMs on a benchmark with explicit visual dependencies and multi-image reasoning tasks.
Large Vision-Language Models LVLMs multimodal mathematical reasoning explicit visual dependencies