VGR: Visual Grounded Reasoning
In the field of multimodal chain-of-thought (CoT) reasoning, existing
approaches predominantly rely on reasoning on pure language space, which
inherently suffers from language bias and is largely confined to math or
science domains. This narrow focus limits their ability to handle complex
visual reasoning tasks that demand comprehensive understanding of image
details. To address these limitations, this paper introduces VGR, a novel
reasoning multimodal large language model (MLLM) with enhanced fine-grained
visual perception capabilities. Unlike traditional MLLMs that answer the
question or reasoning solely on the language space, our VGR first detects
relevant regions that may help to solve problems, and then provides precise
answers based on replayed image regions. To achieve this, we conduct a
large-scale SFT dataset called VGR -SFT that contains reasoning data with mixed
vision grounding and language deduction. The inference pipeline of VGR allows
the model to choose bounding boxes for visual reference and a replay stage is
introduced to integrates the corresponding regions into the reasoning process,
enhancing multimodel comprehension. Experiments on the LLaVA-NeXT-7B baseline
show that VGR achieves superior performance on multi-modal benchmarks requiring
comprehensive image detail understanding. Compared to the baseline, VGR uses
only 30\% of the image token count while delivering scores of +4.1 on MMStar,
+7.1 on AI2D, and a +12.9 improvement on ChartQA.
684fc67360b4a34dbe007b3c
Jiacong Wang
claimed_verified
2025-06-17T07:22:25.360Z
64d201b1c2bd235422fb1d14
/avatars/e50581aa66391cedae94e116e759b9ec.svg
wang
user
stormthunder
684fc67360b4a34dbe007b3d
Zijiang Kang
684fc67360b4a34dbe007b3e
Haochen Wang
claimed_verified
2025-07-11T08:09:00.648Z
6499809cf19fc795e7724e43
/avatars/4b3adce8c85e2f3ef05318ded6c89c3e.svg
HaochenWang
user
HaochenWang
684fc67360b4a34dbe007b3f
Haiyong Jiang
684fc67360b4a34dbe007b40
Jiawen Li
684fc67360b4a34dbe007b41
Bohong Wu
claimed_verified
2025-06-16T09:52:40.461Z
64722a616facfb01d8ae8349
/avatars/1dce23ae5ebd9996770cf5efe910b857.svg
Wu Bohong
user
bongbohong
684fc67360b4a34dbe007b42
Ya Wang
684fc67360b4a34dbe007b43
Jiao Ran
684fc67360b4a34dbe007b44
Xiao Liang
684fc67360b4a34dbe007b45
Chao Feng
684fc67360b4a34dbe007b46
Jun Xiao
VGR: Visual Grounded Reasoning
In the field of multimodal chain-of-thought (CoT) reasoning, existing
approaches predominantly rely on reasoning on pure language space, which
inherently suffers from language bias and is largely confined to math or
science domains. This narrow focus limits their ability to handle complex
visual reasoning tasks that demand comprehensive understanding of image
details. To address these limitations, this paper introduces VGR, a novel
reasoning multimodal large language model (MLLM) with enhanced fine-grained
visual perception capabilities. Unlike traditional MLLMs that answer the
question or reasoning solely on the language space, our VGR first detects
relevant regions that may help to solve problems, and then provides precise
answers based on replayed image regions. To achieve this, we conduct a
large-scale SFT dataset called VGR -SFT that contains reasoning data with mixed
vision grounding and language deduction. The inference pipeline of VGR allows
the model to choose bounding boxes for visual reference and a replay stage is
introduced to integrates the corresponding regions into the reasoning process,
enhancing multimodel comprehension. Experiments on the LLaVA-NeXT-7B baseline
show that VGR achieves superior performance on multi-modal benchmarks requiring
comprehensive image detail understanding. Compared to the baseline, VGR uses
only 30\% of the image token count while delivering scores of +4.1 on MMStar,
+7.1 on AI2D, and a +12.9 improvement on ChartQA.
VGR, a novel multimodal large language model, improves visual reasoning by detecting relevant image regions and integrating them into the reasoning process, outperforming existing models on multimodal benchmarks with reduced resource usage.
multimodal chain-of-thought reasoning MLLM VGR enhanced fine-grained visual perception SFT dataset bounding boxes replay stage multimodal comprehension LLaVA-NeXT-7B MMStar AI2D ChartQA