Vision Matters: Simple Visual Perturbations Can Boost Multimodal Math
  Reasoning
Despite the rapid progress of multimodal large language models (MLLMs), they
have largely overlooked the importance of visual processing. In a simple yet
revealing experiment, we interestingly find that language-only models, when
provided with image captions, can achieve comparable or even better performance
than MLLMs that consume raw visual inputs. This suggests that current MLLMs may
generate accurate visual descriptions but fail to effectively integrate them
during reasoning. Motivated by this, we propose a simple visual perturbation
framework that enhances perceptual robustness without requiring algorithmic
modifications or additional training data. Our approach introduces three
targeted perturbations: distractor concatenation, dominance-preserving mixup,
and random rotation, that can be easily integrated into existing post-training
pipelines including SFT, DPO, and GRPO. Through extensive experiments across
multiple datasets, we demonstrate consistent improvements in mathematical
reasoning performance, with gains comparable to those achieved through
algorithmic changes. Additionally, we achieve competitive performance among
open-source 7B RL-tuned models by training Qwen2.5-VL-7B with visual
perturbation. Through comprehensive ablation studies, we analyze the
effectiveness of different perturbation strategies, revealing that each
perturbation type contributes uniquely to different aspects of visual
reasoning. Our findings highlight the critical role of visual perturbation in
multimodal mathematical reasoning: better reasoning begins with better seeing.
Our code is available at https://github.com/YutingLi0606/Vision-Matters.
684ae230dbd21a9cc27b10ba
Yuting Li
claimed_verified
2025-06-13T14:59:36.571Z
66e3e801ec0343c253da37c3
/avatars/72fc67fa2b25473fd3acba1fbd132f62.svg
Yuting Li
user
Yuting6
684ae230dbd21a9cc27b10bb
Lai Wei
claimed_verified
2025-06-13T07:40:16.040Z
64a16b1aeacb4b50ba1c889d
https://cdn-avatars.huggingface.co/v1/production/uploads/64a16b1aeacb4b50ba1c889d/EhWoBGL6LlFGIVTUsp2F4.jpeg
Lai Wei
user
WaltonFuture
684ae230dbd21a9cc27b10bc
Kaipeng Zheng
684ae230dbd21a9cc27b10bd
Jingyuan Huang
684ae230dbd21a9cc27b10be
Linghe Kong
684ae230dbd21a9cc27b10bf
Lichao Sun
684ae230dbd21a9cc27b10c0
Weiran Huang
Vision Matters: Simple Visual Perturbations Can Boost Multimodal Math
  Reasoning
Despite the rapid progress of multimodal large language models (MLLMs), they
have largely overlooked the importance of visual processing. In a simple yet
revealing experiment, we interestingly find that language-only models, when
provided with image captions, can achieve comparable or even better performance
than MLLMs that consume raw visual inputs. This suggests that current MLLMs may
generate accurate visual descriptions but fail to effectively integrate them
during reasoning. Motivated by this, we propose a simple visual perturbation
framework that enhances perceptual robustness without requiring algorithmic
modifications or additional training data. Our approach introduces three
targeted perturbations: distractor concatenation, dominance-preserving mixup,
and random rotation, that can be easily integrated into existing post-training
pipelines including SFT, DPO, and GRPO. Through extensive experiments across
multiple datasets, we demonstrate consistent improvements in mathematical
reasoning performance, with gains comparable to those achieved through
algorithmic changes. Additionally, we achieve competitive performance among
open-source 7B RL-tuned models by training Qwen2.5-VL-7B with visual
perturbation. Through comprehensive ablation studies, we analyze the
effectiveness of different perturbation strategies, revealing that each
perturbation type contributes uniquely to different aspects of visual
reasoning. Our findings highlight the critical role of visual perturbation in
multimodal mathematical reasoning: better reasoning begins with better seeing.
Our code is available at https://github.com/YutingLi0606/Vision-Matters.
Visual perturbation enhances multimodal large language models' mathematical reasoning without algorithmic modifications or additional data.
multimodal large language models visual processing image captions perceptual robustness distractor concatenation dominance-preserving mixup random rotation post-training pipelines mathematical reasoning open-source RL-tuned models Qwen2.5-VL-7B ablation studies visual reasoning