More Thinking, Less Seeing? Assessing Amplified Hallucination in
  Multimodal Reasoning Models
Test-time compute has empowered multimodal large language models to generate
extended reasoning chains, yielding strong performance on tasks such as
multimodal math reasoning. However, this improved reasoning ability often comes
with increased hallucination: as generations become longer, models tend to
drift away from image-grounded content and rely more heavily on language
priors. Attention analysis shows that longer reasoning chains lead to reduced
focus on visual inputs, which contributes to hallucination. To systematically
study this phenomenon, we introduce RH-AUC, a metric that quantifies how a
model's perception accuracy changes with reasoning length, allowing us to
evaluate whether the model preserves visual grounding during reasoning. We also
release RH-Bench, a diagnostic benchmark that spans a variety of multimodal
tasks, designed to assess the trade-off between reasoning ability and
hallucination. Our analysis reveals that (i) larger models typically achieve a
better balance between reasoning and perception, and (ii) this balance is
influenced more by the types and domains of training data than by its overall
volume. These findings underscore the importance of evaluation frameworks that
jointly consider both reasoning quality and perceptual fidelity.
683d8d3ef63376ea8e7bf358
Chengzhi Liu
claimed_verified
2025-06-02T13:39:21.988Z
65e71f6bcd3df9b0f6b2678b
https://cdn-avatars.huggingface.co/v1/production/uploads/65e71f6bcd3df9b0f6b2678b/8mdex6eI80TGezot_GGI3.jpeg
Chengzhi Liu
user
LCZZZZ
683d8d3ef63376ea8e7bf359
Zhongxing Xu
683d8d3ef63376ea8e7bf35a
Qingyue Wei
683d8d3ef63376ea8e7bf35b
Juncheng Wu
683d8d3ef63376ea8e7bf35c
James Zou
683d8d3ef63376ea8e7bf35d
Xin Eric Wang
claimed_verified
2025-07-24T09:15:00.197Z
64679a226192d39142245e5e
/avatars/05abee0b6317f100923936ca2099e9eb.svg
Xin Eric Wang
user
xw-eric
683d8d3ef63376ea8e7bf35e
Yuyin Zhou
683d8d3ef63376ea8e7bf35f
Sheng Liu
More Thinking, Less Seeing? Assessing Amplified Hallucination in
  Multimodal Reasoning Models
Test-time compute has empowered multimodal large language models to generate
extended reasoning chains, yielding strong performance on tasks such as
multimodal math reasoning. However, this improved reasoning ability often comes
with increased hallucination: as generations become longer, models tend to
drift away from image-grounded content and rely more heavily on language
priors. Attention analysis shows that longer reasoning chains lead to reduced
focus on visual inputs, which contributes to hallucination. To systematically
study this phenomenon, we introduce RH-AUC, a metric that quantifies how a
model's perception accuracy changes with reasoning length, allowing us to
evaluate whether the model preserves visual grounding during reasoning. We also
release RH-Bench, a diagnostic benchmark that spans a variety of multimodal
tasks, designed to assess the trade-off between reasoning ability and
hallucination. Our analysis reveals that (i) larger models typically achieve a
better balance between reasoning and perception, and (ii) this balance is
influenced more by the types and domains of training data than by its overall
volume. These findings underscore the importance of evaluation frameworks that
jointly consider both reasoning quality and perceptual fidelity.
A new metric and benchmark are introduced to evaluate multimodal large language models' ability to maintain visual grounding while performing extended reasoning, revealing that larger models and specific training data types improve this balance.
multimodal large language models extended reasoning chains hallucination attention analysis RH-AUC perception accuracy RH-Bench multimodal tasks perceptual fidelity