MMRefine: Unveiling the Obstacles to Robust Refinement in Multimodal
  Large Language Models
This paper introduces MMRefine, a MultiModal Refinement benchmark designed to
evaluate the error refinement capabilities of Multimodal Large Language Models
(MLLMs). As the emphasis shifts toward enhancing reasoning during inference,
MMRefine provides a framework that evaluates MLLMs' abilities to detect and
correct errors across six distinct scenarios beyond just comparing final
accuracy before and after refinement. Furthermore, the benchmark analyzes the
refinement performance by categorizing errors into six error types. Experiments
with various open and closed MLLMs reveal bottlenecks and factors impeding
refinement performance, highlighting areas for improvement in effective
reasoning enhancement. Our code and dataset are publicly available at
https://github.com/naver-ai/MMRefine.
6843d95d3ec10bdd8ab4d81c
Gio Paik
claimed_verified
2025-06-07T13:33:52.701Z
66976a7fa7fd582ae754850e
https://cdn-avatars.huggingface.co/v1/production/uploads/66976a7fa7fd582ae754850e/jQXfIqKh1FYGxjyFwSmSY.jpeg
Gio Paik
user
skyil7
6843d95d3ec10bdd8ab4d81d
Geewook Kim
6843d95d3ec10bdd8ab4d81e
Jinbae Im
MMRefine: Unveiling the Obstacles to Robust Refinement in Multimodal
  Large Language Models
This paper introduces MMRefine, a MultiModal Refinement benchmark designed to
evaluate the error refinement capabilities of Multimodal Large Language Models
(MLLMs). As the emphasis shifts toward enhancing reasoning during inference,
MMRefine provides a framework that evaluates MLLMs' abilities to detect and
correct errors across six distinct scenarios beyond just comparing final
accuracy before and after refinement. Furthermore, the benchmark analyzes the
refinement performance by categorizing errors into six error types. Experiments
with various open and closed MLLMs reveal bottlenecks and factors impeding
refinement performance, highlighting areas for improvement in effective
reasoning enhancement. Our code and dataset are publicly available at
https://github.com/naver-ai/MMRefine.
MMRefine evaluates the error refinement capabilities of Multimodal Large Language Models through a benchmark that categorizes errors and identifies performance bottlenecks.
MultiModal Refinement Multimodal Large Language Models error refinement error types bottleneck analysis