Reinforced Visual Perception with Tools
Visual reasoning, a cornerstone of human intelligence, encompasses complex
perceptual and logical processes essential for solving diverse visual problems.
While advances in computer vision have produced powerful models for various
perceptual tasks, leveraging these for general visual reasoning remains
challenging. Prior work demonstrates that augmenting LLMs with vision models
via supervised finetuning improves performance, but faces key limitations such
as expensive data generation, reliance on careful data filtering, and poor
generalization. To address these issues, we propose ReVPT to enhance
multi-modal LLMs' abilities to reason about and use visual tools through
reinforcement learning. We introduce a novel RL algorithm based on GRPO,
designed to train models to reason with a suite of four visual tools. Through
extensive experiments, we show that our method achieves state-of-the-art
performance on several perception-heavy benchmarks, including SAT, CV-Bench,
BLINK and MMStar, significantly outperforming the supervised and text-based RL
finetuning baselines. Notably, Our ReVPT-3B and ReVPT-7B outperform the
instruct models by 9.03% and 9.44% on CV-Bench. Finally, we bring to the
community new insights on RL-based visual tool-usage through extensive
ablations. Our code is available at https://github.com/ls-kelvin/REVPT.
68beda55c123124955ef6267
Zetong Zhou
68beda55c123124955ef6268
Dongping Chen
68beda55c123124955ef6269
Zixian Ma
68beda55c123124955ef626a
Zhihan Hu
68beda55c123124955ef626b
Mingyang Fu
68beda55c123124955ef626c
Sinan Wang
68beda55c123124955ef626d
Yao Wan
68beda55c123124955ef626e
Zhou Zhao
68beda55c123124955ef626f
Ranjay Krishna
Reinforced Visual Perception with Tools
Visual reasoning, a cornerstone of human intelligence, encompasses complex
perceptual and logical processes essential for solving diverse visual problems.
While advances in computer vision have produced powerful models for various
perceptual tasks, leveraging these for general visual reasoning remains
challenging. Prior work demonstrates that augmenting LLMs with vision models
via supervised finetuning improves performance, but faces key limitations such
as expensive data generation, reliance on careful data filtering, and poor
generalization. To address these issues, we propose ReVPT to enhance
multi-modal LLMs' abilities to reason about and use visual tools through
reinforcement learning. We introduce a novel RL algorithm based on GRPO,
designed to train models to reason with a suite of four visual tools. Through
extensive experiments, we show that our method achieves state-of-the-art
performance on several perception-heavy benchmarks, including SAT, CV-Bench,
BLINK and MMStar, significantly outperforming the supervised and text-based RL
finetuning baselines. Notably, Our ReVPT-3B and ReVPT-7B outperform the
instruct models by 9.03% and 9.44% on CV-Bench. Finally, we bring to the
community new insights on RL-based visual tool-usage through extensive
ablations. Our code is available at https://github.com/ls-kelvin/REVPT.
ReVPT enhances multi-modal LLMs' visual reasoning capabilities using reinforcement learning, achieving state-of-the-art performance on visual benchmarks.
LLMs vision models supervised finetuning reinforcement learning GRPO visual tools SAT CV-Bench BLINK MMStar instruct models