SAIL-RL: Guiding MLLMs in When and How to Think via Dual-Reward RL
  Tuning
We introduce SAIL-RL, a reinforcement learning (RL) post-training framework
that enhances the reasoning capabilities of multimodal large language models
(MLLMs) by teaching them when and how to think. Existing approaches are limited
by outcome-only supervision, which rewards correct answers without ensuring
sound reasoning, and by uniform thinking strategies, which often lead to
overthinking on simple tasks and underthinking on complex ones. SAIL-RL
addresses these challenges with a dual reward system: the Thinking Reward,
which evaluates reasoning quality through factual grounding, logical coherence,
and answer consistency, and the Judging Reward, which adaptively determines
whether deep reasoning or direct answering is appropriate. Experiments on the
state-of-the-art SAIL-VL2 show that SAIL-RL improves reasoning and multimodal
understanding benchmarks at both 4B and 8B scales, achieving competitive
performance against commercial closed-source models such as GPT-4o, and
substantially reduces hallucinations, establishing it as a principled framework
for building more reliable and adaptive MLLMs. The code will be available at
https://github.com/BytedanceDouyinContent/SAIL-RL.
690c18d960494e4fa7675677
Fangxun Shu
690c18d960494e4fa7675678
Yongjie Ye
690c18d960494e4fa7675679
Yue Liao
690c18d960494e4fa767567a
Zijian Kang
690c18d960494e4fa767567b
Weijie Yin
690c18d960494e4fa767567c
Jiacong Wang
690c18d960494e4fa767567d
Xiao Liang
690c18d960494e4fa767567e
Shuicheng Yan
690c18d960494e4fa767567f
Chao Feng
SAIL-RL: Guiding MLLMs in When and How to Think via Dual-Reward RL
  Tuning
We introduce SAIL-RL, a reinforcement learning (RL) post-training framework
that enhances the reasoning capabilities of multimodal large language models
(MLLMs) by teaching them when and how to think. Existing approaches are limited
by outcome-only supervision, which rewards correct answers without ensuring
sound reasoning, and by uniform thinking strategies, which often lead to
overthinking on simple tasks and underthinking on complex ones. SAIL-RL
addresses these challenges with a dual reward system: the Thinking Reward,
which evaluates reasoning quality through factual grounding, logical coherence,
and answer consistency, and the Judging Reward, which adaptively determines
whether deep reasoning or direct answering is appropriate. Experiments on the
state-of-the-art SAIL-VL2 show that SAIL-RL improves reasoning and multimodal
understanding benchmarks at both 4B and 8B scales, achieving competitive
performance against commercial closed-source models such as GPT-4o, and
substantially reduces hallucinations, establishing it as a principled framework
for building more reliable and adaptive MLLMs. The code will be available at
https://github.com/BytedanceDouyinContent/SAIL-RL.
SAIL-RL enhances multimodal large language models' reasoning capabilities through a dual reward system, improving benchmarks and reducing hallucinations.
reinforcement learning multimodal large language models outcome-only supervision Thinking Reward Judging Reward factual grounding logical coherence answer consistency deep reasoning direct answering hallucinations