Directional Reasoning Injection for Fine-Tuning MLLMs
Multimodal large language models (MLLMs) are rapidly advancing, yet their
reasoning ability often lags behind that of strong text-only counterparts.
Existing methods to bridge this gap rely on supervised fine-tuning over
large-scale multimodal reasoning data or reinforcement learning, both of which
are resource-intensive. A promising alternative is model merging, which
interpolates parameters between reasoning-enhanced LLMs and multimodal
variants. However, our analysis shows that naive merging is not always a "free
lunch": its effectiveness varies drastically across model families, with some
(e.g., LLaVA, Idefics) benefiting while others (e.g., Qwen) suffer performance
degradation. To address this, we propose Directional Reasoning Injection for
Fine-Tuning (DRIFT) MLLMs, a lightweight method that transfers reasoning
knowledge in the gradient space, without destabilizing multimodal alignment.
DRIFT precomputes a reasoning prior as the parameter-space difference between
reasoning and multimodal variants, then uses it to bias gradients during
multimodal fine-tuning. This approach preserves the simplicity of standard
supervised fine-tuning pipelines while enabling efficient reasoning transfer.
Extensive experiments on multimodal reasoning benchmarks, including MathVista
and MathVerse, demonstrate that DRIFT consistently improves reasoning
performance over naive merging and supervised fine-tuning, while matching or
surpassing training-heavy methods at a fraction of the cost.
68f8a910c7cbb0ef2aa92df0
Chao Huang
claimed_verified
2025-10-22T15:42:05.689Z
67257ee0938e718957c9c100
/avatars/db1f792ee1a5d860ea4e98c11a016a1b.svg
Chao Huang
user
ChaoHuangCS
68f8a910c7cbb0ef2aa92df1
Zeliang Zhang
68f8a910c7cbb0ef2aa92df2
Jiang Liu
68f8a910c7cbb0ef2aa92df3
Ximeng Sun
68f8a910c7cbb0ef2aa92df4
Jialian Wu
68f8a910c7cbb0ef2aa92df5
Xiaodong Yu
68f8a910c7cbb0ef2aa92df6
Ze Wang
68f8a910c7cbb0ef2aa92df7
Chenliang Xu
68f8a910c7cbb0ef2aa92df8
Emad Barsoum
68f8a910c7cbb0ef2aa92df9
Zicheng Liu
Directional Reasoning Injection for Fine-Tuning MLLMs
Multimodal large language models (MLLMs) are rapidly advancing, yet their
reasoning ability often lags behind that of strong text-only counterparts.
Existing methods to bridge this gap rely on supervised fine-tuning over
large-scale multimodal reasoning data or reinforcement learning, both of which
are resource-intensive. A promising alternative is model merging, which
interpolates parameters between reasoning-enhanced LLMs and multimodal
variants. However, our analysis shows that naive merging is not always a "free
lunch": its effectiveness varies drastically across model families, with some
(e.g., LLaVA, Idefics) benefiting while others (e.g., Qwen) suffer performance
degradation. To address this, we propose Directional Reasoning Injection for
Fine-Tuning (DRIFT) MLLMs, a lightweight method that transfers reasoning
knowledge in the gradient space, without destabilizing multimodal alignment.
DRIFT precomputes a reasoning prior as the parameter-space difference between
reasoning and multimodal variants, then uses it to bias gradients during
multimodal fine-tuning. This approach preserves the simplicity of standard
supervised fine-tuning pipelines while enabling efficient reasoning transfer.
Extensive experiments on multimodal reasoning benchmarks, including MathVista
and MathVerse, demonstrate that DRIFT consistently improves reasoning
performance over naive merging and supervised fine-tuning, while matching or
surpassing training-heavy methods at a fraction of the cost.
DRIFT, a lightweight method, enhances multimodal large language models' reasoning ability by transferring knowledge in gradient space, outperforming naive merging and supervised fine-tuning with reduced computational cost.
multimodal large language models reasoning ability supervised fine-tuning reinforcement learning model merging parameter-space reasoning-enhanced LLMs multimodal variants directional reasoning injection gradient space reasoning prior multimodal alignment MathVista MathVerse