Revisiting Model Interpolation for Efficient Reasoning
Model merging, typically on Instruct and Thinking models, has shown
remarkable performance for efficient reasoning. In this paper, we
systematically revisit the simplest merging method that interpolates two
weights directly. Particularly, we observe that model interpolation follows a
three-stage evolutionary paradigm with distinct behaviors on the reasoning
trajectory. These dynamics provide a principled guide for navigating the
performance-cost trade-off. Empirical results demonstrate that a strategically
interpolated model surprisingly surpasses sophisticated model merging baselines
on both efficiency and effectiveness. We further validate our findings with
extensive ablation studies on model layers, modules, and decoding strategies.
Ultimately, this work demystifies model interpolation and offers a practical
framework for crafting models with precisely targeted reasoning capabilities.
Code is available at https://github.com/wutaiqiang/MI{Github}.
68ef0dce486b78128f0e342f
Taiqiang Wu
claimed_verified
2025-10-16T06:41:49.305Z
6621cea88850e38ffbb1854f
https://cdn-avatars.huggingface.co/v1/production/uploads/6621cea88850e38ffbb1854f/LeytEEjSwnnqB-zFN1Tgt.jpeg
Taki WU
user
taki555
68ef0dce486b78128f0e3430
Runming Yang
68ef0dce486b78128f0e3431
Tao Liu
68ef0dce486b78128f0e3432
Jiahao Wang
68ef0dce486b78128f0e3433
Ngai Wong
Revisiting Model Interpolation for Efficient Reasoning
Model merging, typically on Instruct and Thinking models, has shown
remarkable performance for efficient reasoning. In this paper, we
systematically revisit the simplest merging method that interpolates two
weights directly. Particularly, we observe that model interpolation follows a
three-stage evolutionary paradigm with distinct behaviors on the reasoning
trajectory. These dynamics provide a principled guide for navigating the
performance-cost trade-off. Empirical results demonstrate that a strategically
interpolated model surprisingly surpasses sophisticated model merging baselines
on both efficiency and effectiveness. We further validate our findings with
extensive ablation studies on model layers, modules, and decoding strategies.
Ultimately, this work demystifies model interpolation and offers a practical
framework for crafting models with precisely targeted reasoning capabilities.
Code is available at https://github.com/wutaiqiang/MI{Github}.