Efficient Model Development through Fine-tuning Transfer
Modern LLMs struggle with efficient updates, as each new pretrained model
version requires repeating expensive alignment processes. This challenge also
applies to domain- or language-specific models, where fine-tuning on
specialized data must be redone for every new base model release. In this
paper, we explore the transfer of fine-tuning updates between model versions.
Specifically, we derive the diff vector from one source model version, which
represents the weight changes from fine-tuning, and apply it to the base model
of a different target version. Through empirical evaluations on various
open-weight model versions, we show that transferring diff vectors can
significantly improve the target base model, often achieving performance
comparable to its fine-tuned counterpart. For example, reusing the fine-tuning
updates from Llama 3.0 8B leads to an absolute accuracy improvement of 10.7% on
GPQA over the base Llama 3.1 8B without additional training, surpassing Llama
3.1 8B Instruct. In a multilingual model development setting, we show that this
approach can significantly increase performance on target-language tasks
without retraining, achieving an absolute improvement of 4.7% and 15.5% on
Global MMLU for Malagasy and Turkish, respectively, compared to Llama 3.1 8B
Instruct. Our controlled experiments reveal that fine-tuning transfer is most
effective when the source and target models are linearly connected in the
parameter space. Additionally, we demonstrate that fine-tuning transfer offers
a stronger and more computationally efficient starting point for further
fine-tuning. Finally, we propose an iterative recycling-then-finetuning
approach for continuous model development, which improves both efficiency and
effectiveness. Our findings suggest that fine-tuning transfer is a viable
strategy to reduce training costs while maintaining model performance.
67e49dc4376c1c8380394731
Pin-Jie Lin
claimed_verified
2025-07-07T11:26:47.938Z
61c6133ed9ca62b30bb7726e
/avatars/74471b38c62fef4233b34045868c71cf.svg
Pin-Jie Lin
user
linusdd44804
67e49dc4376c1c8380394732
Rishab Balasubramanian
67e49dc4376c1c8380394733
Fengyuan Liu
67e49dc4376c1c8380394734
Nikhil Kandpal
67e49dc4376c1c8380394735
Tu Vu
Efficient Model Development through Fine-tuning Transfer
Modern LLMs struggle with efficient updates, as each new pretrained model
version requires repeating expensive alignment processes. This challenge also
applies to domain- or language-specific models, where fine-tuning on
specialized data must be redone for every new base model release. In this
paper, we explore the transfer of fine-tuning updates between model versions.
Specifically, we derive the diff vector from one source model version, which
represents the weight changes from fine-tuning, and apply it to the base model
of a different target version. Through empirical evaluations on various
open-weight model versions, we show that transferring diff vectors can
significantly improve the target base model, often achieving performance
comparable to its fine-tuned counterpart. For example, reusing the fine-tuning
updates from Llama 3.0 8B leads to an absolute accuracy improvement of 10.7% on
GPQA over the base Llama 3.1 8B without additional training, surpassing Llama
3.1 8B Instruct. In a multilingual model development setting, we show that this
approach can significantly increase performance on target-language tasks
without retraining, achieving an absolute improvement of 4.7% and 15.5% on
Global MMLU for Malagasy and Turkish, respectively, compared to Llama 3.1 8B
Instruct. Our controlled experiments reveal that fine-tuning transfer is most
effective when the source and target models are linearly connected in the
parameter space. Additionally, we demonstrate that fine-tuning transfer offers
a stronger and more computationally efficient starting point for further
fine-tuning. Finally, we propose an iterative recycling-then-finetuning
approach for continuous model development, which improves both efficiency and
effectiveness. Our findings suggest that fine-tuning transfer is a viable
strategy to reduce training costs while maintaining model performance.
Transfer of fine-tuning updates between model versions improves performance without additional training, facilitating efficient model development and cost reduction.
LLMs fine-tuning diff vector parameter space fine-tuning transfer iterative recycling-then-finetuning