LIFT the Veil for the Truth: Principal Weights Emerge after Rank
  Reduction for Reasoning-Focused Supervised Fine-Tuning
Recent studies have shown that supervised fine-tuning of LLMs on a small
number of high-quality datasets can yield strong reasoning capabilities.
However, full fine-tuning (Full FT), while powerful, is computationally
expensive and susceptible to overfitting and catastrophic forgetting,
particularly when data is limited. Sparse fine-tuning, which previously
achieved notable success by updating only a small subset of model parameters,
offers a promising trade-off between efficiency and effectiveness. Yet, it has
lagged behind in the LLM era due to the difficulty of identifying parameters
truly critical for reasoning. In this work, we state that weights with the
largest magnitude after low-rank approximation are critical weights for
fine-tuning, which we call Principal Weights. Surprisingly, while
magnitude-based sparse fine-tuning performs poorly as a baseline on LLM
fine-tuning, it becomes highly effective after rank reduction. These insights
motivate our method: Low-rank Informed Sparse Fine-Tuning (LIFT). LIFT only
updates the top 5% Principal Weights throughout training and consistently
achieves better performance on reasoning tasks than Full FT, while maintaining
memory efficiency on par with popular parameter-efficient fine-tuning methods.
In addition to strong performance on target domains such as arithmetic
reasoning, LIFT also retains up to 20% more source-domain knowledge, compared
to Full FT and LoRA. Our code is available at:
https://github.com/zihanghliu/LIFT.
683ec2d53c81cc903bbb418c
Zihang Liu
683ec2d53c81cc903bbb418d
Tianyu Pang
683ec2d53c81cc903bbb418e
Oleg Balabanov
683ec2d53c81cc903bbb418f
Chaoqun Yang
683ec2d53c81cc903bbb4190
Tianjin Huang
683ec2d53c81cc903bbb4191
Lu Yin
683ec2d53c81cc903bbb4192
Yaoqing Yang
683ec2d53c81cc903bbb4193
Shiwei Liu
LIFT the Veil for the Truth: Principal Weights Emerge after Rank
  Reduction for Reasoning-Focused Supervised Fine-Tuning
Recent studies have shown that supervised fine-tuning of LLMs on a small
number of high-quality datasets can yield strong reasoning capabilities.
However, full fine-tuning (Full FT), while powerful, is computationally
expensive and susceptible to overfitting and catastrophic forgetting,
particularly when data is limited. Sparse fine-tuning, which previously
achieved notable success by updating only a small subset of model parameters,
offers a promising trade-off between efficiency and effectiveness. Yet, it has
lagged behind in the LLM era due to the difficulty of identifying parameters
truly critical for reasoning. In this work, we state that weights with the
largest magnitude after low-rank approximation are critical weights for
fine-tuning, which we call Principal Weights. Surprisingly, while
magnitude-based sparse fine-tuning performs poorly as a baseline on LLM
fine-tuning, it becomes highly effective after rank reduction. These insights
motivate our method: Low-rank Informed Sparse Fine-Tuning (LIFT). LIFT only
updates the top 5% Principal Weights throughout training and consistently
achieves better performance on reasoning tasks than Full FT, while maintaining
memory efficiency on par with popular parameter-efficient fine-tuning methods.
In addition to strong performance on target domains such as arithmetic
reasoning, LIFT also retains up to 20% more source-domain knowledge, compared
to Full FT and LoRA. Our code is available at:
https://github.com/zihanghliu/LIFT.
Leveraging low-rank approximation to identify critical weights for sparse fine-tuning of large language models enhances performance and efficiency compared to full fine-tuning.
LLMs supervised fine-tuning full fine-tuning sparse fine-tuning low-rank approximation Principal Weights Low-rank Informed Sparse Fine-Tuning LIFT memory efficiency parameter-efficient fine-tuning reasoning tasks arithmetic reasoning source-domain knowledge LoRA