Analyzing the Effects of Supervised Fine-Tuning on Model Knowledge from
  Token and Parameter Levels
Large language models (LLMs) acquire substantial world knowledge during
pre-training, which is further shaped by post-training techniques such as
supervised fine-tuning (SFT). However, the impact of SFT on a model's knowledge
remains underexplored, limiting our ability to control knowledge change
behavior in fine-tuned models. To address this gap, we evaluate closed-book
question answering (CBQA) performance across five LLMs from the LLaMA-2 and
LLaMA-3 families. Surprisingly, models fine-tuned on 1,920 samples perform up
to 14% worse than those fine-tuned on only 240 samples. Furthermore, varying
the level of knowledge mastery in the fine-tuning data leads to performance
fluctuations of over 12%. To investigate these effects, we analyze model
behavior at both the token and parameter levels. Our analysis reveals that up
to 90% of parameter updates during SFT do not contribute to knowledge
enhancement. Restoring these updates can improve performance on the CBQA task,
depending on the characteristics of the fine-tuning data. These insights offer
practical guidance for developing fine-tuning strategies that more effectively
strengthen model knowledge.
68d205331ca7156988a8ecbd
Junjie Ye
claimed_verified
2025-09-23T02:40:25.849Z
66384be673c2c55f2ded89fa
/avatars/1d8721074f0f51fab405f81474f2035f.svg
Junjie Ye
user
Junjie-Ye
68d205331ca7156988a8ecbe
Yuming Yang
claimed_verified
2025-09-23T10:07:02.770Z
655c6b1abfb531437a54c0e6
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/V8Md2mMX83hrSowKk6qMS.jpeg
Yuming Yang
user
Umean
68d205331ca7156988a8ecbf
Yang Nan
68d205331ca7156988a8ecc0
Shuo Li
68d205331ca7156988a8ecc1
Qi Zhang
68d205331ca7156988a8ecc2
Tao Gui
68d205331ca7156988a8ecc3
Xuanjing Huang
68d205331ca7156988a8ecc4
Peng Wang
68d205331ca7156988a8ecc5
Zhongchao Shi
68d205331ca7156988a8ecc6
Jianping Fan
Analyzing the Effects of Supervised Fine-Tuning on Model Knowledge from
  Token and Parameter Levels
Large language models (LLMs) acquire substantial world knowledge during
pre-training, which is further shaped by post-training techniques such as
supervised fine-tuning (SFT). However, the impact of SFT on a model's knowledge
remains underexplored, limiting our ability to control knowledge change
behavior in fine-tuned models. To address this gap, we evaluate closed-book
question answering (CBQA) performance across five LLMs from the LLaMA-2 and
LLaMA-3 families. Surprisingly, models fine-tuned on 1,920 samples perform up
to 14% worse than those fine-tuned on only 240 samples. Furthermore, varying
the level of knowledge mastery in the fine-tuning data leads to performance
fluctuations of over 12%. To investigate these effects, we analyze model
behavior at both the token and parameter levels. Our analysis reveals that up
to 90% of parameter updates during SFT do not contribute to knowledge
enhancement. Restoring these updates can improve performance on the CBQA task,
depending on the characteristics of the fine-tuning data. These insights offer
practical guidance for developing fine-tuning strategies that more effectively
strengthen model knowledge.
Supervised fine-tuning of large language models can negatively impact closed-book question answering performance, with up to 90% of parameter updates not contributing to knowledge enhancement.
large language models pre-training supervised fine-tuning closed-book question answering token parameter updates knowledge enhancement