Beyond Log Likelihood: Probability-Based Objectives for Supervised
  Fine-Tuning across the Model Capability Continuum
Supervised fine-tuning (SFT) is the standard approach for post-training large
language models (LLMs), yet it often shows limited generalization. We trace
this limitation to its default training objective: negative log likelihood
(NLL). While NLL is classically optimal when training from scratch,
post-training operates in a different paradigm and could violate its optimality
assumptions, where models already encode task-relevant priors and supervision
can be long and noisy. To this end, we study a general family of
probability-based objectives and characterize their effectiveness under
different conditions. Through comprehensive experiments and extensive ablation
studies across 7 model backbones, 14 benchmarks, and 3 domains, we uncover a
critical dimension that governs objective behavior: the model-capability
continuum. Near the model-strong end, prior-leaning objectives that downweight
low-probability tokens (e.g., -p, -p^{10}, thresholded variants)
consistently outperform NLL; toward the model-weak end, NLL dominates; in
between, no single objective prevails. Our theoretical analysis further
elucidates how objectives trade places across the continuum, providing a
principled foundation for adapting objectives to model capability. Our code is
available at https://github.com/GaotangLi/Beyond-Log-Likelihood.
68dddc4c6024653e8a3ed083
Gaotang Li
claimed_verified
2025-10-08T08:03:29.456Z
654d784d71a30c4bca09a319
/avatars/ab9f93122903ccd662267232bab30ad8.svg
Gaotang Li
user
gaotang
68dddc4c6024653e8a3ed084
Ruizhong Qiu
68dddc4c6024653e8a3ed085
Xiusi Chen
68dddc4c6024653e8a3ed086
Heng Ji
68dddc4c6024653e8a3ed087
Hanghang Tong
Beyond Log Likelihood: Probability-Based Objectives for Supervised
  Fine-Tuning across the Model Capability Continuum
Supervised fine-tuning (SFT) is the standard approach for post-training large
language models (LLMs), yet it often shows limited generalization. We trace
this limitation to its default training objective: negative log likelihood
(NLL). While NLL is classically optimal when training from scratch,
post-training operates in a different paradigm and could violate its optimality
assumptions, where models already encode task-relevant priors and supervision
can be long and noisy. To this end, we study a general family of
probability-based objectives and characterize their effectiveness under
different conditions. Through comprehensive experiments and extensive ablation
studies across 7 model backbones, 14 benchmarks, and 3 domains, we uncover a
critical dimension that governs objective behavior: the model-capability
continuum. Near the model-strong end, prior-leaning objectives that downweight
low-probability tokens (e.g., -p, -p^{10}, thresholded variants)
consistently outperform NLL; toward the model-weak end, NLL dominates; in
between, no single objective prevails. Our theoretical analysis further
elucidates how objectives trade places across the continuum, providing a
principled foundation for adapting objectives to model capability. Our code is
available at https://github.com/GaotangLi/Beyond-Log-Likelihood.
Research identifies probability-based objectives that outperform negative log likelihood for fine-tuning large language models, depending on model capability.
supervised fine-tuning large language models negative log likelihood probability-based objectives model-capability continuum prior-leaning objectives