Distillation and Refinement of Reasoning in Small Language Models for
  Document Re-ranking
We present a novel approach for training small language models for
reasoning-intensive document ranking that combines knowledge distillation with
reinforcement learning optimization. While existing methods often rely on
expensive human annotations or large black-box language models, our methodology
leverages web data and a teacher LLM to automatically generate high-quality
training examples with relevance explanations. By framing document ranking as a
reinforcement learning problem and incentivizing explicit reasoning
capabilities, we train a compact 3B parameter language model that achieves
state-of-the-art performance on the BRIGHT benchmark. Our model ranks third on
the leaderboard while using substantially fewer parameters than other
approaches, outperforming models that are over 20 times larger. Through
extensive experiments, we demonstrate that generating explanations during
inference, rather than directly predicting relevance scores, enables more
effective reasoning with smaller language models. The self-supervised nature of
our method offers a scalable and interpretable solution for modern information
retrieval systems.
67f4a21bb1e7ca3656ab6195
Chris Samarinas
claimed_verified
2025-04-09T07:37:46.626Z
6216cfcd6a99db28e0b3155a
https://cdn-avatars.huggingface.co/v1/production/uploads/1676512583831-6216cfcd6a99db28e0b3155a.jpeg
Chris Samarinas
user
algoprog
67f4a21bb1e7ca3656ab6196
Hamed Zamani
Distillation and Refinement of Reasoning in Small Language Models for
  Document Re-ranking
We present a novel approach for training small language models for
reasoning-intensive document ranking that combines knowledge distillation with
reinforcement learning optimization. While existing methods often rely on
expensive human annotations or large black-box language models, our methodology
leverages web data and a teacher LLM to automatically generate high-quality
training examples with relevance explanations. By framing document ranking as a
reinforcement learning problem and incentivizing explicit reasoning
capabilities, we train a compact 3B parameter language model that achieves
state-of-the-art performance on the BRIGHT benchmark. Our model ranks third on
the leaderboard while using substantially fewer parameters than other
approaches, outperforming models that are over 20 times larger. Through
extensive experiments, we demonstrate that generating explanations during
inference, rather than directly predicting relevance scores, enables more
effective reasoning with smaller language models. The self-supervised nature of
our method offers a scalable and interpretable solution for modern information
retrieval systems.
A novel approach combines knowledge distillation and reinforcement learning to train small, efficient language models for document ranking using web data and teacher LLMs, achieving state-of-the-art performance with fewer parameters.
knowledge distillation reinforcement learning document ranking web data teacher LLM BRIGHT benchmark state-of-the-art performance parameter efficiency relevance explanations self-supervised learning information retrieval systems