Sailing AI by the Stars: A Survey of Learning from Rewards in
  Post-Training and Test-Time Scaling of Large Language Models
Recent developments in Large Language Models (LLMs) have shifted from
pre-training scaling to post-training and test-time scaling. Across these
developments, a key unified paradigm has arisen: Learning from Rewards, where
reward signals act as the guiding stars to steer LLM behavior. It has
underpinned a wide range of prevalent techniques, such as reinforcement
learning (in RLHF, DPO, and GRPO), reward-guided decoding, and post-hoc
correction. Crucially, this paradigm enables the transition from passive
learning from static data to active learning from dynamic feedback. This endows
LLMs with aligned preferences and deep reasoning capabilities. In this survey,
we present a comprehensive overview of the paradigm of learning from rewards.
We categorize and analyze the strategies under this paradigm across training,
inference, and post-inference stages. We further discuss the benchmarks for
reward models and the primary applications. Finally we highlight the challenges
and future directions. We maintain a paper collection at
https://github.com/bobxwu/learning-from-rewards-llm-papers.
6821acfb2808328b91c0e365
Xiaobao Wu
admin_assigned
2025-05-12T13:17:15.826Z
64cb02869e30a46f7b80b355
/avatars/81ce4ba78826b54f0e1b53eeaff87ee6.svg
Xiaobao Wu
user
bobxwu
Sailing AI by the Stars: A Survey of Learning from Rewards in
  Post-Training and Test-Time Scaling of Large Language Models
Recent developments in Large Language Models (LLMs) have shifted from
pre-training scaling to post-training and test-time scaling. Across these
developments, a key unified paradigm has arisen: Learning from Rewards, where
reward signals act as the guiding stars to steer LLM behavior. It has
underpinned a wide range of prevalent techniques, such as reinforcement
learning (in RLHF, DPO, and GRPO), reward-guided decoding, and post-hoc
correction. Crucially, this paradigm enables the transition from passive
learning from static data to active learning from dynamic feedback. This endows
LLMs with aligned preferences and deep reasoning capabilities. In this survey,
we present a comprehensive overview of the paradigm of learning from rewards.
We categorize and analyze the strategies under this paradigm across training,
inference, and post-inference stages. We further discuss the benchmarks for
reward models and the primary applications. Finally we highlight the challenges
and future directions. We maintain a paper collection at
https://github.com/bobxwu/learning-from-rewards-llm-papers.
A survey of the Learning from Rewards paradigm in Large Language Models, encompassing reinforcement learning techniques and reward-guided decoding strategies, enabling dynamic feedback and aligned preferences.
Learning from Rewards reinforcement learning RLHF DPO GRPO reward-guided decoding post-hoc correction active learning reward models dynamic feedback aligned preferences