Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models
Large language models (LLMs) excel at reasoning, yet post-training remains
critical for aligning their behavior with task goals. Existing reinforcement
learning (RL) methods often depend on costly human annotations or external
reward models. We propose Reinforcement Learning via Self-Confidence (RLSC),
which uses the model's own confidence as reward signals-eliminating the need
for labels, preference models, or reward engineering. Applied to
Qwen2.5-Math-7B with only 16 samples per question and 10 or 20 training steps,
RLSC improves accuracy by +13.4% on AIME2024, +21.2% on MATH500, +21.7% on
Minerva Math, +20.8% on Olympiadbench, and +9.7% on AMC23. RLSC provides a
simple, scalable post-training method for inference models, requiring only a
small number of samples and unlabelled supervision.
68492dcf42e4f9106973f437
Pengyi Li
extracted_pending
2025-06-11T07:18:40.287Z
6734e315c1aadce903f73aea
/avatars/95d95c49419372debc201cb63c354b86.svg
Li Pengyi
user
LiPengyi29
68492dcf42e4f9106973f438
Matvey Skripkin
claimed_verified
2025-06-12T12:42:44.217Z
6626c5d0a329de26e7eb16fa
/avatars/124f389f768fb666efd8b5a9b54c3b3c.svg
Matvey Skripkin
user
barracuda049
68492dcf42e4f9106973f439
Alexander Zubrey
claimed_verified
2025-06-14T06:24:41.423Z
65f950b448d4f7baa4ead4c3
https://cdn-avatars.huggingface.co/v1/production/uploads/65f950b448d4f7baa4ead4c3/ur4k8lgH8M5QMF_XckzbO.jpeg
Alexander
user
cocount42
68492dcf42e4f9106973f43a
Andrey Kuznetsov
claimed_verified
2025-06-12T12:42:42.242Z
643984dceb7c5616ef3f5d54
https://cdn-avatars.huggingface.co/v1/production/uploads/643984dceb7c5616ef3f5d54/10JRkblrRIEVci6UJwvPz.jpeg
Andrey Kuznetsov
user
kuznetsoffandrey
68492dcf42e4f9106973f43b
Ivan Oseledets
claimed_verified
2025-06-12T12:42:40.519Z
6169a581d05945bfd8718dfa
/avatars/1892ab06a7ddb557232777de3cbec470.svg
Ivan Oseledets
user
oseledets
Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models
Large language models (LLMs) excel at reasoning, yet post-training remains
critical for aligning their behavior with task goals. Existing reinforcement
learning (RL) methods often depend on costly human annotations or external
reward models. We propose Reinforcement Learning via Self-Confidence (RLSC),
which uses the model's own confidence as reward signals-eliminating the need
for labels, preference models, or reward engineering. Applied to
Qwen2.5-Math-7B with only 16 samples per question and 10 or 20 training steps,
RLSC improves accuracy by +13.4% on AIME2024, +21.2% on MATH500, +21.7% on
Minerva Math, +20.8% on Olympiadbench, and +9.7% on AMC23. RLSC provides a
simple, scalable post-training method for inference models, requiring only a
small number of samples and unlabelled supervision.
Reinforcement Learning via Self-Confidence (RLSC) improves large language model accuracy using the model's confidence as a reward signal, eliminating the need for human labels or reward engineering.
Reinforcement Learning Large language models self-confidence RLSC