Self-Taught Self-Correction for Small Language Models
Although large language models (LLMs) have achieved remarkable performance
across various tasks, they remain prone to errors. A key challenge is enabling
them to self-correct. While prior research has relied on external tools or
large proprietary models, this work explores self-correction in small language
models (SLMs) through iterative fine-tuning using solely self-generated data.
We introduce the Self-Taught Self-Correction (STaSC) algorithm, which
incorporates multiple algorithmic design choices. Experimental results on a
question-answering task demonstrate that STaSC effectively learns
self-correction, leading to significant performance improvements. Our analysis
further provides insights into the mechanisms of self-correction and the impact
of different design choices on learning dynamics and overall performance. To
support future research, we release our user-friendly codebase and lightweight
models.
67d2c04b5d0737ce36ec58c8
Viktor Moskvoretskii
extracted_confirmed
2025-03-14T07:26:40.686Z
63bbfd74141c7d395c471768
https://cdn-avatars.huggingface.co/v1/production/uploads/1673264437106-noauth.jpeg
Viktor Moskvoretskii
user
VityaVitalich
67d2c04b5d0737ce36ec58c9
Chris Biemann
67d2c04b5d0737ce36ec58ca
Irina Nikishina
Self-Taught Self-Correction for Small Language Models
Although large language models (LLMs) have achieved remarkable performance
across various tasks, they remain prone to errors. A key challenge is enabling
them to self-correct. While prior research has relied on external tools or
large proprietary models, this work explores self-correction in small language
models (SLMs) through iterative fine-tuning using solely self-generated data.
We introduce the Self-Taught Self-Correction (STaSC) algorithm, which
incorporates multiple algorithmic design choices. Experimental results on a
question-answering task demonstrate that STaSC effectively learns
self-correction, leading to significant performance improvements. Our analysis
further provides insights into the mechanisms of self-correction and the impact
of different design choices on learning dynamics and overall performance. To
support future research, we release our user-friendly codebase and lightweight
models.
The Self-Taught Self-Correction (STaSC) algorithm improves the performance of small language models through iterative self-correction using self-generated data.
Large language models small language models iterative fine-tuning self-correction Self-Taught Self-Correction (STaSC) learning dynamics codebase lightweight models