Self-Correction Bench: Revealing and Addressing the Self-Correction
  Blind Spot in LLMs
Although large language models (LLMs) have become transformative, they still
make mistakes and can explore unproductive reasoning paths. Self-correction is
an important capability for a trustworthy LLM, particularly an autoregressive
LLM. While LLMs can identify error in user input, they exhibit a systematic
'Self-Correction Blind Spot' - failing to correct identical error in their own
outputs. To systematically study this phenomenon, we introduce Self-Correction
Bench, a systematic framework to measure this phenomenon through controlled
error injection at three complexity levels. Testing 14 models, we find an
average 64.5% blind spot rate. We find multiple evidences that this limitation
relates to training data composition: human training demonstrations
predominantly show error-free responses rather than error-correction sequences,
unlike RL-trained models that learn error correction through outcome feedback.
Remarkably, simply appending "Wait" reduces blind spots by 89.3%, suggesting
that the capability exists but requires activation. Our work highlights a
critical limitation in current LLMs and offers potential avenues for improving
their reliability and trustworthiness.
68678e2a11736b002cf34d5e
Ken Tsui
claimed_verified
2025-07-05T07:53:06.330Z
60e50ce5350d181892d5a636
https://cdn-avatars.huggingface.co/v1/production/uploads/60e50ce5350d181892d5a636/qjYikZtE-2kjub-6XG5Kn.jpeg
Ken Tsui
user
kenhktsui
Self-Correction Bench: Revealing and Addressing the Self-Correction
  Blind Spot in LLMs
Although large language models (LLMs) have become transformative, they still
make mistakes and can explore unproductive reasoning paths. Self-correction is
an important capability for a trustworthy LLM, particularly an autoregressive
LLM. While LLMs can identify error in user input, they exhibit a systematic
'Self-Correction Blind Spot' - failing to correct identical error in their own
outputs. To systematically study this phenomenon, we introduce Self-Correction
Bench, a systematic framework to measure this phenomenon through controlled
error injection at three complexity levels. Testing 14 models, we find an
average 64.5% blind spot rate. We find multiple evidences that this limitation
relates to training data composition: human training demonstrations
predominantly show error-free responses rather than error-correction sequences,
unlike RL-trained models that learn error correction through outcome feedback.
Remarkably, simply appending "Wait" reduces blind spots by 89.3%, suggesting
that the capability exists but requires activation. Our work highlights a
critical limitation in current LLMs and offers potential avenues for improving
their reliability and trustworthiness.
Self-Correction Bench measures the self-correction blind spot in large language models, finding that training primarily on error-free responses contributes to this issue; appending "Wait" notably improves their ability to correct errors in their outputs.
LLMs Self-Correction autoregressive LLMs Self-Correction Blind Spot Self-Correction Bench error injection human training demonstrations RL-trained models