Feedback Friction: LLMs Struggle to Fully Incorporate External Feedback
Recent studies have shown LLMs possess some ability to improve their
responses when given external feedback. However, it remains unclear how
effectively and thoroughly these models can incorporate extrinsic feedback. In
an ideal scenario, if LLMs receive near-perfect and complete feedback, we would
expect them to fully integrate the feedback and change their incorrect answers
to correct ones. In this paper, we systematically investigate LLMs' ability to
incorporate feedback by designing a controlled experimental environment. For
each problem, a solver model attempts a solution, then a feedback generator
with access to near-complete ground-truth answers produces targeted feedback,
after which the solver tries again. We evaluate this pipeline across a diverse
range of tasks, including math reasoning, knowledge reasoning, scientific
reasoning, and general multi-domain evaluations with state-of-the-art language
models including Claude 3.7 (with and without extended thinking). Surprisingly,
even under these near-ideal conditions, solver models consistently show
resistance to feedback, a limitation that we term FEEDBACK FRICTION. To
mitigate this limitation, we experiment with sampling-based strategies like
progressive temperature increases and explicit rejection of previously
attempted incorrect answers, which yield improvements but still fail to help
models achieve target performance. We also perform a rigorous exploration of
potential causes of FEEDBACK FRICTION, ruling out factors such as model
overconfidence and data familiarity. We hope that highlighting this issue in
LLMs and ruling out several apparent causes will help future research in
self-improvement.
684fbab760b4a34dbe007b14
Dongwei Jiang
claimed_verified
2025-08-13T07:30:41.689Z
63eba2774dcaaf087638e3d6
https://cdn-avatars.huggingface.co/v1/production/uploads/1676386919413-noauth.jpeg
Jiang
user
Dongwei
684fbab760b4a34dbe007b15
Alvin Zhang
extracted_confirmed
2025-06-16T14:28:56.348Z
65ed3f714515999256c1f9c9
/avatars/615b2cd9f227226cb9d1219706556567.svg
Alvin Zhang
user
AlvinBZ04
684fbab760b4a34dbe007b16
Andrew Wang
684fbab760b4a34dbe007b17
Nicholas Andrews
claimed_verified
2025-06-23T08:30:09.087Z
666a838f71a81dca8d5dc97e
/avatars/38792103ebf3cdb7733bb6258930193e.svg
Nicholas Andrews
user
noandrews
684fbab760b4a34dbe007b18
Daniel Khashabi
claimed_verified
2025-07-05T07:54:09.431Z
5f6540c65e78cc6b0ed3199d
/avatars/0280d4df417855965a0964d22766c012.svg
Daniel Khashabi
user
danyaljj
Feedback Friction: LLMs Struggle to Fully Incorporate External Feedback
Recent studies have shown LLMs possess some ability to improve their
responses when given external feedback. However, it remains unclear how
effectively and thoroughly these models can incorporate extrinsic feedback. In
an ideal scenario, if LLMs receive near-perfect and complete feedback, we would
expect them to fully integrate the feedback and change their incorrect answers
to correct ones. In this paper, we systematically investigate LLMs' ability to
incorporate feedback by designing a controlled experimental environment. For
each problem, a solver model attempts a solution, then a feedback generator
with access to near-complete ground-truth answers produces targeted feedback,
after which the solver tries again. We evaluate this pipeline across a diverse
range of tasks, including math reasoning, knowledge reasoning, scientific
reasoning, and general multi-domain evaluations with state-of-the-art language
models including Claude 3.7 (with and without extended thinking). Surprisingly,
even under these near-ideal conditions, solver models consistently show
resistance to feedback, a limitation that we term FEEDBACK FRICTION. To
mitigate this limitation, we experiment with sampling-based strategies like
progressive temperature increases and explicit rejection of previously
attempted incorrect answers, which yield improvements but still fail to help
models achieve target performance. We also perform a rigorous exploration of
potential causes of FEEDBACK FRICTION, ruling out factors such as model
overconfidence and data familiarity. We hope that highlighting this issue in
LLMs and ruling out several apparent causes will help future research in
self-improvement.
LLMs show resistance to feedback, termed feedback friction, even under ideal conditions, and sampling-based strategies only partially mitigate this issue.
LLMs feedback generation solver models math reasoning knowledge reasoning scientific reasoning state-of-the-art language models Claude 3.7 extended thinking feedback friction progressive temperature increases explicit rejection