RIGID THINKING: LLMs Struggle to Fully Incorporate External Feedback
Dongwei Jiang, Bowei Zhang, Andrew Wang, Nicholas Andrews, Daniel Khashabi
Recent studies have shown LLMs possess \textit{some} ability to improve their responses when given external feedback.However, it remains unclear how effectively and thoroughly these models can incorporate extrinsic feedback.  In an ideal scenario, if LLMs receive near-perfect and complete feedback, we would expect them to \emph{fully} integrate the feedback and change their incorrect answers to correct ones.In this paper, we systematically investigate LLMs' ability to incorporate feedback by designing a controlled experimental environment. For each problem, a solver model attempts a solution, then a feedback generator with access to \textit{near-complete} ground-truth answers produces targeted feedback, after which the solver tries again. We evaluate this pipeline across a diverse range of tasks, including mathematical reasoning, knowledge reasoning, complex scientific reasoning, and general multi-domain evaluations with state-of-the-art language models as solver and feedback generator. Surprisingly, even under these near-ideal conditions, solver models consistently show resistance to feedback, a limitation that we term RIGID THINKING. To mitigate this limitation, we experiment with sampling-based strategies like progressive temperature increases and explicit rejection of previously attempted incorrect answers, which yield improvements but still fail to help models achieve target performance. We perform a rigorous exploration of potential causes of RIGID THINKING, ruling out factors such as model overconfidence and data familiarity.