ERGO: Entropy-guided Resetting for Generation Optimization in Multi-turn
  Language Models
Large Language Models (LLMs) suffer significant performance degradation in
multi-turn conversations when information is presented incrementally. Given
that multi-turn conversations characterize everyday interactions with LLMs,
this degradation poses a severe challenge to real world usability. We
hypothesize that abrupt increases in model uncertainty signal misalignment in
multi-turn LLM interactions, and we exploit this insight to dynamically realign
conversational context. We introduce ERGO (Entropy-guided Resetting for
Generation Optimization), which continuously quantifies internal uncertainty
via Shannon entropy over next token distributions and triggers adaptive prompt
consolidation when a sharp spike in entropy is detected. By treating
uncertainty as a first class signal rather than a nuisance to eliminate, ERGO
embraces variability in language and modeling, representing and responding to
uncertainty. In multi-turn tasks with incrementally revealed instructions, ERGO
yields a 56.6% average performance gain over standard baselines, increases
aptitude (peak performance capability) by 24.7%, and decreases unreliability
(variability in performance) by 35.3%, demonstrating that uncertainty aware
interventions can improve both accuracy and reliability in conversational AI.
68f292388589920bf4d318ea
Haziq Mohammad Khalid
claimed_verified
2025-10-18T04:47:22.562Z
669577dd5d561534552a3960
https://cdn-avatars.huggingface.co/v1/production/uploads/669577dd5d561534552a3960/BrLfQGoOwcjPn2J-3ePI9.png
Haziq Mohammad Khalid
user
Haziq-exe
68f292388589920bf4d318eb
Athikash Jeyaganthan
68f292388589920bf4d318ec
Timothy Do
68f292388589920bf4d318ed
Yicheng Fu
68f292388589920bf4d318ee
Sean O'Brien
68f292388589920bf4d318ef
Vasu Sharma
68f292388589920bf4d318f0
Kevin Zhu
ERGO: Entropy-guided Resetting for Generation Optimization in Multi-turn
  Language Models
Large Language Models (LLMs) suffer significant performance degradation in
multi-turn conversations when information is presented incrementally. Given
that multi-turn conversations characterize everyday interactions with LLMs,
this degradation poses a severe challenge to real world usability. We
hypothesize that abrupt increases in model uncertainty signal misalignment in
multi-turn LLM interactions, and we exploit this insight to dynamically realign
conversational context. We introduce ERGO (Entropy-guided Resetting for
Generation Optimization), which continuously quantifies internal uncertainty
via Shannon entropy over next token distributions and triggers adaptive prompt
consolidation when a sharp spike in entropy is detected. By treating
uncertainty as a first class signal rather than a nuisance to eliminate, ERGO
embraces variability in language and modeling, representing and responding to
uncertainty. In multi-turn tasks with incrementally revealed instructions, ERGO
yields a 56.6% average performance gain over standard baselines, increases
aptitude (peak performance capability) by 24.7%, and decreases unreliability
(variability in performance) by 35.3%, demonstrating that uncertainty aware
interventions can improve both accuracy and reliability in conversational AI.
ERGO, an entropy-guided resetting method, improves conversational AI performance by dynamically realigning context based on internal uncertainty, leading to enhanced accuracy and reliability in multi-turn interactions.
Large Language Models multi-turn conversations model uncertainty Shannon entropy adaptive prompt consolidation entropy-guided resetting uncertainty-aware interventions