From Loops to Oops: Fallback Behaviors of Language Models Under
  Uncertainty
Large language models (LLMs) often exhibit undesirable behaviors, such as
hallucinations and sequence repetitions. We propose to view these behaviors as
fallbacks that models exhibit under uncertainty, and investigate the connection
between them. We categorize fallback behaviors -- sequence repetitions,
degenerate text, and hallucinations -- and extensively analyze them in models
from the same family that differ by the amount of pretraining tokens, parameter
count, or the inclusion of instruction-following training. Our experiments
reveal a clear and consistent ordering of fallback behaviors, across all these
axes: the more advanced an LLM is (i.e., trained on more tokens, has more
parameters, or instruction-tuned), its fallback behavior shifts from sequence
repetitions, to degenerate text, and then to hallucinations. Moreover, the same
ordering is observed throughout a single generation, even for the
best-performing models; as uncertainty increases, models shift from generating
hallucinations to producing degenerate text and then sequence repetitions.
Lastly, we demonstrate that while common decoding techniques, such as random
sampling, might alleviate some unwanted behaviors like sequence repetitions,
they increase harder-to-detect hallucinations.
668e312938d04b545cc501f3
Maor Ivgi
claimed_verified
2024-07-10T08:19:30.786Z
610b7f53b1d06c607ab4436d
https://cdn-avatars.huggingface.co/v1/production/uploads/610b7f53b1d06c607ab4436d/Bqw7fIsjpuukFP11S-awA.jpeg
Maor
user
Mivg
668e312938d04b545cc501f4
Ori Yoran
claimed_verified
2024-07-23T07:48:34.994Z
610b7ccff5dd7bcefedaff57
/avatars/a1e77460105a56f84371fa02d530a71e.svg
Yoran
user
Ori
668e312938d04b545cc501f5
Jonathan Berant
admin_assigned
2024-07-10T10:49:55.365Z
60811c9a4ad99100d63ce36c
https://cdn-avatars.huggingface.co/v1/production/uploads/1619119757620-60811c9a4ad99100d63ce36c.png
Jonathan Berant
user
joberant
668e312938d04b545cc501f6
Mor Geva
claimed_verified
2024-07-10T10:20:22.150Z
610b729f9da682cd54ad9adf
https://cdn-avatars.huggingface.co/v1/production/uploads/1628140189042-noauth.jpeg
Mor Geva
user
mega
From Loops to Oops: Fallback Behaviors of Language Models Under
  Uncertainty
Large language models (LLMs) often exhibit undesirable behaviors, such as
hallucinations and sequence repetitions. We propose to view these behaviors as
fallbacks that models exhibit under uncertainty, and investigate the connection
between them. We categorize fallback behaviors -- sequence repetitions,
degenerate text, and hallucinations -- and extensively analyze them in models
from the same family that differ by the amount of pretraining tokens, parameter
count, or the inclusion of instruction-following training. Our experiments
reveal a clear and consistent ordering of fallback behaviors, across all these
axes: the more advanced an LLM is (i.e., trained on more tokens, has more
parameters, or instruction-tuned), its fallback behavior shifts from sequence
repetitions, to degenerate text, and then to hallucinations. Moreover, the same
ordering is observed throughout a single generation, even for the
best-performing models; as uncertainty increases, models shift from generating
hallucinations to producing degenerate text and then sequence repetitions.
Lastly, we demonstrate that while common decoding techniques, such as random
sampling, might alleviate some unwanted behaviors like sequence repetitions,
they increase harder-to-detect hallucinations.
LLMs exhibit a consistent sequence of fallback behaviors—sequence repetitions, degenerate text, and hallucinations—under uncertainty, and this ordering is observed across models with varying levels of training.
large language models hallucinations sequence repetitions degenerate text pretraining tokens parameter count instruction-following training uncertainty