A Single Character can Make or Break Your LLM Evals
Common Large Language model (LLM) evaluations rely on demonstration examples
to steer models' responses to the desired style. While the number of examples
used has been studied and standardized, the choice of how to format examples is
less investigated. In evaluation protocols and real world usage, users face the
choice how to separate in-context examples: use a comma? new line? semi-colon?
hashtag? etc.? Surprisingly, we find this seemingly minor choice can
dramatically alter model response quality. Across leading model families
(Llama, Qwen, Gemma), performance on MMLU for example can vary by pm 23%
depending on the choice of delimiter. In fact, one can manipulate model
rankings to put any model in the lead by only modifying the single character
separating examples. We find LLMs' brittleness pervades topics, model families,
and doesn't improve with scale. By probing attention head scores, we find that
good-performing delimiters steer attention towards key tokens in the input.
Finally, we explore methods to improve LLMs' robustness to the choice of
delimiter. We find specifying the selected delimiter in the prompt boosts
robustness and offer practical recommendations for the best-performing
delimiters to select.
68e7d43144180c42eca8b0e5
Jingtong Su
68e7d43144180c42eca8b0e6
Jianyu Zhang
68e7d43144180c42eca8b0e7
Karen Ullrich
68e7d43144180c42eca8b0e8
Léon Bottou
68e7d43144180c42eca8b0e9
Mark Ibrahim
A Single Character can Make or Break Your LLM Evals
Common Large Language model (LLM) evaluations rely on demonstration examples
to steer models' responses to the desired style. While the number of examples
used has been studied and standardized, the choice of how to format examples is
less investigated. In evaluation protocols and real world usage, users face the
choice how to separate in-context examples: use a comma? new line? semi-colon?
hashtag? etc.? Surprisingly, we find this seemingly minor choice can
dramatically alter model response quality. Across leading model families
(Llama, Qwen, Gemma), performance on MMLU for example can vary by pm 23%
depending on the choice of delimiter. In fact, one can manipulate model
rankings to put any model in the lead by only modifying the single character
separating examples. We find LLMs' brittleness pervades topics, model families,
and doesn't improve with scale. By probing attention head scores, we find that
good-performing delimiters steer attention towards key tokens in the input.
Finally, we explore methods to improve LLMs' robustness to the choice of
delimiter. We find specifying the selected delimiter in the prompt boosts
robustness and offer practical recommendations for the best-performing
delimiters to select.
The choice of delimiter in formatting in-context examples significantly impacts the performance of large language models across different families and tasks.
Large Language Model (LLM) MMLU attention head scores delimiter robustness prompt