Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs
Prompt sensitivity, referring to the phenomenon where paraphrasing (i.e.,
repeating something written or spoken using different words) leads to
significant changes in large language model (LLM) performance, has been widely
accepted as a core limitation of LLMs. In this work, we revisit this issue and
ask: Is the widely reported high prompt sensitivity truly an inherent weakness
of LLMs, or is it largely an artifact of evaluation processes? To answer this
question, we systematically evaluate 7 LLMs (e.g., GPT and Gemini family)
across 6 benchmarks, including both multiple-choice and open-ended tasks on 12
diverse prompt templates. We find that much of the prompt sensitivity stems
from heuristic evaluation methods, including log-likelihood scoring and rigid
answer matching, which often overlook semantically correct responses expressed
through alternative phrasings, such as synonyms or paraphrases. When we adopt
LLM-as-a-Judge evaluations, we observe a substantial reduction in performance
variance and a consistently higher correlation in model rankings across
prompts. Our findings suggest that modern LLMs are more robust to prompt
templates than previously believed, and that prompt sensitivity may be more an
artifact of evaluation than a flaw in the models.
68b862c4d43cadaf7a688a41
Andong Hua
68b862c4d43cadaf7a688a42
Kenan Tang
claimed_verified
2025-09-04T20:11:58.541Z
65e249e3774c93c2bc7a9088
https://cdn-avatars.huggingface.co/v1/production/uploads/65e249e3774c93c2bc7a9088/VIqzekU_r2svl6nOBXU6c.jpeg
Kenan Tang
user
kenantang
68b862c4d43cadaf7a688a43
Chenhe Gu
68b862c4d43cadaf7a688a44
Jindong Gu
68b862c4d43cadaf7a688a45
Eric Wong
68b862c4d43cadaf7a688a46
Yao Qin
Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs
Prompt sensitivity, referring to the phenomenon where paraphrasing (i.e.,
repeating something written or spoken using different words) leads to
significant changes in large language model (LLM) performance, has been widely
accepted as a core limitation of LLMs. In this work, we revisit this issue and
ask: Is the widely reported high prompt sensitivity truly an inherent weakness
of LLMs, or is it largely an artifact of evaluation processes? To answer this
question, we systematically evaluate 7 LLMs (e.g., GPT and Gemini family)
across 6 benchmarks, including both multiple-choice and open-ended tasks on 12
diverse prompt templates. We find that much of the prompt sensitivity stems
from heuristic evaluation methods, including log-likelihood scoring and rigid
answer matching, which often overlook semantically correct responses expressed
through alternative phrasings, such as synonyms or paraphrases. When we adopt
LLM-as-a-Judge evaluations, we observe a substantial reduction in performance
variance and a consistently higher correlation in model rankings across
prompts. Our findings suggest that modern LLMs are more robust to prompt
templates than previously believed, and that prompt sensitivity may be more an
artifact of evaluation than a flaw in the models.
Modern LLMs exhibit less prompt sensitivity than previously thought, with much of the reported variability due to heuristic evaluation methods rather than inherent model weaknesses.
prompt sensitivity large language models LLMs GPT Gemini family benchmarks multiple-choice open-ended tasks prompt templates log-likelihood scoring rigid answer matching LLM-as-a-Judge evaluations performance variance model rankings