When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness
  Methods for LLMs
Large Language Models (LLMs) are highly sensitive to subtle, non-semantic
variations in prompt phrasing and formatting. In this work, we present the
first systematic evaluation of 5 methods for improving prompt robustness within
a unified experimental framework. We benchmark these techniques on 8 models
from Llama, Qwen and Gemma families across 52 tasks from Natural Instructions
dataset. Our evaluation covers robustness methods from both fine-tuned and
in-context learning paradigms, and tests their generalization against multiple
types of distribution shifts. Finally, we extend our analysis to GPT-4.1 and
DeepSeek V3 to assess frontier models' current robustness to format
perturbations. Our findings offer actionable insights into the relative
effectiveness of these robustness methods, enabling practitioners to make
informed decisions when aiming for stable and reliable LLM performance in
real-world applications. Code:
https://github.com/AIRI-Institute/when-punctuation-matters.
68a2fa12a4caabb4320e6525
Mikhail Seleznyov
claimed_verified
2025-08-19T08:05:10.952Z
654621f45cd5692b3a9d08cb
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/NJj1FJyYHNPRXb0QBXjv-.jpeg
Mikhail Seleznyov
user
myyycroft
68a2fa12a4caabb4320e6526
Mikhail Chaichuk
68a2fa12a4caabb4320e6527
Gleb Ershov
claimed_verified
2025-08-21T13:32:18.040Z
644ba2d170dd5a3130101902
https://cdn-avatars.huggingface.co/v1/production/uploads/644ba2d170dd5a3130101902/iMTKcYuTNVH28yhyQWZIP.jpeg
Gleb E
user
GLBHLB
68a2fa12a4caabb4320e6528
Alexander Panchenko
claimed_verified
2025-08-19T11:21:58.330Z
605473729d7c1d4d81b7e52b
https://cdn-avatars.huggingface.co/v1/production/uploads/1662046050710-605473729d7c1d4d81b7e52b.jpeg
Alexander Panchenko
user
apanc
68a2fa12a4caabb4320e6529
Elena Tutubalina
68a2fa12a4caabb4320e652a
Oleg Somov
When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness
  Methods for LLMs
Large Language Models (LLMs) are highly sensitive to subtle, non-semantic
variations in prompt phrasing and formatting. In this work, we present the
first systematic evaluation of 5 methods for improving prompt robustness within
a unified experimental framework. We benchmark these techniques on 8 models
from Llama, Qwen and Gemma families across 52 tasks from Natural Instructions
dataset. Our evaluation covers robustness methods from both fine-tuned and
in-context learning paradigms, and tests their generalization against multiple
types of distribution shifts. Finally, we extend our analysis to GPT-4.1 and
DeepSeek V3 to assess frontier models' current robustness to format
perturbations. Our findings offer actionable insights into the relative
effectiveness of these robustness methods, enabling practitioners to make
informed decisions when aiming for stable and reliable LLM performance in
real-world applications. Code:
https://github.com/AIRI-Institute/when-punctuation-matters.
A systematic evaluation of prompt robustness methods for Large Language Models across various models and tasks reveals insights into their effectiveness against format perturbations.
Large Language Models prompt robustness fine-tuned in-context learning distribution shifts GPT-4.1 DeepSeek V3