What Makes a Good Natural Language Prompt?
As large language models (LLMs) have progressed towards more human-like and
human--AI communications have become prevalent, prompting has emerged as a
decisive component. However, there is limited conceptual consensus on what
exactly quantifies natural language prompts. We attempt to address this
question by conducting a meta-analysis surveying more than 150
prompting-related papers from leading NLP and AI conferences from 2022 to 2025
and blogs. We propose a property- and human-centric framework for evaluating
prompt quality, encompassing 21 properties categorized into six dimensions. We
then examine how existing studies assess their impact on LLMs, revealing their
imbalanced support across models and tasks, and substantial research gaps.
Further, we analyze correlations among properties in high-quality natural
language prompts, deriving prompting recommendations. We then empirically
explore multi-property prompt enhancements in reasoning tasks, observing that
single-property enhancements often have the greatest impact. Finally, we
discover that instruction-tuning on property-enhanced prompts can result in
better reasoning models. Our findings establish a foundation for
property-centric prompt evaluation and optimization, bridging the gaps between
human--AI communication and opening new prompting research directions.
684ae1fbdbd21a9cc27b0f51
Do Xuan Long
claimed_verified
2025-06-14T06:24:38.018Z
63a9a0d13453852ef53c0b37
/avatars/411c4ffc2a3e89047a23c6e7442f6ed5.svg
Do Xuan Long
user
dxlong2000
684ae1fbdbd21a9cc27b0f52
Duy Dinh
684ae1fbdbd21a9cc27b0f53
Ngoc-Hai Nguyen
684ae1fbdbd21a9cc27b0f54
Kenji Kawaguchi
684ae1fbdbd21a9cc27b0f55
Nancy F. Chen
684ae1fbdbd21a9cc27b0f56
Shafiq Joty
684ae1fbdbd21a9cc27b0f57
Min-Yen Kan
What Makes a Good Natural Language Prompt?
As large language models (LLMs) have progressed towards more human-like and
human--AI communications have become prevalent, prompting has emerged as a
decisive component. However, there is limited conceptual consensus on what
exactly quantifies natural language prompts. We attempt to address this
question by conducting a meta-analysis surveying more than 150
prompting-related papers from leading NLP and AI conferences from 2022 to 2025
and blogs. We propose a property- and human-centric framework for evaluating
prompt quality, encompassing 21 properties categorized into six dimensions. We
then examine how existing studies assess their impact on LLMs, revealing their
imbalanced support across models and tasks, and substantial research gaps.
Further, we analyze correlations among properties in high-quality natural
language prompts, deriving prompting recommendations. We then empirically
explore multi-property prompt enhancements in reasoning tasks, observing that
single-property enhancements often have the greatest impact. Finally, we
discover that instruction-tuning on property-enhanced prompts can result in
better reasoning models. Our findings establish a foundation for
property-centric prompt evaluation and optimization, bridging the gaps between
human--AI communication and opening new prompting research directions.
A framework for evaluating and optimizing natural language prompts in large language models is proposed, revealing correlations between prompt properties and their impact on reasoning tasks.
large language models prompting meta-analysis property-centric framework instruction-tuning reasoning tasks