Beyond One-Size-Fits-All: Inversion Learning for Highly Effective NLG
  Evaluation Prompts
Evaluating natural language generation (NLG) systems is challenging due to
the diversity of valid outputs. While human evaluation is the gold standard, it
suffers from inconsistencies, lack of standardisation, and demographic biases,
limiting reproducibility. LLM-based evaluation offers a scalable alternative
but is highly sensitive to prompt design, where small variations can lead to
significant discrepancies. In this work, we propose an inversion learning
method that learns effective reverse mappings from model outputs back to their
input instructions, enabling the automatic generation of highly effective,
model-specific evaluation prompts. Our method requires only a single evaluation
sample and eliminates the need for time-consuming manual prompt engineering,
thereby improving both efficiency and robustness. Our work contributes toward a
new direction for more robust and efficient LLM-based evaluation.
681889891bcdcf80a49d9ef8
Hanhua Hong
claimed_verified
2025-05-05T14:07:10.803Z
6648d6fff7e9d961cfb57260
https://cdn-avatars.huggingface.co/v1/production/uploads/6648d6fff7e9d961cfb57260/bhjNLs-4xXGNZRV54z_h9.jpeg
Hanhua Hong
user
kou199024
681889891bcdcf80a49d9ef9
Chenghao Xiao
claimed_verified
2025-05-06T16:53:07.494Z
63108cc834c7d77420b0fd68
/avatars/2721e573a417a8ec0b81ee048c4b42ba.svg
chenghao xiao
user
gowitheflow
681889891bcdcf80a49d9efa
Yang Wang
extracted_confirmed
2025-05-05T15:18:25.522Z
60f313f4adf471cbdf8bb66a
https://cdn-avatars.huggingface.co/v1/production/uploads/60f313f4adf471cbdf8bb66a/5NJFqnldE_0fdE_mEvz9V.jpeg
Yang Wang
user
yangwang825
681889891bcdcf80a49d9efb
Yiqi Liu
claimed_verified
2025-05-05T14:07:07.758Z
6399fecbdfc6c5bb8f164073
/avatars/e63ac1a4de53d1fe3e039d0e9dff5c25.svg
Yiqi Liu
user
Yliu566
681889891bcdcf80a49d9efc
Wenge Rong
681889891bcdcf80a49d9efd
Chenghua Lin
Beyond One-Size-Fits-All: Inversion Learning for Highly Effective NLG
  Evaluation Prompts
Evaluating natural language generation (NLG) systems is challenging due to
the diversity of valid outputs. While human evaluation is the gold standard, it
suffers from inconsistencies, lack of standardisation, and demographic biases,
limiting reproducibility. LLM-based evaluation offers a scalable alternative
but is highly sensitive to prompt design, where small variations can lead to
significant discrepancies. In this work, we propose an inversion learning
method that learns effective reverse mappings from model outputs back to their
input instructions, enabling the automatic generation of highly effective,
model-specific evaluation prompts. Our method requires only a single evaluation
sample and eliminates the need for time-consuming manual prompt engineering,
thereby improving both efficiency and robustness. Our work contributes toward a
new direction for more robust and efficient LLM-based evaluation.
Inversion learning automates the generation of effective evaluation prompts for language models, improving robustness and efficiency over manual processes.
natural language generation NLG human evaluation inconsistencies standardisation demographic biases LLM-based evaluation prompt design inversion learning reverse mappings input instructions single evaluation sample