Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates
Automatic LLM benchmarks, such as AlpacaEval 2.0, Arena-Hard-Auto, and
MT-Bench, have become popular for evaluating language models due to their
cost-effectiveness and scalability compared to human evaluation. Achieving high
win rates on these benchmarks can significantly boost the promotional impact of
newly released language models. This promotional benefit may motivate tricks,
such as manipulating model output length or style to game win rates, even
though several mechanisms have been developed to control length and disentangle
style to reduce gameability. Nonetheless, we show that even a "null model" that
always outputs a constant response (irrelevant to input instructions) can cheat
automatic benchmarks and achieve top-ranked win rates: an 86.5% LC win rate on
AlpacaEval 2.0; an 83.0 score on Arena-Hard-Auto; and a 9.55 score on MT-Bench.
Moreover, the crafted cheating outputs are transferable because we assume that
the instructions of these benchmarks (e.g., 805 samples of AlpacaEval 2.0) are
private and cannot be accessed. While our experiments are primarily
proof-of-concept, an adversary could use LLMs to generate more imperceptible
cheating responses, unethically benefiting from high win rates and promotional
impact. Our findings call for the development of anti-cheating mechanisms for
reliable automatic benchmarks. The code is available at
https://github.com/sail-sg/Cheating-LLM-Benchmarks.
670747babbfdd3d3073d1e8e
Xiaosen Zheng
claimed_verified
2024-10-10T08:00:47.197Z
6274a2315d12b3a734adebc9
https://cdn-avatars.huggingface.co/v1/production/uploads/6274a2315d12b3a734adebc9/zLQDbszAvWh0F2BjdKock.jpeg
Xiaosen Zheng
user
xszheng2020
670747babbfdd3d3073d1e8f
Tianyu Pang
claimed_verified
2025-04-20T15:05:09.799Z
63d91b6d255ef6add20e1b38
https://cdn-avatars.huggingface.co/v1/production/uploads/1675921369867-63d91b6d255ef6add20e1b38.jpeg
Tianyu Pang
user
P2333
670747babbfdd3d3073d1e90
Chao Du
claimed_verified
2024-11-15T09:43:35.992Z
632407c892e07e3ca20aca28
/avatars/23b51b37b12b51a0947f687d1de4d3b5.svg
Chao Du
user
duchao
670747babbfdd3d3073d1e91
Qian Liu
claimed_verified
2024-10-11T15:47:23.767Z
612ee6a7b960e78c6d2319d4
https://cdn-avatars.huggingface.co/v1/production/uploads/612ee6a7b960e78c6d2319d4/2Hu9BaAyXbyh1vt0v1Qui.jpeg
Qian Liu
user
SivilTaram
670747babbfdd3d3073d1e92
Jing Jiang
670747babbfdd3d3073d1e93
Min Lin
Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates
Automatic LLM benchmarks, such as AlpacaEval 2.0, Arena-Hard-Auto, and
MT-Bench, have become popular for evaluating language models due to their
cost-effectiveness and scalability compared to human evaluation. Achieving high
win rates on these benchmarks can significantly boost the promotional impact of
newly released language models. This promotional benefit may motivate tricks,
such as manipulating model output length or style to game win rates, even
though several mechanisms have been developed to control length and disentangle
style to reduce gameability. Nonetheless, we show that even a "null model" that
always outputs a constant response (irrelevant to input instructions) can cheat
automatic benchmarks and achieve top-ranked win rates: an 86.5% LC win rate on
AlpacaEval 2.0; an 83.0 score on Arena-Hard-Auto; and a 9.55 score on MT-Bench.
Moreover, the crafted cheating outputs are transferable because we assume that
the instructions of these benchmarks (e.g., 805 samples of AlpacaEval 2.0) are
private and cannot be accessed. While our experiments are primarily
proof-of-concept, an adversary could use LLMs to generate more imperceptible
cheating responses, unethically benefiting from high win rates and promotional
impact. Our findings call for the development of anti-cheating mechanisms for
reliable automatic benchmarks. The code is available at
https://github.com/sail-sg/Cheating-LLM-Benchmarks.
A "null model" can manipulate automatic language model benchmarks to achieve high win rates, highlighting the need for anti-cheating mechanisms in such evaluations.
LLM AlpacaEval 2.0 Arena-Hard-Auto MT-Bench win rates null model anti-cheating mechanisms