Challenges in Trustworthy Human Evaluation of Chatbots
Open community-driven platforms like Chatbot Arena that collect user
preference data from site visitors have gained a reputation as one of the most
trustworthy publicly available benchmarks for LLM performance. While now
standard, it is tricky to implement effective guardrails to collect
high-quality annotations from humans. In this paper, we demonstrate that three
sources of bad annotations, both malicious and otherwise, can corrupt the
reliability of open leaderboard rankings. In particular, we show that only 10\%
of poor quality votes by apathetic (site visitors not appropriately
incentivized to give correct votes) or adversarial (bad actors seeking to
inflate the ranking of a target model) annotators can change the rankings of
models by up to 5 places on the leaderboard. Finally, we discuss open
challenges in ensuring high-quality human annotations.
67530912232ea247a627e51a
Wenting Zhao
67530912232ea247a627e51b
Alexander M. Rush
67530912232ea247a627e51c
Tanya Goyal
Challenges in Trustworthy Human Evaluation of Chatbots
Open community-driven platforms like Chatbot Arena that collect user
preference data from site visitors have gained a reputation as one of the most
trustworthy publicly available benchmarks for LLM performance. While now
standard, it is tricky to implement effective guardrails to collect
high-quality annotations from humans. In this paper, we demonstrate that three
sources of bad annotations, both malicious and otherwise, can corrupt the
reliability of open leaderboard rankings. In particular, we show that only 10\%
of poor quality votes by apathetic (site visitors not appropriately
incentivized to give correct votes) or adversarial (bad actors seeking to
inflate the ranking of a target model) annotators can change the rankings of
models by up to 5 places on the leaderboard. Finally, we discuss open
challenges in ensuring high-quality human annotations.
Bad annotations from apathetic or adversarial users can significantly impact leaderboard rankings in community-driven LLM benchmark platforms like Chatbot Arena.
LLM Chatbot Arena user preference data high-quality annotations guardrails poor quality votes apathetic annotators adversarial annotators leaderboard rankings