Bridging Human and LLM Judgments: Calibration, Alignment, and Bias Detection
Felipe Maia Polo, Xinhe Wang, Mikhail Yurochkin, Gongjun Xu, Moulinath Banerjee, Yuekai Sun
LLM-as-a-judge (LLMJ) has become popular for scalably evaluating language model outputs on open-ended user queries. However, LLM judges do not always align with human annotators, exhibiting systematic and undesired differences, e.g., biases toward certain writing styles. In this paper, we introduce the first unified statistical framework that jointly models human and LLM judge ratings under both absolute and relative (e.g., pairwise comparisons) evaluation paradigms. We posit the existence of a latent human preference score that is shared between humans and LLM judges for each prompt–response pair, and allow LLM judgments to deviate systematically via a linear transformation of a covariate vector, which encodes potential sources of LLM judge biases. Our core contribution is a statistical model that links LLM-assigned scores to the latent human signal and a set of biasing covariates, for which we propose a fitting algorithm and establish asymptotic normality for resulting estimators, enabling formal hypothesis tests of human-LLM discrepancies. Empirically, our model (i) improves the probability calibration of LLM judgments, (ii) aligns LLM ratings more closely with human judgments, and (iii) quantifies and tests systematic LLM biases across evaluation scenarios. We verify the efficacy and provide insightful results using four LLM judges and queries from BigGen Bench and Chatbot Arena.