When an LLM is apprehensive about its answers -- and when its
  uncertainty is justified
Uncertainty estimation is crucial for evaluating Large Language Models
(LLMs), particularly in high-stakes domains where incorrect answers result in
significant consequences. Numerous approaches consider this problem, while
focusing on a specific type of uncertainty, ignoring others. We investigate
what estimates, specifically token-wise entropy and model-as-judge (MASJ),
would work for multiple-choice question-answering tasks for different question
topics. Our experiments consider three LLMs: Phi-4, Mistral, and Qwen of
different sizes from 1.5B to 72B and 14 topics. While MASJ performs similarly
to a random error predictor, the response entropy predicts model error in
knowledge-dependent domains and serves as an effective indicator of question
difficulty: for biology ROC AUC is 0.73. This correlation vanishes for the
reasoning-dependent domain: for math questions ROC-AUC is 0.55. More
principally, we found out that the entropy measure required a reasoning amount.
Thus, data-uncertainty related entropy should be integrated within uncertainty
estimates frameworks, while MASJ requires refinement. Moreover, existing
MMLU-Pro samples are biased, and should balance required amount of reasoning
for different subdomains to provide a more fair assessment of LLMs performance.
67c6e6735aea9d8918635ac2
Petr Sychev
claimed_verified
2025-03-04T12:01:33.230Z
6728224623d75cbd1cdbe568
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/4sb6TjuzeDc8-PG9hYhjW.jpeg
Petr Sychev
user
sspetya
67c6e6735aea9d8918635ac3
Andrey Goncharov
extracted_pending
2025-03-04T11:39:33.550Z
675708985b91dea24c3ef642
https://cdn-avatars.huggingface.co/v1/production/uploads/675708985b91dea24c3ef642/8KmerI1LwJEBHM2vrC54d.jpeg
Andrey Goncharov
user
aigoncharov
67c6e6735aea9d8918635ac4
Daniil Vyazhev
claimed_verified
2025-03-04T12:01:34.869Z
659e049c01805191e5f67b12
/avatars/4f33e39d85f8fbdfaeb34143e5038b92.svg
Vyazhev
user
DanielVyazhev
67c6e6735aea9d8918635ac5
Edvard Khalafyan
67c6e6735aea9d8918635ac6
Alexey Zaytsev
When an LLM is apprehensive about its answers -- and when its
  uncertainty is justified
Uncertainty estimation is crucial for evaluating Large Language Models
(LLMs), particularly in high-stakes domains where incorrect answers result in
significant consequences. Numerous approaches consider this problem, while
focusing on a specific type of uncertainty, ignoring others. We investigate
what estimates, specifically token-wise entropy and model-as-judge (MASJ),
would work for multiple-choice question-answering tasks for different question
topics. Our experiments consider three LLMs: Phi-4, Mistral, and Qwen of
different sizes from 1.5B to 72B and 14 topics. While MASJ performs similarly
to a random error predictor, the response entropy predicts model error in
knowledge-dependent domains and serves as an effective indicator of question
difficulty: for biology ROC AUC is 0.73. This correlation vanishes for the
reasoning-dependent domain: for math questions ROC-AUC is 0.55. More
principally, we found out that the entropy measure required a reasoning amount.
Thus, data-uncertainty related entropy should be integrated within uncertainty
estimates frameworks, while MASJ requires refinement. Moreover, existing
MMLU-Pro samples are biased, and should balance required amount of reasoning
for different subdomains to provide a more fair assessment of LLMs performance.
Token-wise entropy and model-as-judge (MASJ) are evaluated as uncertainty estimation methods for multiple-choice question-answering tasks, showing that entropy performs better in knowledge-dependent domains but requires reasoning, while MASJ needs refinement.
token-wise entropy model-as-judge (MASJ) knowledge-dependent domains reasoning-dependent domains ROC AUC MMLU-Pro