DateLogicQA: Benchmarking Temporal Biases in Large Language Models
This paper introduces DateLogicQA, a benchmark with 190 questions covering
diverse date formats, temporal contexts, and reasoning types. We propose the
Semantic Integrity Metric to assess tokenization quality and analyse two
biases: Representation-Level Bias, affecting embeddings, and Logical-Level
Bias, influencing reasoning outputs. Our findings provide a comprehensive
evaluation of LLMs' capabilities and limitations in temporal reasoning,
highlighting key challenges in handling temporal data accurately. The GitHub
repository for our work is available at
https://github.com/gagan3012/EAIS-Temporal-Bias
676577a8abcd70b404ad67ca
Gagan Bhatia
claimed_verified
2024-12-20T15:42:23.951Z
60394599033b61166496163b
https://cdn-avatars.huggingface.co/v1/production/uploads/1614366097007-noauth.jpeg
Gagan Bhatia
user
gagan3012
676577a8abcd70b404ad67cb
MingZe Tang
676577a8abcd70b404ad67cc
Cristina Mahanta
676577a8abcd70b404ad67cd
Madiha Kazi
DateLogicQA: Benchmarking Temporal Biases in Large Language Models
This paper introduces DateLogicQA, a benchmark with 190 questions covering
diverse date formats, temporal contexts, and reasoning types. We propose the
Semantic Integrity Metric to assess tokenization quality and analyse two
biases: Representation-Level Bias, affecting embeddings, and Logical-Level
Bias, influencing reasoning outputs. Our findings provide a comprehensive
evaluation of LLMs' capabilities and limitations in temporal reasoning,
highlighting key challenges in handling temporal data accurately. The GitHub
repository for our work is available at
https://github.com/gagan3012/EAIS-Temporal-Bias
DateLogicQA benchmark evaluates LLMs' temporal reasoning capabilities and identifies biases in embeddings and reasoning outputs.
tokenization quality Semantic Integrity Metric Representation-Level Bias Logical-Level Bias temporal reasoning temporal data handling