AuditoryBench++: Can Language Models Understand Auditory Knowledge
  without Hearing?
Even without directly hearing sounds, humans can effortlessly reason about
auditory properties, such as pitch, loudness, or sound-source associations,
drawing on auditory commonsense. In contrast, language models often lack this
capability, limiting their effectiveness in multimodal interactions. As an
initial step to address this gap, we present AuditoryBench++, a comprehensive
benchmark for evaluating auditory knowledge and reasoning in text-only
settings. The benchmark encompasses tasks that range from basic auditory
comparisons to contextually grounded reasoning, enabling fine-grained analysis
of how models process and integrate auditory concepts. In addition, we
introduce AIR-CoT, a novel auditory imagination reasoning method that generates
and integrates auditory information during inference through span detection
with special tokens and knowledge injection. Extensive experiments with recent
LLMs and Multimodal LLMs demonstrate that AIR-CoT generally outperforms both
the off-the-shelf models and those augmented with auditory knowledge. The
project page is available at https://auditorybenchpp.github.io.
68d22a281ca7156988a8edf4
Hyunjong Ok
68d22a281ca7156988a8edf5
Suho Yoo
claimed_verified
2025-09-23T10:05:34.245Z
651930780e3a5553d4a04c66
/avatars/35f1a91a2ed920253274c010abc6741d.svg
suho yoo
user
suhoyoo
68d22a281ca7156988a8edf6
Hyeonjun Kim
68d22a281ca7156988a8edf7
Jaeho Lee
AuditoryBench++: Can Language Models Understand Auditory Knowledge
  without Hearing?
Even without directly hearing sounds, humans can effortlessly reason about
auditory properties, such as pitch, loudness, or sound-source associations,
drawing on auditory commonsense. In contrast, language models often lack this
capability, limiting their effectiveness in multimodal interactions. As an
initial step to address this gap, we present AuditoryBench++, a comprehensive
benchmark for evaluating auditory knowledge and reasoning in text-only
settings. The benchmark encompasses tasks that range from basic auditory
comparisons to contextually grounded reasoning, enabling fine-grained analysis
of how models process and integrate auditory concepts. In addition, we
introduce AIR-CoT, a novel auditory imagination reasoning method that generates
and integrates auditory information during inference through span detection
with special tokens and knowledge injection. Extensive experiments with recent
LLMs and Multimodal LLMs demonstrate that AIR-CoT generally outperforms both
the off-the-shelf models and those augmented with auditory knowledge. The
project page is available at https://auditorybenchpp.github.io.
AuditoryBench++ and AIR-CoT enhance text-only models' auditory reasoning and knowledge integration, outperforming existing models in multimodal interactions.
AuditoryBench++ AIR-CoT auditory imagination reasoning span detection knowledge injection multimodal interactions