TAU: A Benchmark for Cultural Sound Understanding Beyond Semantics
Large audio-language models are advancing rapidly, yet most evaluations
emphasize speech or globally sourced sounds, overlooking culturally distinctive
cues. This gap raises a critical question: can current models generalize to
localized, non-semantic audio that communities instantly recognize but
outsiders do not? To address this, we present TAU (Taiwan Audio Understanding),
a benchmark of everyday Taiwanese "soundmarks." TAU is built through a pipeline
combining curated sources, human editing, and LLM-assisted question generation,
producing 702 clips and 1,794 multiple-choice items that cannot be solved by
transcripts alone. Experiments show that state-of-the-art LALMs, including
Gemini 2.5 and Qwen2-Audio, perform far below local humans. TAU demonstrates
the need for localized benchmarks to reveal cultural blind spots, guide more
equitable multimodal evaluation, and ensure models serve communities beyond the
global mainstream.
68dc98004159d1f2418f9b5f
Yi-Cheng Lin
68dc98004159d1f2418f9b60
Yu-Hua Chen
68dc98004159d1f2418f9b61
Jia-Kai Dong
claimed_verified
2025-10-24T16:15:08.402Z
67ecfe11534d68f5a87834c2
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/ZnmCQzju-Okic9XdvJEvN.png
董家愷
user
Snooow1029
68dc98004159d1f2418f9b62
Yueh-Hsuan Huang
68dc98004159d1f2418f9b63
Szu-Chi Chen
68dc98004159d1f2418f9b64
Yu-Chen Chen
68dc98004159d1f2418f9b65
Chih-Yao Chen
68dc98004159d1f2418f9b66
Yu-Jung Lin
68dc98004159d1f2418f9b67
Yu-Ling Chen
68dc98004159d1f2418f9b68
Zih-Yu Chen
68dc98004159d1f2418f9b69
I-Ning Tsai
68dc98004159d1f2418f9b6a
Hsiu-Hsuan Wang
68dc98004159d1f2418f9b6b
Ho-Lam Chung
68dc98004159d1f2418f9b6c
Ke-Han Lu
68dc98004159d1f2418f9b6d
Hung-yi Lee
TAU: A Benchmark for Cultural Sound Understanding Beyond Semantics
Large audio-language models are advancing rapidly, yet most evaluations
emphasize speech or globally sourced sounds, overlooking culturally distinctive
cues. This gap raises a critical question: can current models generalize to
localized, non-semantic audio that communities instantly recognize but
outsiders do not? To address this, we present TAU (Taiwan Audio Understanding),
a benchmark of everyday Taiwanese "soundmarks." TAU is built through a pipeline
combining curated sources, human editing, and LLM-assisted question generation,
producing 702 clips and 1,794 multiple-choice items that cannot be solved by
transcripts alone. Experiments show that state-of-the-art LALMs, including
Gemini 2.5 and Qwen2-Audio, perform far below local humans. TAU demonstrates
the need for localized benchmarks to reveal cultural blind spots, guide more
equitable multimodal evaluation, and ensure models serve communities beyond the
global mainstream.
TAU, a benchmark of culturally specific Taiwanese soundmarks, reveals that state-of-the-art large audio-language models underperform compared to local humans, highlighting the need for localized evaluations.
audio-language models soundmarks benchmark LLM-assisted question generation multimodal evaluation