Beyond Transcription: Mechanistic Interpretability in ASR
Interpretability methods have recently gained significant attention,
particularly in the context of large language models, enabling insights into
linguistic representations, error detection, and model behaviors such as
hallucinations and repetitions. However, these techniques remain underexplored
in automatic speech recognition (ASR), despite their potential to advance both
the performance and interpretability of ASR systems. In this work, we adapt and
systematically apply established interpretability methods such as logit lens,
linear probing, and activation patching, to examine how acoustic and semantic
information evolves across layers in ASR systems. Our experiments reveal
previously unknown internal dynamics, including specific encoder-decoder
interactions responsible for repetition hallucinations and semantic biases
encoded deep within acoustic representations. These insights demonstrate the
benefits of extending and applying interpretability techniques to speech
recognition, opening promising directions for future research on improving
model transparency and robustness.
68af59ed245176306494caee
Neta Glazer
claimed_verified
2025-08-28T08:55:33.796Z
62659a1d24a066c92d76d2ee
/avatars/9ac398922f67350e7be6121f2cbcebcb.svg
glazer
user
netag
68af59ed245176306494caef
Yael Segal-Feldman
admin_assigned
2025-08-28T18:55:03.027Z
669cc8a75bd3f749a3d09c5b
/avatars/3faf705394ccb3740c78f2caab50ad1d.svg
Yael Segal-Feldman
user
YaelAiola
68af59ed245176306494caf0
Hilit Segev
admin_assigned
2025-08-28T18:55:09.255Z
6492bc477ddf9471f55de3b8
/avatars/ba42cf14b1ae558bd29e3ff51e59531a.svg
Hilit Segev
user
Hilit
68af59ed245176306494caf1
Aviv Shamsian
admin_assigned
2025-08-28T18:55:15.902Z
6488391759e0256b92194209
/avatars/fbc1c3fbee06e5ef0b8fa5671f111795.svg
Aviv Shamsian
user
AvivSham
68af59ed245176306494caf2
Asaf Buchnick
68af59ed245176306494caf3
Gill Hetz
68af59ed245176306494caf4
Ethan Fetaya
68af59ed245176306494caf5
Joseph Keshet
68af59ed245176306494caf6
Aviv Navon
claimed_verified
2025-08-28T08:55:31.653Z
63563ac12d14fcd7d83729d6
/avatars/4e78baf7edb286b2518fb4a2fa6dbe04.svg
Aviv Navon
user
AvivNavon
Beyond Transcription: Mechanistic Interpretability in ASR
Interpretability methods have recently gained significant attention,
particularly in the context of large language models, enabling insights into
linguistic representations, error detection, and model behaviors such as
hallucinations and repetitions. However, these techniques remain underexplored
in automatic speech recognition (ASR), despite their potential to advance both
the performance and interpretability of ASR systems. In this work, we adapt and
systematically apply established interpretability methods such as logit lens,
linear probing, and activation patching, to examine how acoustic and semantic
information evolves across layers in ASR systems. Our experiments reveal
previously unknown internal dynamics, including specific encoder-decoder
interactions responsible for repetition hallucinations and semantic biases
encoded deep within acoustic representations. These insights demonstrate the
benefits of extending and applying interpretability techniques to speech
recognition, opening promising directions for future research on improving
model transparency and robustness.
Interpretability methods like logit lens, linear probing, and activation patching are applied to ASR to uncover internal dynamics, repetition hallucinations, and semantic biases, enhancing model transparency and robustness.
logit lens linear probing activation patching encoder-decoder interactions repetition hallucinations semantic biases