DeCRED: Decoder-Centric Regularization for Encoder-Decoder Based Speech
  Recognition
This paper presents a simple yet effective regularization for the internal
language model induced by the decoder in encoder-decoder ASR models, thereby
improving robustness and generalization in both in- and out-of-domain settings.
The proposed method, Decoder-Centric Regularization in Encoder-Decoder
(DeCRED), adds auxiliary classifiers to the decoder, enabling next token
prediction via intermediate logits. Empirically, DeCRED reduces the mean
internal LM BPE perplexity by 36.6% relative to 11 test sets. Furthermore, this
translates into actual WER improvements over the baseline in 5 of 7 in-domain
and 3 of 4 out-of-domain test sets, reducing macro WER from 6.4% to 6.3% and
18.2% to 16.2%, respectively. On TEDLIUM3, DeCRED achieves 7.0% WER, surpassing
the baseline and encoder-centric InterCTC regularization by 0.6% and 0.5%,
respectively. Finally, we compare DeCRED with OWSM v3.1 and Whisper-medium,
showing competitive WERs despite training on much less data with fewer
parameters.
689c5447fab6fdd2e52ac97d
Alexander Polok
689c5447fab6fdd2e52ac97e
Santosh Kesiraju
689c5447fab6fdd2e52ac97f
Karel Beneš
689c5447fab6fdd2e52ac980
Bolaji Yusuf
689c5447fab6fdd2e52ac981
Lukáš Burget
689c5447fab6fdd2e52ac982
Jan Černocký
DeCRED: Decoder-Centric Regularization for Encoder-Decoder Based Speech
  Recognition
This paper presents a simple yet effective regularization for the internal
language model induced by the decoder in encoder-decoder ASR models, thereby
improving robustness and generalization in both in- and out-of-domain settings.
The proposed method, Decoder-Centric Regularization in Encoder-Decoder
(DeCRED), adds auxiliary classifiers to the decoder, enabling next token
prediction via intermediate logits. Empirically, DeCRED reduces the mean
internal LM BPE perplexity by 36.6% relative to 11 test sets. Furthermore, this
translates into actual WER improvements over the baseline in 5 of 7 in-domain
and 3 of 4 out-of-domain test sets, reducing macro WER from 6.4% to 6.3% and
18.2% to 16.2%, respectively. On TEDLIUM3, DeCRED achieves 7.0% WER, surpassing
the baseline and encoder-centric InterCTC regularization by 0.6% and 0.5%,
respectively. Finally, we compare DeCRED with OWSM v3.1 and Whisper-medium,
showing competitive WERs despite training on much less data with fewer
parameters.
Decoder-Centric Regularization in Encoder-Decoder (DeCRED) improves ASR robustness and generalization by adding auxiliary classifiers to the decoder, reducing internal language model perplexity and WER.
encoder-decoder ASR models Decoder-Centric Regularization in Encoder-Decoder DeCRED auxiliary classifiers next token prediction internal LM BPE perplexity WER in-domain out-of-domain TEDLIUM3 InterCTC regularization OWSM v3.1 Whisper-medium