Text-only Domain Adaptation using Unified Speech-Text Representation in
  Transducer
Domain adaptation using text-only corpus is challenging in end-to-end(E2E)
speech recognition. Adaptation by synthesizing audio from text through TTS is
resource-consuming. We present a method to learn Unified Speech-Text
Representation in Conformer Transducer(USTR-CT) to enable fast domain
adaptation using the text-only corpus. Different from the previous textogram
method, an extra text encoder is introduced in our work to learn text
representation and is removed during inference, so there is no modification for
online deployment. To improve the efficiency of adaptation, single-step and
multi-step adaptations are also explored. The experiments on adapting
LibriSpeech to SPGISpeech show the proposed method reduces the word error
rate(WER) by relatively 44% on the target domain, which is better than those of
TTS method and textogram method. Also, it is shown the proposed method can be
combined with internal language model estimation(ILME) to further improve the
performance.
64812e0f9860cd75c260b564
Lu Huang
64812e0f9860cd75c260b565
Boyu Li
64812e0f9860cd75c260b566
Jun Zhang
64812e0f9860cd75c260b567
Lu Lu
64812e0f9860cd75c260b568
Zejun Ma
Text-only Domain Adaptation using Unified Speech-Text Representation in
  Transducer
Domain adaptation using text-only corpus is challenging in end-to-end(E2E)
speech recognition. Adaptation by synthesizing audio from text through TTS is
resource-consuming. We present a method to learn Unified Speech-Text
Representation in Conformer Transducer(USTR-CT) to enable fast domain
adaptation using the text-only corpus. Different from the previous textogram
method, an extra text encoder is introduced in our work to learn text
representation and is removed during inference, so there is no modification for
online deployment. To improve the efficiency of adaptation, single-step and
multi-step adaptations are also explored. The experiments on adapting
LibriSpeech to SPGISpeech show the proposed method reduces the word error
rate(WER) by relatively 44% on the target domain, which is better than those of
TTS method and textogram method. Also, it is shown the proposed method can be
combined with internal language model estimation(ILME) to further improve the
performance.
A method for unified speech-text representation in Conformer Transducer using a text-only corpus allows for efficient domain adaptation in end-to-end speech recognition.
Unified Speech-Text Representation Conformer Transducer text encoder single-step adaptation multi-step adaptation LibriSpeech SPGISpeech word error rate WER internal language model estimation