Integrating Knowledge Into End-to-End Speech Recognition From External Text-Only Data

Attention-based encoder-decoder (AED) models have achieved promising performance in speech recognition. However, because of the end-to-end training, an AED model is usually trained with speech-text paired data. It challenging to incorporate external text-only data into models. Another issue that it does not use right context a text token while predicting token. To alleviate above two issues, we propose unified method called LST (Learn Spelling from Teachers) integrate knowledge and leverage whole sentence. The divided stages. First, representation stage, language on text. can be seen as compressed LM. Then, at transferring transferred via teacher-student learning. further sentence, LM causal cloze completer (COR), which estimates probability token, given both left it. Therefore, Different fusion based methods, during decoding, proposed increase any extra complexity inference stage. We conduct experiments scales public Chinese datasets AISHELL-1 AISHELL-2. experimental results demonstrate effectiveness leveraging sentence our method, compared baseline hybrid systems systems.