Improving Mandarin End-to-End Speech Recognition With Word N-Gram Language Model

Despite the rapid progress of end-to-end (E2E) automatic speech recognition (ASR), it has been shown that incorporating external language models (LMs) into decoding can further improve performance E2E ASR systems. To align with modeling units adopted in systems, subword-level (e.g., characters, BPE) LMs are usually used to cooperate current However, use will ignore word-level information, which may limit strength ASR. Although several methods have proposed incorporate ASR, these mainly designed for languages clear word boundaries such as English and cannot be directly applied like Mandarin, each character sequence multiple corresponding sequences. this end, we propose a novel algorithm where lattice is constructed on-the-fly consider all possible sequences partial hypothesis. Then, LM score hypothesis obtained by intersecting generated an N-gram LM. The method examined on both Attention-based Encoder-Decoder (AED) Neural Transducer (NT) frameworks. Experiments suggest our consistently outperforms LMs, including neural network We achieve state-of-the-art results Aishell-1 (CER 4.18%) Aishell-2 5.06%) datasets reduce CER 14.8% relatively 21K-hour Mandarin dataset.