Development of Automatic Speech Recognition and Synthesis Technologies to Support Chinese Learners of English: The CUHK Experience

This paper presents our group’s ongoing research in the area of computer-aided pronunciation training (CAPT) for Chinese learners of English. Our goal is to develop technologies in automatic speech recognition (ASR) to support productive training for learners. We focus on modeling possible errors due to negative transfer from L1 (i.e. Chinese) of Chinese learners to L2 (English). ASR techniques are used for fine phonetic analyses to achieve mispronunciation detection and diagnoses. Learners’ inputs that are grossly erroneous are handled by an utterance rejection technique. We also present initial work in the development of audio and visual speech synthesis to provide corrective feedback for learners. This paper presents an overview of the technologies, related experimental results and ongoing work as well as future plans.