Detection of Non-Native Named Entities Using Prosodic Features for Improved Speech Recognition and Translation

In this work, we describe the use of acoustic-prosodic features to detect and localize non-native named entities spoken by a native speaker in the target language (English) for the purpose of improved speech recognition and translation. The exaggerated variation in accent and duration introduced by the speaker for non-native names is exploited in the detection process through the use of prosodic features like f0 excursions, durational variations and pause information. First, we validate the use of prosodic features in classifying non-native named entities (person names in Chinese, Japanese, Russian, Spanish, Italian, Persian, Indian) in the first mention spoken by native English speakers. We set up the problem as a binary classification task between the non-native named entities and other content words spoken by the speakers in the native language. Results based on a Support Vector Machine (SVM) classifier indicate a 80% classification accuracy for such events. Second, we use the prosody-based SVM classifier to detect and localize named entities at the output of an Automatic Speech Recognizer (ASR).