Nanyang Technological University Model - Based Noise Robust Speech Recognition

Noise robustness is a challenging problem when automatic speech recognition (ASR) system is deployed in real life applications. This report examines techniques to improve the robustness of ASR systems. Particularly, we focus on a group of model-based noise robust techniques, called vector Taylor series (VTS) method, that adapt the acoustic model of ASR systems towards noisy test data using the knowledge of noise corruption process. In this report, the VTS method is extended to efficiently handle non-stationary additive noise and convolutional noise cases. The first work in this report is about improving the VTS method to handle nonstationary additive noises. In the conventional VTS method, a single Gaussian is usually used to model noise, but it is insufficient to handle non-stationary noise case. Although using Gaussian mixture models can improve the modeling of noise, this will result in significant increase of model complexity and computational cost. To avoid these drawbacks, we propose to first use a modified spectral subtraction method to reduce the non-stationary characteristics of the additive noise in the speech, and then apply the VTS method using only a single Gaussian noise model. In the modified spectral subtraction method, the noise characteristics is normalized towards a single Gaussian noise model, hence this method is called noise normalization VTS (NN-VTS). In addition, the mismatch function and the acoustic model compensation of the VTS method are also modified to account for the remaining noise in the features. Initial study on the Aurora-2 task shows that NN-VTS can improve the performance of VTS method in non-stationary environments. The second work in this report is about extending the VTS method to handle cepstral mean normalization (CMN) processed features. CMN is an efficient way to reduce channel distortions in speech features. However, conventional VTS method is unable to work