Hidden Markov Model Based Speech Activity Detection for the ICSI Meeting Project

As part of a project into speech recognition in meeting environments, we have collected a corpus of multi-channel meeting recordings. We expected the identification of speaker activity to be straightforward given that the participants had individual microphones, but simple approaches yielded unacceptably erroneous labelings, mainly due to crosstalk between nearby speakers and wide variations in channel characteristics. We have therefore developed a more sophisticated approach for multichannel speaker activity detection based on a simple hidden Markov model (HMM). A baseline HMM speech activity detector has been extended to use mixtures of Gaussians to achieve robustness for different speakers under different conditions. To further improve the channel independence, normalized features are used. The use of the proposed energy normalization yields a relative reduction in frame error rate by 26.4%. In a postprocessing step the crosscorrelation between different channels is used to detect crosstalk. Using this postprocessing step results in a further reduction of the frame error rate by 12.4%.