Interpolation of stochastic grammar and word bigram models in natural language understanding

The paper shows the effects of combining a stochastic grammar with a word bigram language model by log-linear interpolation. It is divided into three main parts: The first part derives the stochastic grammar model and gives a sound theoretical motivation to incorporate word dependencies such as bigrams. The second part describes two different algorithmic approaches to the combination of both models by log-linear interpolation. The third part reports attribute error rate (AER) results measured on the Philips corpus of train time table inquiries that show a reduction of up to 9% relative. 1. STOCHASTIC MODEL OF NATURAL LANGUAGE UNDERSTANDING The Philips Natural Language Understanding (NLU) module is used in automated inquiry systems (AIS), such as train table enquiries [2], to analyze the word sequence of a user utterance. It does not try to find parse trees that cover the whole word sequence but breaks up the sequence into chunks, where each chunk belongs to a semantically meaningful concept. A stochastic context–free grammar is used to derive the word chunk from a concept. The chunking is useful since the spontaneous speech that occurs in dialogue applications is very ungrammatical. Thus, a robust NLU model concentrates on the useful parts of a user utterance. Other recent works also employ some kind of chunking, e.g. [6, 9]. The stochastic model of the Philips NLU module was developed by H. Aust in [1, p. 81]. Here, we show that this model can be derived from Bayes’ decision rule. This derivation gives a sound theoretical motivation to incorporate word dependencies such as bigrams. Bayes’ decision rule finds the most likely concept sequence K̂ = k̂1, . . . , k̂s, given the sequence O = o1, . . . , ot of acoustic observations. The derivation of the concept sequence K does not directly depend on the acoustic observations O but on a word sequence W = w1, . . . , wN derived from O as an intermediate result: K̂ = argmax K p(K|O)