New feature weighting approaches for speech-act classification

A dialogue system is a software program that enables a user to interact with a computer using a natural language (Kang et al. 2014). Since an essential task of the dialogue system is to understand what the user says, it must be able to determine the user’s intention indicated in the user’s utterance. A speech-act is a linguistic action and implies the user’s intention. Therefore, the dialogue system must identify the speech-act of user’s utterance. Although researchers have developed many techniques for the speech-act classification, they have mainly used the binary feature weighting scheme because it is simpler but more effective than other schemes such as tf (traditional term frequency), idf (inverse document frequency) and tf.idf (Manning and Schütze, 1999; Salton and Buckley, 1998; Sebastiani, 2002). A utterance is usually much shorter than a document, and it means that the utterance has only the small number of features. For example, as two major factors of traditional tf.idf, tf is the number of term occurrence in a document and df (document frequency) is the number of documents that a term occurs in a collection. In particular, since tf rarely becomes more than 2 in an utterance due to the short length of the utterance, terms with more than 2 frequencies make the distribution of term weights biased and it causes the poor performance of speech-act classification.