Evaluating a dialog language generation system: comparing the mountain system to other NLG approaches
نویسندگان
چکیده
This paper describes the MOUNTAIN language generation system, a fully-automatic, data-driven approach to natural language generation aimed at spoken dialog applications. MOUNTAIN uses statistical machine translation techniques and natural corpora to generate human-like language from a structured internal language, such as a representation of the dialog state. We briefly describe the training process for the MOUNTAIN approach, and show results of automatic evaluation in a standard language generation domain: the METEO weather forecasting corpus. Further, we compare output from the MOUNTAIN system to several other NLG systems in the same domain, using both automatic and human-based evaluation metrics; our results show our approach is comparable in quality to other advanced approaches. Finally, we discuss potential extensions, improvements, and other planned tests.
منابع مشابه
Modeling informational novelty in a conversational system with a hybrid statistical and grammar-based approach to surface natural language generation
We present a hybrid statistical and grammar-based system for surface natural language generation (NLG) that uses grammar rules, conditions on using those grammar rules, and corpus statistics to determine the word order. We also describe how this surface NLG module is implemented in a prototype conversational system, and how it attempts to model informational novelty by varying the word order. U...
متن کاملModeling informational novelty in a conversational system with a hybrid statistical and grammar-based approach to natural language generation
We present a hybrid statistical and grammar-based system for surface natural language generation (NLG) that uses grammar rules, conditions on using those grammar rules, and corpus statistics to determine the word order. We also describe how this surface NLG module is implemented in a prototype conversational system, and how it attempts to model informational novelty by varying the word order. U...
متن کاملHYPERBUG: A Scalable Natural Language Generation Approach
A scalable natural language generation (NLG) system called HYPERBUG 1 embedded in an agent-based, multimodal dialog system is presented. To motivate this presentation, several scenarios (including a domain shift) are identified where scalability in dialog systems is really needed, and NLG is argued to be one way of easing this desired scalability. Therefore the novel approach to hybrid NLG in t...
متن کاملDo We Need Deep Generation of Disfluent Dialogue?
Most Natural Language Generation systems focus on the production of perfectly written text for visual presentation. But as NLG tries to move into other areas of language production, such as speech in dialogue systems, it is not clear that approaches for traditional applications are best. Speech exhibits particular features like disfluency that are not found in written discourse. In this paper, ...
متن کامل: Stochastic Natural Language Generation 3 Natural Language Understanding Natural Language Generation Surface
We describe a corpus-based approach to natural language generation (NLG). The approach has been implemented as a component of a spoken dialog system and a series of evaluations were carried out. Our system uses n-gram language models, which have been found useful in other language technology applications, in a generative mode. It is not yet clear whether the simple n-grams can adequately model ...
متن کامل