The KNIGHT Experiments: Empirically Evaluating an Explanation Generation System
نویسندگان
چکیده
Empirically evaluating explanation generators poses a notoriously difficult problem. To address this problem, we constructed KNIGHT, a robust explanation generator that dynamically constructs natural language explanations about scientific phenomena. We then undertook the most extensive and rigorous empirical evaluation ever conducted on aa explanation generator. First, KNIGHT constructed explanations on randomly chosen topics from the Biology Knowledge Base. This is an immense structure that contains more than 180,000 facts. We then enlisted the services of a panel of domain experts to produce explanations on these same topics. Finally, we submitted all of these explanations to a second panel of domain experts, who graxied the explanations on an A-F scale. KNIGHT scored within "half a grade" of the domain experts. Its performance exceeded that of one of the domain experts.
منابع مشابه
Developing and Empirically Evaluating Robust Explanation Generators: The KNIGHT Experiments
To explain complex phenomena, an explanation system must be able to select information from a formal representation of domain knowledge, organize the selected information into multisentential discourse plans, and realize the discourse plans in text. Although recent years have witnessed significant progress in the development of sophisticated computational mechanisms for explanation, empirical r...
متن کامل(Measuring System Entropy Generation in a Complex Economic Network (The Case of Iran
An economic system is comprised of different primary flows that can be captured in macroeconomic models with complex network relations. Theoretically and empirically in this system, weak substitution or complementarity of environmental materials, like energy and other production factors such as capital, is undeniable. This is an effective critique on neoclassical economics. In this paper, we vi...
متن کاملScaling Up Explanation Generation: Large-Scale Knowledge Bases and Empirical Studies
To explain complex phenomena an explanation system must be able to select information from a formal representation of domain knowledge organize the selected information into multi sentential discourse plans and realize the dis course plans in text Although recent years have witnessed signi cant progress in the development of sophisticated computational mechanisms for explanation empirical resul...
متن کاملOptimal Sizing of a Reliable Hydrogen-based Stand-alone Wind-Fuel Cell System
A hybrid wind/ fuel cell generation system is designed to supply power demand. The aim of this design is to minimize the total cost of the hybrid system over an expected 20 years of operation. The optimization problem is solved aimed at providing a reliable supply for the consumer’s demand. The system consists of fuel cells, some wind units, some electrolyzers, a reformer, an anaerobic reactor ...
متن کاملOptimizing the AGC system of a three-unequal-area hydrothermal system based on evolutionary algorithms
This paper focuses on expanding and evaluating an automatic generation control (AGC) system of a hydrothermal system by modelling the appropriate generation rate constraints to operate practically in an economic manner. The hydro area is considered with an electric governor and the thermal area is modelled with a reheat turbine. Furthermore, the integral controllers and electri...
متن کامل