Intrinsic vs. Extrinsic Evaluation Measures for Referring Expression Generation
نویسندگان
چکیده
In this paper we present research in which we apply (i) the kind of intrinsic evaluation metrics that are characteristic of current comparative HLT evaluation, and (ii) extrinsic, human task-performance evaluations more in keeping with NLG traditions, to 15 systems implementing a language generation task. We analyse the evaluation results and find that there are no significant correlations between intrinsic and extrinsic evaluation measures for this task.
منابع مشابه
Attribute Selection for Referring Expression Generation: New Algorithms and Evaluation Methods
Referring expression generation has recently been the subject of the first Shared Task Challenge in NLG. In this paper, we analyse the systems that participated in the Challenge in terms of their algorithmic properties, comparing new techniques to classic ones, based on results from a new human task-performance experiment and from the intrinsic measures that were used in the Challenge. We also ...
متن کاملTowards an Extrinsic Evaluation of Referring Expressions in Situated Dialogs
In the field of referring expression generation, while in the static domain both intrinsic and extrinsic evaluations have been considered, extrinsic evaluation in the dynamic domain, such as in a situated collaborative dialog, has not been discussed in depth. In a dynamic domain, a crucial problem is that referring expressions do not make sense without an appropriate preceding dialog context. I...
متن کاملIntrinsic vs. Extrinsic Motivation in an Interactive Engineering Game
In this paper, we study intrinsic vs. extrinsic motivation in players playing an electrical engineering gaming environment. We used UNTANGLED, a highly interactive game to conduct this study. This game is developed to solve complex mapping problem from electrical engineering using human intuitions. Our goal is to find whether there are differences in the ways anonymous players solved electrical...
متن کاملThe Psychological Function of Nostalgia: Increasing Intrinsic Self-Expression and Decreasing Extrinsic Self-Definition
Previous findings have shown the many functions of nostalgia, for example, increasing mental health, and the current research aimed to explain those findings by testing the theoretical explanation that the benefits of nostalgia come from the fact that nostalgia reminds us of our intrinsic self. Aiming to investigate the effect of nostalgia on authenticity (intrinsic self-expression) and extrins...
متن کاملهستیشناسی مکان تاریخی (تأملی هستیشناسانه در مکان تاریخی در پرتو اصالت وجود)
A review of the history of architectural conservation implies that ontological attitude of scholars and theorists in the field of historic places› conservation is one of the most important principles of conservation theories. Those features of historic places which are considered as more significant at the moment of recognition are often turned into an issue for conservation. However, there is ...
متن کامل