An experiment in comparative evaluation: humans vs. computers
نویسنده
چکیده
This paper reports results from an experiment that was aimed at comparing evaluation metrics for machine translation. Implemented as a workshop at a major conference in 2002, the experiment defined an evaluation task, description of the metrics, as well as test data consisting of human and machine translations of two texts. Several metrics, either applicable by human judges or automated, were used, and the overall results were analyzed. It appeared that most human metrics and automated metrics provided in general consistent rankings of the various candidate translations; the ranking of the human translations matched the one provided by translation professionals; and human translations were distinguished from machine translations.
منابع مشابه
Analogical Reasoning and Case Adaptation in Architectural Design: Computers Vs
This paper depicts the studies of the differences between human designers and computers in analogical reasoning and case adaptation. Four design experiments are undertaken to examine how designers conduct case-based design, apply dimensional and topological adaptation. The paper also examines the differences of case adaptation by novice and experienced designers, and between human judgement in ...
متن کاملReaction Time in Phoneme Recognition: A Comparative Study among Iranian Upper-Intermediate vs. Advanced EFL Learners at Institute Level
The present study aimed to investigate of reaction time in terms of phoneme recognition: A comparative study among Iranian Upper-Intermediate vs. Advanced EFL Learners at Institute level. The main question this study tried to answer was whether there is no difference in reaction time in terms of phoneme recognition in Iranian learners at Institute level. To answer the question, 5Upper-Intermedi...
متن کامل“A Comparative Exploration of the Phenomenological Conception of Creative Imagination and Its Role in the Digital and Non-Digital Architectural Design Processes”
Imagination and its relation to creativity are among the most important issues related to the design category in various areas of architectural inquiry such as architectural design, whose function and role have changed in recent decades due to use and application of computers in the architectural design processes. Given all the changes occurred in the meaning and concept of architecture, by the...
متن کاملLong-term facilitation of ventilation and genioglossus muscle activity is evident in the presence of elevated levels of carbon dioxide in awake humans.
We hypothesized that long-term facilitation (LTF) of minute ventilation and peak genioglossus muscle activity manifests itself in awake healthy humans when carbon dioxide is sustained at elevated levels. Eleven subjects completed two trials. During trial 1, baseline carbon dioxide levels were maintained during and after exposure to eight 4-min episodes of hypoxia. During trial 2, carbon dioxide...
متن کاملSocial Categorization and Cooperation between Humans and Computers
Computers increasingly perform a variety of important tasks and services that influence individuals and organizations, yet few studies tell us about how humans interact with computers and other non-human decision-makers. In four experiments, we asked people to engage in cooperation tasks with computers and with humans. Experiment 1 found that people gave more money to a human than a computer. W...
متن کامل