Koehn What is a Better Translation ? Reflections on Six Years of Running Evaluation Campaigns
نویسنده
چکیده
To illustrate the problem, see Figure 1, where a short Chinese sentence was translated into English by ten different human translators. Each came up with a different translation. This is a very typical example. Given any sentence of non-trivial length, a group of ten translators will come up with ten different translations. In fact, if presented with the same sentence the next day, they will come up with even more different translations.
منابع مشابه
What is a Better Translation? Reflections on Six Years of Running Evaluation Campaigns
We have been actively involved in the development of machine translation systems and the organization of evaluation campaigns for such systems. We report on the challenges and our experience with automatic and manual metrics used in research.
متن کاملTen Years of WMT Evaluation Campaigns: Lessons Learnt
The WMT evaluation campaign (http://www.statmt.org/wmt16) has been run annually since 2006. It is a collection of shared tasks related to machine translation, in which researchers compare their techniques against those of others in the field. The longest running task in the campaign is the translation task, where participants translate a common test set with their MT systems. In addition to the...
متن کاملSimulating human judgment in machine translation evaluation campaigns
We present a Monte Carlo model to simulate human judgments in machine translation evaluation campaigns, such as WMT or IWSLT. We use the model to compare different ranking methods and to give guidance on the number of judgments that need to be collected to obtain sufficiently significant distinctions between systems.
متن کاملHuman translation and machine translation
While most of recent machine translation work has focus on the gisting application (i.e., translating web pages), another important application is to aid human translators. To build better computer aided translation tools, we first need to understand how human translators work. We discuss how human translators work and what tools they typically use. We also build a novel tool that offers post-e...
متن کاملInnovation, Demand, and Responsibility: Some Fundamental Questions About Health Systems; Comment on “What Health System Challenges Should Responsible Innovation in Health Address? Insights From an International Scoping Review”
In this commentary on the exercise of Lehoux et al (this volume) I argue that in discussions on the current challenges of health systems, a better diagnosis of the health system is required. The cause of responsible innovation in health (RIH) requires a better understanding of the dynamics of health systems, in particular how innovation, demand, and responsibility are m...
متن کامل