LG-Eval: A Toolkit for Creating Online Language Evaluation Experiments
نویسندگان
چکیده
In this paper we describe the LG-Eval toolkit for creating online language evaluation experiments. LG-Eval is the direct result of our work setting up and carrying out the human evaluation experiments in several of the Generation Challenges shared tasks. It provides tools for creating experiments with different kinds of rating tools, allocating items to evaluators, and collecting the evaluation scores.
منابع مشابه
An Introduction to Online and Offline Partial Evaluation using a Simple Flowchart Language
These notes present basic principles of partial evaluation us ing the simple imperative language FCL a language of owcharts intro duced by Jones and Gomard Topics include online partial evaluators o ine partial evaluators and binding time analysis The goal of the lec tures is to give a rigorous presentation of the semantics of partial eval uation systems while also providing details of actual i...
متن کاملThe Potentiality of Dynamic Assessment in Massive Open Online Courses (MOOCs): The Case of Listening Comprehension MOOCs
Massive Open Online Courses (MOOCs) as a new shaking educational development provide the scene for achieving social inclusion and dissemination of knowledge. Anyhow, facilitating network learning experiences through creating an adaptive learning environment can pave the way for this open and energetic way to learning. The present study aimed to explore the possible role of Dynamic Assessment (D...
متن کاملPeer-Assessment and Student-Driven Negotiation of Meaning: Two Ingredients for Creating Social Presence in Online EFL Social Contexts
With the current availability of state-of-the-art technology, particularly the Internet, people have expanded their channels of communication. This has similarly led to many people utilizing technology to learn second/foreign languages. Nevertheless, many current computer-assisted language learning (CALL) programs still appear to be lacking in interactivity and what is termed social presence, w...
متن کاملTutorial on Online Partial Evaluation
This paper is a short tutorial introduction to online partial evaluation. We show how to write a simple online partial evaluator for a simple, pure, first-order, functional programming language. In particular, we show that the partial evaluator can be derived as a variation on a compositionally defined interpreter. We demonstrate the use of the resulting partial evaluator for program optimizati...
متن کاملCreating a Coreference Resolution System for Italian
This paper summarizes our work on creating a full-scale coreference resolution (CR) system for Italian, using BART – an open-source modular CR toolkit initially developed for English corpora. We discuss our experiments on language-specific issues of the task. As our evaluation experiments show, a language-agnostic system (designed primarily for English) can achieve a performance level in high f...
متن کامل