Question Generation Shared Task and Evaluation Challenge - Status Report
نویسندگان
چکیده
The First Shared Task Evaluation Challenge on Question Generation took place in 2010 as part of the 3 workshop on Question Generation. The campaign included two tasks: Question Generation from Sentences and Question Generation from Paragraphs. This status report briefly summarizes the motivation, tasks and results. Lessons learned relevant to future QG-STECs are also offered.
منابع مشابه
The First Question Generation Shared Task Evaluation Challenge
The paper briefly describes the First Shared Task Evaluation Challenge on Question Generation that took place in Spring 2010. The campaign included two tasks: Task A – Question Generation from Paragraphs and Task B – Question Generation from Sentences. An overview of each of the tasks is provided.
متن کاملA Detailed Account of The First Question Generation Shared Task Evaluation Challenge
The paper provides a detailed account of the First Shared Task Evaluation Challenge on Question Generation that took place in 2010. The campaign included two tasks that take text as input and produce text, i.e. questions, as output: Task A – Question Generation from Paragraphs and Task B – Question Generation from Sentences. Motivation, data sets, evaluation criteria, guidelines for judges, and...
متن کاملCollecting Reliable Human Judgements on Machine-Generated Language: The Case of the QG-STEC Data
Question generation (QG) is the problem of automatically generating questions from inputs such as declarative sentences. The Shared Evaluation Task Challenge (QG-STEC) Task B that took place in 2010 evaluated several state-of-the-art QG systems. However, analysis of the evaluation results was affected by low inter-rater reliability. We adapted Nonaka & Takeuchi’s knowledge creation cycle to the...
متن کاملA New Approach to Ranking Over-Generated Questions
We discuss several improvements to the Question Generation Shared Task Evaluation Challenge (QGSTEC) system developed at the University of Pennsylvania in 2010. In addition to enhancing the question generation rules, we have implemented two new components to improve the ranking process. We use topic scoring, a technique developed for summarization, to identify important information for question...
متن کاملThe TUNA-REG Challenge 2009: Overview and Evaluation Results
The TUNA-REG’09 Challenge was one of the shared-task evaluation competitions at Generation Challenges 2009. TUNAREG’09 used data from the TUNA Corpus of paired representations of entities and human-authored referring expressions. The shared task was to create systems that generate referring expressions for entities given representations of sets of entities and their properties. Four teams submi...
متن کامل