BACKGROUND Universities in Thailand are preparing for Thailand's integration into the ASEAN Economic Community (AEC) by increasing the number of tests in English language. English language is not the native language of Thailand Differences in English language proficiency may affect scores among test-takers, even when subject knowledge among test-takers is comparable and may falsely represent th...

In the 2012 edition of CLEF, the DI@UE team has signed up for Question Answering for Machine Reading Evaluation (QA4MRE) main task. For each question, our system tries to guess which of the five hypotheses is the more plausible response, taking into account the reading test content and the documents from the background collection on the question topic. For each question, the system applies Name...

Answering multiple-choice questions, where a set of possible answers is provided together with the question, constitutes a simplified but nevertheless challenging area in question answering research. This paper introduces and evaluates two novel techniques for answer selection. It furthermore analyses in how far performance figures obtained using the English language Web as data source can be t...

OBJECTIVES To evaluate Multiple Choice and Short Essay Question items in Basic Medical Sciences by determining item writing flaws (IWFs) of MCQs along with cognitive level of each item in both methods. METHODS This analytical study evaluated the quality of the assessment tools used for the first batch in a newly established medical college in Karachi, Pakistan. First and sixth module assessme...

proper sequencing of multiple choice test items is a matter of concern for many constructors. in this study we compared the variances of three different pharmacology test formats for nursing students. the aim of the study is to see if changing the order of questions will have any effect on the grades of the students. one way analysis of variance (anova) was used to compare the mean score of the...

We describe CMU’s UIMA-based modular automatic question answering (QA) system. This system answers multiplechoice English questions for the world history entrance exam. Questions are preceded by short descriptions providing a historical context. Given the context and question-specific instructions, we generate verifiable assertions for each answer choice. These assertions are evaluated using se...

Multiple choice questions (MCQs) are a common data gathering tool. We extend the Latent Dirichlet Allocation (LDA) framework to a collection of MCQ surveys. Topic discovery is turned into group discovery based on survey response patterns. Question choices are equivalent to vocabulary words and are conditioned on the question and the latent group that is used to cluster the survey responders. Th...

Creating fair and meaningful assessment strategy is one the most difficult areas for every educator. Objective (MCQ) testing has been extensively studied and evaluated as a method for formative and, to a lesser extent, summative assessment. While the objective tests need to be complemented by other assessment strategies in order to assess higher cognitive domains, they nonetheless provide impor...

This paper describes our participation to the Entrance Exams Task of CLEF 2014’s Question Answering Track. The goal is to answer multiple-choice questions on short texts. Our system first retrieves passages relevant to the question, through lexical expansion involving a structured use of the Simple English Wiktionary and WordNet. Then it extracts predicate-argument structures (PAS) from each an...