Evaluating Multiple Aspects of Coherence in Student Essays
نویسندگان
چکیده
Criterion Online Essay Evaluation Service includes a capability that labels sentences in student writing with essay-based discourse elements (e.g., thesis statements). We describe a new system that enhances Criterion’s capability, by evaluating multiple aspects of coherence in essays. This system identifies features of sentences based on semantic similarity measures and discourse structure. A support vector machine uses these features to capture breakdowns in coherence due to relatedness to the essay question and relatedness between discourse elements. Intra-sentential quality is evaluated with rule-based heuristics. Results indicate that the system yields higher performance than a baseline on all three aspects.
منابع مشابه
Holistic Annotation of Discourse Coherence Quality in Noisy Essay Writing
In this paper, we describe a holistic annotation scheme for coherence quality that requires little expertise on the part of the human annotator; we also present computational systems for evaluating coherence quality in essays. A formidable challenge to reliability when annotating discourse coherence comes from differences among annotators in the inferences that they draw when reading an essay. ...
متن کاملConnection between Emotional Intelligence and Coherence in EFL Writing
This research is an investigation of the relationship between the emotional intelligence of a group of Iranian English language learners and the coherence of the essays they produced. A 40-item TOEFL-type test of English proficiency including items on grammar, vocabulary, reading comprehension with the Cronbach’s alpha reliability of .742, was planned to identify the proficient candidates who c...
متن کاملUsing Entity-Based Features to Model Coherence in Student Essays
We show how the Barzilay and Lapata entitybased coherence algorithm (2008) can be applied to a new, noisy data domain – student essays. We demonstrate that by combining Barzilay and Lapata’s entity-based features with novel features related to grammar errors and word usage, one can greatly improve the performance of automated coherence prediction for student essays for different populations.
متن کاملToward Automatic Inference of Causal Structure in Student Essays
With an increasing focus on science and technology in education comes an awareness that students must be able to understand and integrate scientific explanations from multiple sources. As part of a larger project aimed at deepening our understanding of student processes for integrating multiple sources of information, we are developing machine learning and natural language processing techniques...
متن کاملOrganization of ideas in writing: what are raters sensitive to?
Whether foreign language writing is rated using analytic rating scales or holistically, the organization of ideas is invariably one of the aspects assessed. However, it is unclear what raters are sensitive to when rating writing for organization. Which do they value more highly, the physical aspects of organization, such as paragraphing and the existence of organization markers, or deeper textu...
متن کامل