Automated Assessment of Initial Answers to Questions in Conversational Intelligent Tutoring Systems: Are Contextual Embedding Models Really Better?

This paper assesses the ability of semantic text models to assess student responses electronics questions compared with that expert human judges. Recent interest in similarity has led a proliferation can potentially be used for assessing responses. However, it is unclear whether these perform as well early distributional semantics. We assessed 5166 response pairings 219 participants across 118 and scored each 13 different computational models, including use Regular Expressions, semantics, embeddings, contextual combinations features. Expressions performed best out stand-alone models. show other performing comparably Latent Semantic Analysis model was originally current task, small number cases outperforming model. Models trained on domain-specific corpus task better than general language or Newtonian physics. Furthermore, combined RegEx outperformed agreement Tuning performance recent Automatic Short Answer Grading tasks conversational intelligent tutoring systems requires empirical analysis, especially areas such electronics. Therefore, question arises how embedding compare earlier this answering about These results shed light selection appropriate techniques modeling improve accuracy, recall, weighted agreement, ultimately effectiveness automatic scoring ITSs.