Using Partial Credit and Response History to Model User Knowledge

User modelling algorithms such as Performance Factors Analysis and Knowledge Tracing seek to determine a student’s knowledge state by analyzing (among other features) right and wrong answers. Anyone who has ever graded an assignment by hand knows that some answers are “more wrong” than others; i.e. they display less of an understanding of the skill(s) involved. This investigation seeks to understand the effects of progression through wrong answers to right answers in a way to determine how the “level” of wrongness affects future performance. The key findings are that A.) where in a series of opportunities a student reaches the goal impacts future performance, as does B.) the “level” of previous wrongness, even two questions before the current opportunity. Right students are all alike; every wrong student is wrong in his or her own way. (with apologies to Ms. Karenina and Mr. Tolstoy)