Following two meetings as a discussion group in the previous international meetings, we moved forward to work within three specifically defined sub-groups formulating existing and future lines of inquiry with related data sets. We aimed to focus on the use of data, which ranged from qualitative think-aloud accounts to pre-post tests asking what is the role of data in our study. If there is a th...

While different approaches to evaluation will yield different results, depending on the purpose of the evaluation this article describes an evaluation approach that was aimed at investigation the mental models of users of the programme. The study was driven by questions about the differences in mental models of the instructional designers and the learners, the time learners spent working throug...

A while back a topologist friend asked me why the set of prime ideals of a ring deserved to be singled out for study. Another friend, a symplectic geometer, asked why anyone would want to put a scheme structure on the paramaters that classify a type of object (i.e. construct a moduli space). Algebraic geometry does have a reputation for being somewhat impenetrable (and I won’t say it’s undeserv...

This paper addresses the issue of creating a realistic laboratory setting when evaluating the usability of mobile systems. Three laboratory-based think-aloud evaluations of the same mobile system were designed and conducted for the purpose of comparing the impact of different approaches to creating realistic laboratory environments on the results subsequently produced. The three evaluations spa...

Employing mixed-method approach, this case study examined the in situ use of educational computer games in a summer math program to facilitate 4th and 5th graders’ cognitive math achievement, metacognitive awareness, and positive attitudes toward math learning. The results indicated that students developed more positive attitudes toward math learning through five-week computer math gaming, but ...

We examined onset reaction time (RT) in a word naming task using an additive factors method (AFM). The pattern of additive and over-additive joint effects on RT among Instructions (INST: name all, name words), Word Frequency (WF: log(10) HAL), Semantic Neighborhood Density (SND: Inverse Ncount), and Word Type (WT: regular, exception) supported a cognitive chronometric architecture consisting of...

Introduction In this panel session I will be making an introductory presentation, after which we will hear from Tim McAfee and Jodi Jessen. After all of the presenters have been heard from, we will address “questions for the future”. In my talk I would like to review: (a) the value of performance measurement and the diVerent ways in which performance measurement can serve our purposes; (b) the ...

Change-point detection had its origins almost sixty years ago in the work of Page, Shiryayev, and Lorden, who focused on sequential detection of a change-point in an observed process. The process was typically a model for the measured quality of a continuous production process, and the change-point indicated a deterioration in quality that must be detected and corrected. Recently, motivation fr...

We investigated the differential effects of self-explaining a refutational text, compared to thinking aloud or rereading. Undergraduate students (n = 105) read a refutational text about natural selection and were asked to either self-explain, think-aloud, or re-read the text. Then they completed a posttest that assessed general knowledge of natural selection. Students who self-explained the ref...

The production effect, the memorial benefit for information read aloud versus silently, has been touted as a simple memory improvement tool. The current experiments were designed to evaluate the relative costs and benefits of production using a free recall paradigm. Results extend beyond prior work showing a production effect only when production is manipulated within subject, not between, usin...