Following two meetings as a discussion group in the previous international meetings, we moved forward to work within three specifically defined sub-groups formulating existing and future lines of inquiry with related data sets. We aimed to focus on the use of data, which ranged from qualitative think-aloud accounts to pre-post tests asking what is the role of data in our study. If there is a th...

While different approaches to evaluation will yield different results, depending on the purpose of the evaluation this article describes an evaluation approach that was aimed at investigation the mental models of users of the programme. The study was driven by questions about the differences in mental models of the instructional designers and the learners, the time learners spent working throug...

A while back a topologist friend asked me why the set of prime ideals of a ring deserved to be singled out for study. Another friend, a symplectic geometer, asked why anyone would want to put a scheme structure on the paramaters that classify a type of object (i.e. construct a moduli space). Algebraic geometry does have a reputation for being somewhat impenetrable (and I won’t say it’s undeserv...

This paper addresses the issue of creating a realistic laboratory setting when evaluating the usability of mobile systems. Three laboratory-based think-aloud evaluations of the same mobile system were designed and conducted for the purpose of comparing the impact of different approaches to creating realistic laboratory environments on the results subsequently produced. The three evaluations spa...

Employing mixed-method approach, this case study examined the in situ use of educational computer games in a summer math program to facilitate 4th and 5th graders’ cognitive math achievement, metacognitive awareness, and positive attitudes toward math learning. The results indicated that students developed more positive attitudes toward math learning through five-week computer math gaming, but ...

Introduction In this panel session I will be making an introductory presentation, after which we will hear from Tim McAfee and Jodi Jessen. After all of the presenters have been heard from, we will address “questions for the future”. In my talk I would like to review: (a) the value of performance measurement and the diVerent ways in which performance measurement can serve our purposes; (b) the ...

Change-point detection had its origins almost sixty years ago in the work of Page, Shiryayev, and Lorden, who focused on sequential detection of a change-point in an observed process. The process was typically a model for the measured quality of a continuous production process, and the change-point indicated a deterioration in quality that must be detected and corrected. Recently, motivation fr...

Research Question: How do participants behave when asked to rate objects’ visual dimensions? Approach: Qualitative Data Analysis Method: 90 participants were asked to rate 11 objects located along a route of 1.1 km length and asked to explain their rating behaviour using a think-aloud protocol. Results: If asked to rate several objects, participants tend to stop reading the questions as a whole...

The paper proposes an approach to comparative usability evaluation that incorporates important relevant criteria identified in previous work. It applies the proposed approach to a case study of a comparative evaluation of an academic website employing four widely-used usability evaluation methods (UEMs): heuristic evaluation, cognitive walkthroughs, think-aloud protocol and co-discovery learning.

To explore the issue of culture in persuasive technology, we identified strategies distinguishing individualist or collectivist audiences, and developed two versions of a prototype game. In this paper we report on a qualitative study of this game. The game concerned smoking cessation, and was set in a New Zealand context, where one version was designed for individualist New Zealand Europeans, a...