Using Measures of Situation Awareness to Characterize Mental Models in . . .

ZHANG, TAO. Using Measures of Situation Awareness to Characterize Mental Models in Inductive Reasoning Tasks. (Under the direction of Dr. David B. Kaber.) Mental models have been proposed in both the cognitive psychology and human factors literature. However, they have been applied for a variety of related but distinct representations. For system design and user training in practical applications, there remains a need to describe the characteristics of mental models. Situation awareness (SA) has been proposed as a cognitive construct critical in decision making in complex tasks and is considered to be related to operator mental models. However, there have been few empirical studies assessing SA as a basis for mental model characterizations. The primary objective of this research was to develop and validate an empirical method of SA and task performance assessment for characterizing mental models in tasks requiring SA-based decision making and action execution. A pilot study, using a multitasking scenario in virtual reality (VR), was conducted in which participants were required to attend to multiple types of perceptual events occurring randomly in time while carrying on steady physical activity (walking on a treadmill). Responses to SA probes delivered during experiment trials, mental workload ratings and task performance measures were collected and compared with response patterns expected for hypothesized mental model types. Results demonstrated utility of categorical SA responses for identifying different mental model types. However, in the multitasking scenario, the process of developing SA appeared to be substantially influenced by the physical and cognitive task demands and the development of accurate mental models of event distributions appeared to be restricted by this. To extend and further validate the approach developed in the pilot study, an inductive reasoning task paradigm was used in the primary experiment, in which participants were exposed to structured detective case stories. In order to develop a thorough understanding of a story and to identify root causes of crimes, participants needed to form a complex mental model of the story and use this model to predict the process of the crime investigation. Three different types of mental models were hypothesized to occur, including a simple list of story elements, elements grouped based on importance, and an organized network of elements. These models reflected different degrees of understanding of the story during the comprehension aspect of inductive reasoning. Measures of SA collected during participant performance of the task were related to results of another established mental model assessment method, concept mapping, in order to identify any convergence of the two methods. Participant SA and concept map accuracy were also assessed as to their predictability of knowledge test performance at the end of each story. The experiment results showed that a fuzzy inference model was effective for classifications of participant mental models in terms of measures of SA. The model inferences also supported the general hypothesis that mental model complexity increases with cumulative increases in SA across levels (perception, comprehension and projection). SA was found to be independent from concept map measures and predictive of task performance based on the use of mental models in long-term memory (LTM). The experiment results provided further empirical evidence of the utility of SA measures for assessing and describing mental models in complex inductive reasoning tasks. The empirical approach developed and validated in the experiment could be extended to practical applications, such as system design for formation of accurate mental models, development of mental model training programs for error detection and system diagnosis, and investigation of task training protocols for effectiveness in imparting accurate mental models. Using Measures of Situation Awareness to Characterize Mental Models in Inductive Reasoning Tasks