Measuring Situation Awareness during Command and Control Activity: A Comparison of Measures Study

The level of Situation Awareness (SA) that systems and devices provide to individuals and teams is a critical factor that needs assessing throughout the design process. There are various methods available for assessing SA, yet there is little guidance available on which of these methods are the most valid, reliable and accurate. The purpose of this study was to compare three existing SA measures when used to measure participant SA during a command and control experiment. This research aimed to compare three popular SA measures when used to assess participant SA during a command and control task. Specifically the aim was to better understand how each of the approaches used viewed SA during the command and control task. A total of 20 participants undertook to answer Question One of the Combat Estimate (a military planning process) for an experimental warfare scenario using the Brunel command wall system. Question one involves conducting a Battlefield Area Evaluation (BAE) that is designed to enhance the Commander's SA of the enemy and threat. Participant SA was measured using three different techniques: the Situation Awareness Global Assessment Technique (SAGAT), which is a freeze probe technique; the Situation Awareness Rating Technique (SART), which is a post trial subjective rating technique; and the Critical Decision Method (CDM), which is a post task interview approach. The results were then analysed in order to identify any correlations between the SA measures and between participant SA and performance on the command and control task. Of the three methods used, only the participants' SAGAT overall and level 2 scores produced a significant correlation with performance. It was concluded from this that the SAGAT approach was the most accurate at measuring participant SA during the command and control task. There was no significant correlation between participant levels of SA as elicited by the three SA measures used. It was concluded from this that SAGAT, SART and CDM, measure SA from differing perspectives. The extent to which the phenomenon that they are measuring represents the construct of SA is an important area of future research that has emerged from this study. There was also no significant correlation between mental workload and SA found in this study. However, a significant negative correlation was found between the time taken to construct a situation overlay and its accuracy. v The SA measures used were also compared using a qualitative set of evaluation criteria. It was concluded …