Evaluating Support for Improvisation in Simulated Emergency Scenarios

Technological systems involving hazards are typically managed by experienced personnel guided by wellformulated, pre-determined procedures. These procedures are designed to ensure that operations proceed in a safe and cost-effective manner. Yet normal operations in these systems are exposed to unexpected contingencies that can require personnel to develop and deploy new procedures in real-time. Creative thinking in such situations is therefore necessary in order to prevent degradation of operations, particularly when there is potential for personal injury, economic loss or environmental damage. One approach to addressing these situations is improvisation. The research described here discusses a series of studies conducted to evaluate the efficacy of a computer-based system for supporting improvisation in simulated crisis situations. The design and implementation of the system are first discussed, drawing upon prior work in blackboard-based systems. The experimental design is then reviewed, followed by a discussion of how the studies were run using groups of emergency response personnel from the Port of Rotterdam in The Netherlands. The group task was to address unexpected contingencies in a timely fashion. A number of measures of group decision effectiveness and uniqueness are presented. Results of the studies suggest that availability of decision support may have had an uneven influence on solution effectiveness and no influence on solution uniqueness. Possible implications for the design of group decision support systems for improvisation are then discussed, along with a number of observations on conducting experimentally-based research on group improvisation.