Design Scenarios as an Assessment of Adaptive Expertise*

The ability to adapt to new challenges is critical to success in rapidly advancing fields. However, educators and researchers struggle with how to measure and teach for adaptive expertise. This study used a design scenario to assess how undergraduates approach novel design challenges. A scenario presents individual students with a short realistic description of a complex, open-ended design problem. In this study, we developed a scenario from a cardiologist's concerns about the design of an implantable defibrillator. Participants included 63 senior design students and 37 freshmen enrolled in a signal analysis course. After reading the scenario, students responded to three questions: What do you need to do to test the doctor's hypothesis? What questions do you have for the doctor? and, How confident are you in your response? The first question tapped students' efficiency or their ability to devise an appropriate response. The second tapped students' innovation or their ability to consider important facets of the problem. The third question estimated students' confidence/cautiousness. Data were collected at the beginning and end of one semester. Analysis showed that seniors consistently devised more efficient and innovative solutions than did freshmen. Seniors were also more confident in their problem-solving abilities. Over time, all students became more innovative and more confident. Findings are discussed in terms of what they suggest about undergraduates' intellectual development at entry to and exit from a standard fouryear curriculum and how adaptive expertise might be assessed within the context of students' regular academic coursework.