Engaged by Design: Using Simulations to Promote Active Learning

We test our hypothesis that student in-class Internet actions reflect their engagement levels. We predict that an engaging learning environment will result in students performing a higher number of on-task Internet activities. To test our hypothesis, we compare student behaviors during two types of instructional episodes. Students participate in either a traditional, lecture-based lesson or an interactive simulation exercise. To measure student engagement levels, we develop a Classroom Behavioral Analysis System (CBAS) that records all student computer actions during the observed class periods. We then count and label these actions as on-task or off-task, depending on relevance to the classroom activity. We find that students attending the simulation class perform a significantly higher number of on-task actions. These findings support our hypothesis that engaging lessons result in higher levels of on-task Internet activities. Equally important, CBAS accurately reflects student engagement levels and is therefore a promising tool for studying engagement. While engagement and learning seem innately linked, there is little research to quantify relations among classroom activity, student engagement, and positive learning outcomes (Fredricks, et. al., 2004). Part of the difficulty in determining this relation lies in the challenge of measuring student engagement. In a typical classroom situation, at any given time, some students are paying attention to varying degrees and others are not. It is difficult for instructors and researchers to determine the extent to which students are actually engaged with the classroom activities. Behavioral cues, such as students looking at the teacher, may provide some indication of engagement levels; however, students who appear to not be paying attention may be completely engaged and vice-versa. In a classroom context, it is difficult for instructors and researchers to accurately assess which students are engaged and which elements of the classroom activity encourage this engagement. A potential solution to this problem rests in new technologies which offer us a window into student attention that has previously not been available. Previously problematic areas of learning and cognition, such as student attention levels within a classroom, can now be measured using emerging technologies to record and analyze student computer actions (Dickey, 2005; Zhang, et. al., 2005). In this study, we developed a Classroom Behavioral Analysis System (CBAS) to measure student engagement levels as reflected by their on-task and off-task Internet actions. CBAS consisted of monitoring software that recorded all student computer actions performed during a class session, including keystroke activities, active applications, and website visits. At the end of each class session, CBAS reported this information as a log file that we studied to determine whether patterns existed in student engagement levels. This record of Internet activity could then be evaluated in light of the classroom instructional environment. In order to validate CBAS as a tool for measuring engagement, our goal was to study whether student engagement levels were affected by lecture style. To measure these levels, we compared student computer behaviors in a writing class taught by lecture and a writing class taught by using an interactive simulation exercise. We used CBAS to record student Internet activities and then counted student off-task and on-task behaviors. We then compared these behaviors to determine whether student engagement levels, as measured by off-task and on-task Internet actions, were affected by the style of lecture delivery. If CBAS is a valid tool for measuring engagement, on-task behaviors should be higher and off-task behaviors lower in the simulation class than in the lecture class. What mediates the notion that engagement improves learning? Engagement is clearly a central component in many theories of academic learning. In fact, a general assumption of learning studies is that students learn more if they pay attention (Fredricks, et. al., 2004; McMahon & Portelli, 2004). Early studies define engagement in terms of interest (Dewey, 1913), effort (Meece & Blumenfeld, 1988), motivation (Pintrich & DeGroot, 1990; Skinner & Belmont, 1993), and time on task (Berliner, 1979, 1990; Lentz, 1998). In these studies, a conceptual correlation between engagement and positive learning outcomes is established by linking interest, for example, to active learning (Dewey, 1913; Schraw & Lehman, 2001), or effort to goal achievement (Brophy, et. al., 1983; Meece & Blumenfeld, 1988). Research on active engagement consistently shows that when students are focused on a task, they are more likely to apply effort toward their learning experience (Ames 1992; Brophy, et al., 1983). A trend in recent research is to extend our existing understanding of motivation by studying the cognitive strategies that result from its varying levels (Pintrich, 2002; Pressley, 1986; Winne, 1992). Metacognitive control, which is evident in students’ ability to not only know what to do in a learning situation (cognitive strategies), but when to do it, is measured by self-efficacy cues, self-regulation, and goal setting. Pressley (1986) and Winne (1992) argue that metacognitive control is teachable. Current studies of classroom engagement consistently find that the classroom environment, including the lesson plan and lecture delivery style, can affect students’ practice of metacognitive control (Dickey, 2005; Winne, 2006). Importantly, students demonstrating cognitive strategies such as task-mastery goals report higher levels of engagement and perform better on assigned tasks (Ames & Archer, 1988; Meece, 1988). Students who believe they are capable of performing assigned tasks (self-efficacy belief) also demonstrate high academic performance (Ames & Archer, 1983; Schunk, 1989, 1991; Zimmerman & Martinez-Pons, 1990). Research on self-efficacy shows correlations between self-efficacy beliefs and active learning strategies, as well as self-efficacy beliefs and improved performance on achievement tests (Ames, 1992; Pintrich & DeGroot, 1990; Schunk, 1985, 1989; Zimmerman & Martinez-Pons, 1990). Students’ self-efficacy beliefs reflect how well they believe they will perform on a task. According to Mayer (2003), these beliefs come from four sources within the student’s classroom experience: “interpreting one’s own performance, interpreting the performance of others, interpreting others’ expressions of your capabilities, and interpreting one’s physiological state” (p. 470). While factors such as experience and aptitude play a role in the selfefficacy beliefs students bring to the classroom, Ames (1992) asserts that self-efficacy beliefs are also formed during the instructional episode. Since students revise their self-efficacy beliefs based on interpretations of peer and teacher interactions, the presentation of information during the lesson is essential to encouraging high self-efficacy and therefore active engagement in the learning process (Zimmerman & Martinez-Pons, 1990). A basic tenet of these cognitive theories of learning and instruction is that students learn more deeply when they are engaged in active learning than when they are passive recipients of information (Grabinger, 1996; Mayer, 2003; Pearce, 2005). Active learning occurs when a learner engages in active cognitive processing during learning, such as attending to relevant information, organizing the selected information into a coherent cognitive structure, and integrating the information with existing knowledge (Mayer, 2001, 2003). Active learning takes place in an environment where a student is not a passive listener but is instead an active participant in his/her learning experience (Gee, 2003; Jonassen, 1996, 1999). How can new technologies be used to study student engagement? Computer lab settings provide fresh opportunities for measuring classroom behaviors because students use the computer for both course-relevant and recreational activities. In this setting, technologies can be used to collect student behavior data such as applications used, time spent using each application, Internet activities, frequency of attention shifts within program use, and keystrokes. These computer actions offer a window into the cognitive interplay between student and computer. Computer actions show where students focus their attention during the lecture, the duration of this focus, and when this focus shifts. In light of the new measurement opportunities made possible by emerging technologies, it makes sense to return to Berliner’s research (1987, 1990) on student engagement as measured by time on task. In Berliner’s (1979) study of engagement, he assumes a causal relationship between “engaged time” and variability in academic achievement. In our study, we applied Berliner’s concept of time on task to measure student engagement levels. We used CBAS to record student computer actions during a class session and then coded the actions as on-task or off-task. We specifically focused on Internet actions because they provide a clear record of on-task use, which for our study included using the course website, reference pages, or online writing labs. In contrast, we defined off-task use as visiting websites with non-course relevant materials such as sports, gambling, or banking websites. We hypothesize that student computer actions, specifically, their Internet use, will reflect their engagement levels.