An Ecobehavioral Assessment of the Teaching Behaviors of Teacher Candidates During Their Special Education Internship Experiences

In the last 20 years, teacher preparation programs have come under close scrutiny by the public and governmental agencies charged with monitoring teacher quality and the academic achievement of American students. Both regular and special education teacher preparation programs struggle with the requirement to collect valid and reliable evidence of teacher candidate performance and their effect on student learning. This study incorporated an ecobehavioral assessment tool (MSCISSAR) in the evaluation of 13 special education teacher candidates during their internship experiences. Special education teacher candidates taught in deaf education classrooms and self-contained and resource rooms for students with disabilities. Results showed that the instructional arrangements, teaching behaviors and student responses were similar to studies using inservice teachers and students with disabilities as subjects. The incorporation of data gathered through the MS-CISSAR program could be used to meet university and NCATE requirements for evidence of teacher candidate performance. Teacher Candidate Assessment 3 An Ecobehavioral Assessment of the Teaching Behaviors of Special Education Teacher Candidates During Their Internship Experiences In today's political climate, teacher instruction and student achievement are often seen as two sides of the same coin. If teachers do their job, students will achieve. If students do not achieve, then teachers are not doing their job. Although many educators balk at the simplistic explanation, public opinion is clear. Sixtyeight percent of Americans believe that every state in America should require a nationally standardized test to measure student achievement. Fifty-three percent favored using a single test to determine student promotion to the next grade. Fiftyseven percent supported a test to determine if students should graduate from high school (Rose & Gallup, 2002). The public is not off the mark in their demands for teachers who can demonstrate educational excellence in the classroom. Qualified teachers have a significant impact on student learning (see Darling-Hammond, 2000). What teachers know and what teachers can do in the classroom have significant influence on what students learn (National Commission on Teaching and America’s Future, 1996). With that knowledge in the minds of educational policymakers in America, high-stakes testing and teacher accountability are inextricably intertwined. Unfortunately, many schools, teachers, and teacher preparation programs remain unprepared to deliver the quality of educational excellence demanded by the public. Teacher Candidate Assessment 4 Expectations of accountability have changed in the last few years. Teachers have responded in a variety of ways. Some teachers strive to improve their use of instructional strategies considered “best practice” by researchers (Vogler, 2002) while teachers lament the atrophy of their creative talents. Many teachers simply leave the field (Tye & O'Brien, 2002). Accountability pressures have filtered down to teacher candidates. In a study of what teacher candidates fear most about their first year of teaching, Gee (2001) found that 45% of the teacher interns indicated fears around "accountability" (planning and implementing state standards and the state-mandated assessment test). Teacher candidate fears around accountability may in part, stem from perceived weaknesses in teacher training programs. The U.S. Secretary of Education, in the Annual Report on Teacher Quality, (U.S. Department of Education, 2002), stated that teacher education and certification are not related to teacher effectiveness. Although many researchers disagree (see DarlingHammond & Youngs, 2002), it remains likely that many of today's teacher education programs are unprepared to provide the kind of training, data collection and support to help teacher candidates learn to manage the pressures of being a classroom teacher in the 21 st century (Wise & Liebbrand, 2000). Beginning in 2000, Colleges of Education accredited by The National Council for Accreditation of Teacher Education (NCATE) must provide multiple samples of reliable and valid evidence that their teacher candidates in regular and Teacher Candidate Assessment 5 special education have mastery of content knowledge in their respective fields, pedagogical knowledge and the effect of the instruction of teacher candidates on student learning. NCATE requires teacher training programs to assess the effectiveness of their programs and use that information to improve aspects of their programs. Teacher training programs are expected to set benchmarks for acceptable teacher performance. Although evidence of grade point averages, Graduate Record Exam scores, portfolios, lesson plans, written reflections, and videos of classroom performance are accepted by NCATE as evidence of teaching expertise, as are data on state licensing exam scores, employer evaluations, and placement rates, the NCATE suggestions provide few measurements of actual teacher performance. Only the videotape sample can show demonstrable evidence that the teacher candidate can actually teach. Further, the No Child Left Behind Act (2001) requires teacher education programs to file formal reports summarizing teacher test results for teacher candidates graduating from their programs. These summary data must also be shared with the public, and states are required to rank order teacher education programs based on these data. Many Colleges of Education have few alternatives for assessment of teaching performance. Indeed, most schools of education have yet to design assessments of actual teaching performance that outline acceptable and unacceptable performance levels (Wise & Liebbrand, 2000). In other words, the Teacher Candidate Assessment 6 NCATE guidelines come up short when helping us answer the question, “can this teacher candidate effectively teach?” The evidence considered acceptable to NCATE may focus more heavily on evidence of "teacher quality,” rather than "teaching quality." Yet, how well a teacher candidate teaches is critical. The quality of teacher preparation can account for 40%-60% of the total variance in student achievement after accounting for student demographics (DarlingHammond, 2000). Effective Teaching and Academic Achievement Researchers over the last three decades have found that academic achievement is, in part, a function of time spent learning the content area combined with the level of active academic responding in tasks that are directly related to the skills that will ultimately be assessed as evidence of academic achievement (Greenwood, Carta, Kamps, & Arreaga-Mayer, 1990). There is a clear correlation between academic gain and academic responding in the classroom. Several studies have described academic responding specifically as reading, (both aloud and silently), writing, academic talk, and task participation, such as manipulating counters for math or using a computer mouse. Academic responding, defined in this manner, has been positively correlated with achievement on standardized tests (Bulgren & Carta, 1993; Greenwood, 1991; Greenwood et al., 1990). In 1994, Greenwood, Terry, Marquis and Walker Teacher Candidate Assessment 7 provided evidence of a causal path between, academic responding, academic achievement and teacher instruction. Low levels of academic responding have been associated with school failure (Cooper & Speece, 1990). However, effective instructional practices have been positively correlated with engaged behavior (Greenwood et al., 1990) and various instructional procedures have been identified as positively or negatively influencing academic responding (Greenwood, Delquadri, Stanley, Sasso, & Whoroton, 1981). Using the right tool, student academic responding is a classroom variable that can be observed and measured. Ecobehavioral Assessment The Ecobehavioral Assessment Instrument An ecobehavioral assessment is a conceptual system of analysis designed to measure behaviors in one or more environments. An ecobehavioral assessment focuses on alternately sampling ecological and behavioral variables and systematically recording them in close temporal sequence. Analysis can reveal both sequential and concurrent interrelationships between the environment and a person's responses (Greenwood, Schulte, Dinwiddie, Kohler, & Carta, 1986). Using an ecobehavioral assessment tool that incorporates a momentary time sampling method, the effects of teachers' choices for instructional methods, instructional delivery, instructional arrangements, and teacher-student interactions are seen in the context of what the students do (academically respond, manage Teacher Candidate Assessment 8 tasks, or engage in competing behaviors). Measures of the teacher's behavior and a target student's behavior in the context of the classroom and instruction are recorded sequentially and repeatedly. The primary focus for ecobehavioral assessment is the identification of the variables surrounding the presence of student academic responding or engagement. Assuming academic responding is correlated with achievement on standardized tests and assuming that we can effectively measure levels of academic responding in students, we should be able to use an ecobehavioral assessment tool to capture the effects of the instruction provided by teacher candidates in the same manner that we collect data on practicing teachers. Using an ecobehavioral assessment tool, teacher educators in special education may have the capability of observing teacher candidates and charting the covariation of student with disabilities’ behaviors in specific environments and in the presence of instructional stimuli. Ecobehavioral Assessment Tools and Teacher Training Teacher education programs are expected to find valid and reliable measures of teacher candidate performance, assess the effectiveness of their programs and use that information to improve their programs (Wise & Liebbrand, 2000). The incorporation of an ecobehavioral assessment tool as a part of the evaluation of preservice teacher candidates can provide valid and reliable data on what the candidates do during a teaching episode. Teacher Candidate Assessment 9 Purpose This study was a field test of the incorporation of an ecobehavioral assessment tool as part of the evaluation of special education teacher candidates’ internship evaluations. The purposes of the present study were to (a) field test the appropriateness and usability of an ecobehavioral assessment (EBASS) tool as a tool for assessment of teaching behaviors in teacher candidates in special education; and (b) describe the teacher behaviors, the ecological and instructional arrangements used by teacher candidates in special education, and the behaviors of the randomly selected students they taught. Research Questions The study was guided by the following research questions: 1. In what ways can ecobehavioral assessment data collected on special education teacher candidates during their internship experiences be incorporated as an assessment of their teaching behaviors? 2. What are the ecological arrangements (instructional grouping, and tasks) used by the selected teacher candidates? 3. What teacher behaviors are most commonly incorporated into lessons taught by teacher candidates? Teacher Candidate Assessment 10 4. To what extent do the behaviors of randomly selected target students represent the following categories: academic responses, task management responses, and competing responses? Method Participants Participants were recruited from the pool of students preparing for internship in the fall of 2002 at an urban university in the southeast. Each potential participant received a letter describing the study and a description of the process for informed consent. Consenting participants also completed a postresearch survey. Thirteen special education teacher interns participated in this study. Eight of the teacher candidates were majors in deaf education. Five of the teacher candidates were majors in exceptional student education. All participants were candidates in an initial certification degree program, receiving either a bachelor’s or a master’s degree in education. One target student was randomly selected for each observation. Settings The 13 special education teacher candidates taught students with one or more disabilities in several typical special education settings: Five teacher candidates taught in public schools in the following classrooms: (a) varying exceptionalities (VE) resource room (grades 1-5), (b) self-contained severe emotional disturbance (grades K-3); (c) self-contained severe emotional Teacher Candidate Assessment 11 disturbance (grades 4/5), (d) self-contained trainable mentally handicapped (grades 7/8), (e) self-contained physically impaired (grades 3-5). Eight teacher candidates were observed at the residential school for the deaf in the following classrooms: (a) kindergarten; (b) early primary (grades 2/3); (c) middle school special needs (grades 6/7/8); and (d) high school classes. Instrument This study used the Mainstream Code for Instructional Structure and Student Academic Response (MS-CISSAR), one of three programs in the EcoBehavioral Assessment System Software (EBASS) package developed by Juniper Gardens Children's Project (Greenwood & Shye, 1995). EBASS is a computer software package that includes three computer programs used to observe, assess, and document the effects of instructional interventions used in classroom instruction (Greenwood, Carta, Kamps, Terry, & Delquadri, 1994). The MSCISSAR program includes 105 codes. Observers used laptop computers to record data during the observations. The MS-CISSAR program is based on a momentary time sampling method of data collection divided into three 20-second intervals that repeat throughout the observation. Classroom ecology (setting, activity, task, physical and instructional arrangement, and teacher definition) is recorded in the first 20-second interval. Teacher behaviors are recorded during the second 20second interval. Student behaviors are recorded during the third interval.