An Illustration of a Mantel-Haenszel Procedure to Flag Misbehaving Common Items in Test Equating - Practical Assessment, Research & Evaluation
نویسنده
چکیده
In this study the Mantel-Haenszel procedure, widely used in studies for identifying differential item functioning, is proposed as an alternative to the delta-plot method and applied in a test-equating context for flagging common items that behave differentially across cohorts of examinees. The Mantel-Haenszel procedure has the advantage of conditioning on ability when making comparisons of performance of two examinee groups on an item. There are schemes for interpreting the effect size of differential performance, which can inform the decision as to whether to retain those items in the common-item pool, or to discard them. Data from a statewide assessment are analyzed to illustrate the use of this procedure. Advantages of this methodology are discussed and limitations regarding test design that may make its application difficult are described.
منابع مشابه
An Illustration of a Mantel-Haenszel Procedure to Flag Misbehaving Common Items in Test Equating
In this study the Mantel-Haenszel procedure, widely used in studies for identifying differential item functioning, is proposed as an alternative to the delta-plot method and applied in a test-equating context for flagging common items that behave differentially across cohorts of examinees. The Mantel-Haenszel procedure has the advantage of conditioning on ability when making comparisons of perf...
متن کاملEffects of Misbehaving Common Items on Aggregate Scores and an Application of the Mantel-Haenszel Statistic in Test Equating
Consistent behavior is a desirable characteristic that common items are expected to have when administered to different groups. Findings from the literature have established that items do not always behave in consistent ways; item indices and IRT item parameter estimates of the same items differ when obtained from different administrations. Content effects, such as discrepancies in instructiona...
متن کاملAcademic Discipline DIF in an English Language Proficiency Test
The purpose of this study was to detect differentially functioning items in the University of Tehran English Proficiency Test (UTEPT) which is a high stake test of English developed and administered by the Language Testing Centre of the University of Tehran. This paper is based on the answers of 400 test takers to the test. All participants earned a master degree either in humanities or science...
متن کاملA Review of the Effects on IRT Item Parameter Estimates with a Focus on Misbehaving Common Items in Test Equating
Many studies have investigated the topic of change or drift in item parameter estimates in the context of item response theory (IRT). Content effects, such as instructional variation and curricular emphasis, as well as context effects, such as the wording, position, or exposure of an item have been found to impact item parameter estimates. The issue becomes more critical when items with estimat...
متن کاملA new approach for differential item functioning detection using Mantel-Haenszel methods. The GMHDIF program.
To date, the statistical software designed for assessing differential item functioning (DIF) with Mantel-Haenszel procedures has employed the following statistics: the Mantel-Haenszel chi-square statistic, the generalized Mantel-Haenszel test and the Mantel test. These statistics permit detecting DIF in dichotomous and polytomous items, although they limit the analysis to two groups. On the con...
متن کامل