Minimizing the Testlet Effect: Identifying Critical Testlet Features by Means of Tree-Based Regression
نویسندگان
چکیده
Standardized tests often group items around a common stimulus. Such groupings of items are called testlets. The potential dependency among items within a testlet is generally ignored in practice, even though a basic assumption of item response theory (IRT) is that individual items are independent of one another. A technique called tree-based regression (TBR) was applied to identify key features of stimuli that could properly predict the dependence structure of testlet data. Knowledge about these features might help to develop item sets with small testlet effects. This study illustrates the merits of TBR in the analysis of test
منابع مشابه
LSAC RESEARCH REPORT SERIES Identifying Critical Testlet Features Using Tree-Based Regression: An Illustration With the Analytical Reasoning Section of the LSAT
The Law School Admission Council (LSAC) is a nonprofit corporation that provides unique, state-of-the-art admission products and services to ease the admission process for law schools and their applicants worldwide. More than 200 law schools in the United States, Canada, and Australia are members of the Council and benefit from LSAC's services. may change without notice at any time. Up-to-date ...
متن کاملTestlet-Based Multidimensional Adaptive Testing
Multidimensional adaptive testing (MAT) is a highly efficient method for the simultaneous measurement of several latent traits. Currently, no psychometrically sound approach is available for the use of MAT in testlet-based tests. Testlets are sets of items sharing a common stimulus such as a graph or a text. They are frequently used in large operational testing programs like TOEFL, PISA, PIRLS,...
متن کاملRandom parameter structure and the testlet model: extension of the Rasch testlet model.
The current Rasch testlet model (RT) assumes independence of the testlet effect and the target dimension. This article investigated the impact of the violation of that assumption on RT and the performance of an extended Rasch testlet model (ET) in which the random parameter variance-covariance matrix is estimated without any constraints. Our simulation results showed that ET was the same or sup...
متن کاملRobust Automated Test Assembly for Testlet-Based Tests: An Illustration with Analytical Reasoning Items
In many high-stakes testing programs, testlets are used to increase efficiency. Since responses to items belonging to the same testlet not only depend on the latent ability but also on correct reading, understanding, and interpretation of the stimulus, the assumption of local independence does not hold. Testlet response theory (TRT) models have been developed to deal with this dependency. For b...
متن کاملA Comparison of Item and Testlet Selection Procedures
Testlet response theory (TRT) is a measurement model that can capture local dependency in testlet-based tests. One of the purported advantages of TRT over the more commonly-used polytomous IRT approach to modeling testlet-based tests is that it allows for ad hoc testlet construction in a testlet-based computer adaptive test (CAT). The goal of this study was to investigate the merits of such a C...
متن کامل