The Comparison of Two Models for Evaluation of Pre-internship Comprehensive Test: Classical and Latent Trait

Introduction: Despite the widespread use of pre-internship comprehensive test and its importance in medical students’ assessment, there is a paucity of the studies that can provide a systematic psychometric analysis of the items of this test. Thus, the present study sought to assess March 2011 pre-internship test using classical and latent trait models and compare their results. Methods: In this cross-sectional descriptive research, item analysis based on classical model was conducted by calculation of difficulty and discrimination coefficients, estimation of reliability by Cronbach’s alpha coefficient method and distracters analysis by comparing distracter proportions using EXCEL software. NOHARM4 software was also used to assess test dimensionality. To assess items parameters (difficulty, discrimination, guessing, information functions, and standard error of measurement) latent trait models and BILOG-MG3 software were used. Results: According to the classical test theory, 30 items (15%) were in the acceptable range of difficulty and discrimination coefficients. Test reliability coefficient was 0.913. Seventy three items (36.5%) had problems with the distracters. The analysis of test dimensionality indicated that the test was unidimensional. Using Three-Parameter Logistic Model, we obtained the mean and standard deviation of items parameters, i.e. item difficulty (0.321, 1.874), item discrimination (1.021, 0.666), and item guessing (0.209, 0.082). Maximum test information function was between 1.0 to 2.8 ability levels, and the test had lower information compared to the higher levels of ability at cutting scores. Conclusion: The findings indicate that analyses of latent trait models can be used to overcome the limitations of the classical test theory.