Does number of options in multiple choice tests affect item facility and discrimination? An examination of test-taker preferences

Multiple Choice tests are utilized widely in educational assessment because of their objectivity, ease of scoring, and reliability. This study aimed to compare IF and ID of MC vocabulary test items and attempted to find whether these indices are affected by the number of options. To this end, four 20 item stem equivalent vocabulary tests (3-, 4-, 5-, and 6-option MC) were administered to 194 (106 male and 88 female) pre-intermediate students. Besides, an attitude questionnaire was utilized to examine the attitudes of test takers towards MC test format. Results of one-way ANOVA showed that altering number of options in MC tests does not affect Item Discrimination (ID); however, there were significant differences between Item Facility (IF) of 3-, 5-, and 6-option and 4-, 5-, and 6-option MC test but not between 3- and 4-option MC test, suggesting that 6-option test is the most difficult test. Also, the results of questionnaire revealed test takers’ preference towards the use of 3-option MC. Findings demonstrated that increasing the number of options makes a test more difficult and that choosing the right number of option for MC tests is controversial. Testers are recommended to consider various factors while choosing the right number of options.