First-time failure rates of candidates for board certification: an educational outcome measure.

BACKGROUND Few objective standards are available to assess the educational effectiveness of ophthalmology residency programs. As a possible measure, we evaluated the first-time failure (FTF) rate in the examinations of the American Board of Ophthalmology, defined as a first-attempt failure in the written examination or a first-attempt failure in the oral examination after having passed the written examination on the first attempt. METHOD We tracked data on all residents who graduated between June 30, 1999, and December 31, 2003, from commencement of training to certification, including rates of overall FTF, written and oral FTF, and program FTF. Performance was analyzed for several factors, including program size. RESULTS The FTF rate was 28% overall and ranged from 0% to 89% across 118 programs (median, 27%). Programs with fewer than 16 graduates per 5 years were significantly more likely to have higher FTF rates than larger programs. Thirty-two programs accounted for 50% of the FTF rate. CONCLUSIONS The FTF rate is a potentially useful measure. However, the small size of many programs contributes to some imprecision. Therefore, this measure should be used in conjunction with other factors when assessing the educational effectiveness of ophthalmology residency programs. Although the eventual certification rate was high, graduates from a few programs appeared inadequately prepared to take the examinations.