Evaluating Capture-Recapture Models with Two Inspectors

Capture-recapture (CR) models have been proposed as an objective method for controlling software inspections. CR models were originally developed to estimate the size of animal populations. They have also been used to estimate the number of defects in an inspected artifact. Armed with this estimate, one can decide whether the artifact requires a reinspection to ensure that a minimal inspection effectiveness level has been attained. Little evaluative research has been performed thus far on the utility of CR models for inspections with two inspectors. Furthermore, these studies have focused on the relative error of the defect content estimates exclusively. In this paper we report on an extensive Monte Carlo simulation that evaluated six capture-recapture models for two inspectors assuming a code inspections context. In addition to relative error, we evaluate the accuracy of the reinspection decision. The latter is more congruent with the manner in which these models would be used in practice. Our results indicate that the most appropriate capture-recapture model for two inspectors is an estimator originally developed by Chapman that allows for inspectors with different capabilities. This will have a relatively high decision accuracy and will perform better than the default decision of no reinspections. Furthermore, we identify the conditions under which this estimator will perform best.