Comparing the Fault Detection E ectiveness of Mutation and Data Flow Testing: An Empirical Study

We report results from an experiment to compare the fault detection eeectiveness of mutation, its variants, and data ow testing. As mutation is known to be a costly criterion to satisfy when compared with several other coverage criteria, we compared the fault detection eeectiveness of two variants of the mutation criterion with that of the all-uses criterion. Adequate test sets were generated randomly, as opposed to by human testers as in some previous studies. We view our results in the light of results from earlier studies comparing mutation with path-based testing strategies. We identify and discuss factors that one might consider while evaluating an adequacy criterion for use in practice. Results from our experiments strengthen the hypothesis that an adequacy criterion based on one of the two variants of mutation has superior fault detection eeectiveness than that of the all-uses criterion.