Comparing ϕ and the F-measure as performance metrics for software-related classifications

Abstract Context The F-measure has been widely used as a performance metric when selecting binary classifiers for prediction, but it also criticized, especially given the availability of alternatives such ϕ (also known Matthews Correlation Coefficient). Objectives Our goals are to (1) investigate possible issues related in depth and show how can address them, (2) explore relationships between . Method Based on definitions F-measure, we derive few mathematical properties these two metrics them. To demonstrate practical effects properties, illustrate outcomes an empirical study involving 70 Empirical Software Engineering datasets 837 classifiers. Results We that be defined function Precision Recall , which only define rate actually positive software modules dataset. Also, expressed rates actual estimated modules. minimum maximum value any conditions under both rank same order. Conclusions results is sensible useful assessing recommend should not by itself assess classifier, positives always specified well, at least if what extent classifier performs better than random classification. described here re-interpret conclusions previously published papers relied mainly metric.