We address the problem of comparing the performance of classifiers. In this paper we study techniques for generating and evaluating confidence bands on ROC curves. Historically this has been done using one-dimensional confidence intervals by freezing one variable—the false-positive rate, or threshold on the classification scoring function. We adapt two prior methods and introduce a new radial s...

The accurate medical diagnostic of a disease condition is fundamental for a correct medical decision. Disease screening programs are based, in general, in diagnostic tests which provide a binary response: a subject is classified as positive, if the test result is above a given threshold, and negative, otherwise. Therefore, false positive and false negative classifications can be generated. The ...

Assessment of predictive accuracy is a critical aspect of evaluating and comparing models, algorithms or technologies that produce the predictions. In the field of medical diagnosis, receiver operating characteristic (ROC) curves have become the standard tool for this purpose and its use is becoming increasingly common in other fields such as finance, atmospheric science and machine learning. T...

Receiver operating characteristic (ROC) curves evaluate the discriminatory power of a continuous marker to predict a binary outcome. The most popular parametric model for an ROC curve is the binormal model, which assumes that the marker, after a monotone transformation, is normally distributed conditional on the outcome. Here, the authors present an alternative to the binormal model based on th...

In this paper we investigate methods to detect and repair concavities in ROC curves by manipulating model predictions. The basic idea is that, if a point or a set of points lies below the line spanned by two other points in ROC space, we can use this information to repair the concavity. This effectively builds a hybrid model combining the two better models with an inversion of the poorer models...