Receiver operating characteristic (ROC) movies, universal ROC (UROC) curves, and coefficient of predictive ability (CPA)

Abstract Throughout science and technology, receiver operating characteristic (ROC) curves associated area under the curve ( $$\mathrm{AUC}$$ AUC ) measures constitute powerful tools for assessing predictive abilities of features, markers tests in binary classification problems. Despite its immense popularity, ROC analysis has been subject to a fundamental restriction, that it applies dichotomous (yes or no) outcomes only. Here we introduce movies universal (UROC) apply just any linearly ordered outcome, along with an coefficient ability $${\mathrm{CPA}}$$ CPA measure. equals UROC curve, admits appealing interpretations terms probabilities rank based covariances. For , pairwise distinct relates Spearman’s coefficient, same way C index Kendall’s coefficient. movies, curves, nest generalize classical analysis, are bound supersede them wealth applications. Their usage is illustrated data examples from biomedicine meteorology, where yield new insights WeatherBench comparison performance convolutional neural networks physical-numerical models weather prediction.