In imaging tasks where the observer is uncertain whether lesions are present, and where they could be present, the image is searched for lesions. In the free-response paradigm, which closely reflects this task, the observer provides data in the form of a variable number of mark-rating pairs per image. In a companion paper a statistical model of visual search has been proposed that has parameter...

.................................................................................................................VIII

The receiver operating characteristic (ROC) curve, which is defined as a plot of test sensitivity as they coordinate versus its 1-specificity or false positive rate (FPR) as the x coordinate, is an effective method of evaluating the performance of diagnostic tests. The purpose of this article is to provide a nonmathematical introduction to ROC analysis. Important concepts involved in the correc...

If we consider the Brier score (B) in the context of the signal detection theory and assume that it makes sense to consider the existence of B as a parameter for the population (let B be this B), and if we assume that the calibration in the observer's probability estimate is perfect, we find that there is a theoretical relationship between B and the area under the binormal receiver operating ch...

In many practical classiication problems it is important to distinguish false positive from false negative results when evaluating the performance of the classiier. This is of particular importance for medical diagnostic tests. In this context, receiver operating characteristic (ROC) curves have become a standard tool. Here we apply this concept to characterize the performance of a simple neura...

Eyewitness identification is a pivotal issue in applied research because, in practice, a correct identification can help to remove a dangerous criminal from society, but a false identification can lead to the erroneous conviction of an innocent suspect. Consequently, psychologists have tried to ascertain the best procedures for collecting identification evidence, evaluating them using measures ...

The receiver operating characteristic (ROC) curve is used to describe the performance of a diagnostic test which classifies observations into two groups. We introduce a new method for selecting bandwidths when computing kernel estimates of ROC curves. Our technique allows for interaction between the distributions of each group of observations and gives substantial improvement in MISE over other...

The receiver operating characteristic (ROC) curve is used for classification between two populations. Usually two independent random samples from the populations with absolutely continuous cumulative distribution functions (CDF) F and G are considered. Then the ROC curve is defined as ROC(t) = 1−G(F−1(1− t)) for 0 < t < 1 if the inverse F−1 exists. Classical ROC curve estimator is based on empi...

The receiver operating characteristic (ROC) curve is used to describe the performance of a diagnostic test which classifies observations into two groups. We introduce new methods for selecting bandwidths when computing kernel estimates of ROC curves. Our techniques allow for interaction between the distributions of each group of observations and give substantial improvement in MISE over other p...

Precision-recall (PR) and receiver operating characteristic (ROC) curves are valuable measures of classifier performance. Here, we present the R-package PRROC, which allows for computing and visualizing both PR and ROC curves. In contrast to available R-packages, PRROC allows for computing PR and ROC curves and areas under these curves for soft-labeled data using a continuous interpolation betw...