We propose serial correlation robust asymptotic confidence bands for the receiver operating characteristic (ROC) curves estimated by quasi-maximum likelihood in the binormal model. Our simulation experiments confirm that this new method performs fairly well in finite samples. The conventional procedure is found to be markedly undersized in terms of yielding empirical coverage probabilities lowe...

Measures including sensitivity, specificity, and positive and negative predictive values have been traditionally used to assess a diagnostic test's ability to detect the presence or absence of disease. Receiver operating characteristic (ROC) curve analysis allows visual evaluation of the trade-offs between sensitivity and specificity associated with different values of the test result, or diffe...

Receiver operating characteristic (ROC) curve, plotting true positive rates against false positive rates as threshold varies, is an important tool for evaluating biomarkers in diagnostic medicine studies. By definition, ROC curve is monotone increasing from 0 to 1 and is invariant to any monotone transformation of test results. And it is often a curve with certain level of smoothness when test ...

The receiver operating characteristic (ROC) curve is used to describe the performance of a diagnostic test which classifies observations into two groups. We introduce new methods for selecting bandwidths when computing kernel estimates of ROC curves. Our techniques allow for interaction between the distributions of each group of observations and give substantial improvement in MISE over other p...

High detection probabilities with low false alarm rates, even in the presence of as much as – 28 dB signal to clutter can be obtained, if use is made of the coherence information contained in the input signal. Such a detection system is described and its Receiver Operating Curves (ROC) characterized from a reduced experimental set of data. The system has an input filter with a rectangular passb...

Automatic detection of sentences in speech is useful to enrich speech recognition output and ease subsequent language processing modules. In the recent NIST evaluations for this task, an error rate was used to evaluate system performance. A variety of metrics such as F-measure, ROC or DET curves have also been explored in other studies. This paper aims to take a closer look at the evaluation is...

We develop a permutation test for assessing a difference in the areas under the curve (AUCs) in a paired setting where both modalities are given to each diseased and nondiseased subject. We propose that permutations be made between subjects specifically by shuffling the diseased/nondiseased labels of the subjects within each modality. As these permutations are made within modality, the permutat...

An empirical study of the relationship between object-oriented (OO) metrics and error-severity categories is presented. The focus of the study is to identify threshold values of software metrics using receiver operating characteristic curves. The study used the three releases of the Eclipse project and found threshold values for some OO metrics that separated no-error classes from classes that ...

BACKGROUND ROC curves have become the standard for describing and comparing the accuracy of diagnostic tests. Not surprisingly, ROC curves are used often by clinical chemists. Our aims were to observe how the accuracy of clinical laboratory diagnostic tests is assessed, compared, and reported in the literature; to identify common problems with the use of ROC curves; and to offer some possible s...