In receiver operating characteristic (ROC) analysis, the sampling variability can result in uncertainties of performance measures. Thus, while evaluating and comparing the performances of algorithms, the measurement uncertainties must be taken into account. The key issue is how to calculate the uncertainties of performance measures in ROC analysis. Our ultimate goal is to perform the significan...

Evaluating classifier performance with ROC curves is popular in the machine learning community. To date, the only method to assess confidence of ROC curves is to construct ROC bands. In the case of severe class imbalance, ROC bands become unreliable. We propose a generic framework for classifier evaluation to identify the confident segment of an ROC curve. Confidence is measured by Tango’s 95%-...

It has been known for over 40 years that there are two fundamentally different kinds of detection tasks in the theory of signal detectability. The Type 1 task is to distinguish between events defined independently of the observer; the Type 2 task is to distinguish between one's own correct and incorrect decisions about those Type 1 events. For the Type 1 task, the behavior of the detector can b...

background: the diagnosis of pulmonary tuberculosis (ptb) is complicated and time-consuming currently. there was association of ptb with serum tumor markers. in this study we aimed to evaluate the predictive role of serum ca125, ca199 and cea as diagnostic tools for ptb. methods: this study was designed as a case-control study with 565 subjects who visited the yijishan hospital from jun to dec ...

background: metabolic syndrome (metsy), an important predisposing factor for the most of noncommunicable diseases, has become a global pandemic. given different definitions used for the metsy, recently using a score termed “continuous metsy risk score (cmetsys)” is recommended. the aim of this study was to provide a cmetsys in a population-based sample of iranian adults and to assess its determ...

A synthesis of the empirical Bayes method and the method of estimating equations is used to combine individual receiver operating characteristic (ROC) area estimates from different studies of the same diagnostic test into a single estimate. This single estimate represents the population mean from which individual areas under the ROC curves were sampled. The only data needed to carry out the met...

In this paper we establish the empirical likelihood method for the two-sample case in a general framework. We show that the result of Qin and Zhao (2000) basically covers the following two-sample models: the differences of two sample means, smooth Huber estimators, distribution and quantile functions, ROC curves, probability-probability (P-P) and quantile-quantile (Q-Q) plots. Finally, the stru...

Results The patients with active disease had greater IL-6 levels than did the patients with inactive disease (p=0.002) and controls (p=0.006).The cutoff value for active disease obtained from the ROC curve was 8,33pg/ml. During the active stage of the disease IL-6 levels were significantly higher in both systemic and polyarticular patients than in the oligoarticular patients. Serum IL-6 concent...

This paper shows that when a classifier is evaluated with nonrandom test data, ROC curves differ from the ROC curves that would be obtained with a random sample. To address this bias, this paper introduces a procedure for plotting ROC curves that are inferred from nonrandom test data. I provide simulations and an example with wine data to illustrate the procedure as well as the magnitude of bia...

In the remember-know paradigm for studying recognition memory, participants distinguish items whose presentations are episodically remembered from those that are merely familiar. A one-dimensional model postulates that remember responses are just high-confidence old judgments, but a meta-analysis of 373 experiments shows that the receiver operating characteristic (ROC) curves predicted by this ...