BACKGROUND The usual kappa statistic requires that all observations be enumerated. However, in free-response assessments, only positive (or abnormal) findings are notified, but negative (or normal) findings are not. This situation occurs frequently in imaging or other diagnostic studies. We propose here a kappa statistic that is suitable for free-response assessments. METHOD We derived the eq...

In recent years, the coefficient of agreement has become the de facto standard to evaluate intercoder agreement for tagging tasks. In this squib, we highlight issues that affect and that the community has largely neglected. First, we discuss the assumptions underlying different computations of the expected agreement component of . Second, we discuss how prevalence and bias affect the measure. I...

Map comparison methods are used to express the similarity between maps. They are amongst others used in validation and calibration procedures of spatial models. It is found that traditional map comparison methods do not distinguish small differences from large differences, which leads to comparison results that are not in accordance with human judgment. In this paper a relatively new method for...

Background Cohen's Kappa is the most used agreement statistic in literature. However, under certain conditions, it is affected by a paradox which returns biased estimates of the statistic itself. Objective The aim of the study is to provide sufficient information which allows the reader to make an informed choice of the correct agreement measure, by underlining some optimal properties of Gwet...

The command kapci calculates 100(1 − α)% confidence intervals for the kappa statistic using an analytical method in the case of dichotomous variables or bootstrap for more complex situations. For instance, kapci allows estimating CI for polychotomous variables using weighted kappa or for cases in which there are more than 2 raters/replications.

OBJECTIVE To compare a novel computerized analysis program with visual cardiotocography (CTG) interpretation results. METHODS Sixty-two intrapartum CTG tracings with 20- to 30-minute sections were independently interpreted using a novel computerized analysis program, as well as the visual interpretations of eight obstetricians, to evaluate the baseline fetal heart rate (FHR), baseline FHR var...

The weighted kappa statistic has been used as an agreement index for ordinal data. Using data on the comparability of primary and proxy respondent reports of alcohol drinking frequency we show that the value of weighted kappa can be sensitive to the choice of weights. The distinction between association and agreement is clarified and it is shown that in some respects weighted kappa behaves more...

The Fuzzy Kappa statistic expresses the agreement between two categorical raster maps. The statistic goes beyond cell-by-cell comparison and gives partial credit to cells based on the categories found in the neighborhood. When matching categories are found at shorter distances the agreement is higher. Like the well-established Kappa statistic the Fuzzy Kappa statistic expresses the mean agreeme...