We assessed the agreement in detection of high-risk human papillomavirus (HPV), as well as specific HPV types, between self- and clinician-obtained specimens for 450 women over 18 years of age attending a community health center in Gugulethu, South Africa. Both self-collected swabs and tampons had high agreement with clinician-obtained brushes when the Roche Reverse Line Blot Assay (RLBA) was u...

Final amendment note: This paper had raised two issues the error-prone classification and mistaken application of the Kappa statistic. The classification critique still holds, and is being taken up with other criticisms at http://www.biorxiv.org/content/early/2017/07/02/146134. The Kappa statistic was an error on my part since I had failed to see another page in Table S1. Please consider this p...

BACKGROUND Missing data present a challenge to many research projects. The problem is often pronounced in studies utilizing self-report scales, and literature addressing different strategies for dealing with missing data in such circumstances is scarce. The objective of this study was to compare six different imputation techniques for dealing with missing data in the Zung Self-reported Depressi...

AAAI 2004 Spring Symposium Exploring Attitude and Affect in Text Department of Linguistics Simon Fraser University Burnaby, B.C., V5A 1S6, Canada Carletta, J. 1996. Assessing agreement on classification tasks: the kappa statistic. 22 (2): 249-154. Hatzivassiloglou, V., and McKeown, K. 1997. Predicting the semantic orientation of adjectives. In , 174-181. Krippendorf, K. 1980. Beverly Hills, CA:...

Five classification algorithms namely J48, Naive Bayes, Multilayer Perceptron, IBK and Bayes Net are evaluated using Mc Nemar’s test over datasets including both nominal and numeric attributes. It was found that Multilayer Perceptron performed better than the two other classification methods for both nominal and numerical datasets. Furthermore, it was observed that the results of our evaluation...

This paper describes a method for evaluating interannotator reliability in an email corpus annotated for type (e.g., question, answer, social chat) when annotators are allowed to assign multiple labels to a message. An augmentation is proposed to Cohen’s kappa statistic which permits all data to be included in the reliability measure and which further permits the identification of more or less ...

In order to assess the reliability of a given characterization of a subject it is often necessary to obtain multiple readings, usually but not always from different individuals or raters. The degree of agreement among the various raters gives some indication as to the consistency of the values. If agreement is high, we feel more confident the ratings reflect the actual circumstance. If agreemen...

This paper describes a multi-method approach to assessment of similarity of categorical maps. The assessment incorporates several newly developed comparison methods. Some are related to the Kappa statistic, others are applications of Fuzzy Set theory. By combining the methods a broad assessment of similarity will be obtained, which makes it possible to find the magnitude, nature and spatial dis...

Sample size requirements that achieve a prespecified expected lower limit for a confidence interval about the intraclass kappa statistic are supplied for the case of multiple raters and a binary outcome variable. The expected lower confidence limit achievable for a given number of subjects and raters is also presented. These results should be useful in the planning stages of an interobserver ag...

The performance of the Capilia TB-Neo assay, a new-generation assay, was assessed by determining its sensitivity, specificity, reproducibility, and cross-reaction with contaminating organisms. The sensitivity and specificity were 99.2 and 96.4% and 89.3 and 100% in pure and mixed-culture isolates, respectively. The kappa statistic was 95.0 and 77.9% in pure and mixed culture isolates, respectiv...