Agreement Between an Isolated Rater and a Group of 1 Raters

11 The agreement between two raters judging items on a categorical scale 12 is traditionally measured by Cohen’s kappa coefficient. We introduce a new 13 coefficient for quantifying the degree of agreement between an isolated rater 14 and a group of raters on a nominal or ordinal scale. The coefficient, which 15 is defined on a population-based model, requires a specific definition of the 16 concept of perfect agreement but possesses the same properties as Cohen’s 17 kappa coefficient. Further, it reduces to the classical kappa when there is 18 only one rater in the group. An intraclass and a weighted versions of the 19 coefficient are also introduced. The new approach overcomes the problem of 20