Seizure duration was estimated using a single channel EEG recording during ECT. Seizure endpoint was defined as the point of last spike. Three psychiatrists independently rated forty seven EEG records. Seizure endpoint could be reliably identified. The reliability coefficient between three raters was 0. 92, indicating high agreement among raters. Modal difference between any pair of raters as w...

Using a mixed methods design, the present study attempted to identify the factors influencing Iranian untrained EFL raters in rating group oral discussion tasks. To fulfil this aim, 16 language learners of varying proficiency levels were selected and randomly assigned to groups of four and performed a group discussion task. Thirty two untrained raters were also selected based on their volunteer...

Ratings on performance tasks from two scoring sessions of an eighth grade mathematics examination, developed by the California State Department of Education, were used (a) to study the feasibility of estimating IRT rater severity information within a scoring session, (b) to investigate the variation in rater severity within rating sessions (which we called rater drift), and (c) to examine the r...

PURPOSE The Upper Limb Physician's Rating Scale (ULPRS) is a tool that assesses movement quality of the upper limbs. It is used as an outcome measure after botulinum toxin type A injection in children with cerebral palsy (CP). This study aimed to investigate the reliability and validity of the ULPRS in children with spastic CP. MATERIALS AND METHODS Thirty children with spastic CP (M:F=17:13)...

Machine learning models are commonly used to detect toxicity in online conversations. These trained on datasets annotated by human raters. We explore how raters' self-described identities impact they annotate comments. first define the concept of Specialized Rater Pools: rater pools formed based identities, rather than at random. three such for this study - specialized raters from U.S. who iden...

AbstractCalculation of a confidence interval for intraclass correlation to assess inter-rater reliability is problematic when the number of raters is small and the rater effect is not negligible. Intervals produced by existing methods are uninformative: the lower bound is often close to zero, even in cases where the reliability is good and the sample size is large. In this paper, we show that t...

BACKGROUND The advent of global clinical trials has necessitated the use of English-based rating instruments in diverse cultures where English is clearly not the primary language. The cross-cultural applicability of rating instruments developed in one language with only one cultural group is an important issue in both research and clinical settings where these instruments might be used. We exam...