Because of the unprecedented growth of information on the web it became difficult to decide what information is truly relevant. We present a method of personalized summarization based on a combination of different raters that process the additional information, such as user characteristics or document metadata to extract from the document information that is important for a particular user or i...

Learner autonomy has been described as the ultimate objective in many language teaching programs since the third quarter of the twentieth century and educators have highlighted the significant role of promoting learner autonomy in the process of language learning and teaching. However, only limited number of studies has been awarded to what leaner autonomy mean to teachers. This study addressed...

OBJECTIVES There is growing interest in multi-source, multi-level feedback for measuring the performance of health care professionals. However, data are often unbalanced (e.g. there are different numbers of raters for each doctor), uncrossed (e.g. raters rate the doctor on only one occasion) and fully nested (e.g. raters for a doctor are unique to that doctor). Estimating the true score varianc...

The SAS system V.8 implements the computation of unweighted and weighted kappa statistics as an option in the FREQ procedure. A major limitation of this implementation is that the kappa statistic can only be evaluated when the number of raters is limited to 2. Extensions to the case of multiple raters due to Fleiss (1971) have not been implemented in the SAS system. A SAS macro called MAGREE.SA...

BACKGROUND We sought to determine inter-rater reliability of the 2009 Appropriate Use Criteria for radionuclide imaging and whether physicians at various levels of training can effectively identify nuclear stress tests with inappropriate indications. METHODS AND RESULTS Four hundred patients were randomly selected from a consecutive cohort of patients undergoing nuclear stress testing at an a...

Factors associated with clinician assessment may play a role in the increasing rate of failed trials. The use of centralized raters, a small group of highly skilled, tightly calibrated, and continuously monitored raters linked to the study sites through videoconferencing can address these issues by reducing the sheer number of raters involved, using rigorous calibration procedures not logistica...

UNLABELLED To determine how early "the stuttering stereotype" is assigned, 160 university students rated a hypothetical vignette depicting either a 3-, 4-, 5-, or 6-year-old with or without the statement "He stutters". A factor analysis of the semantic differential scale showed a three-factor solution comprised of 17 of the 25 bi-polar adjective pairs. The factor labeled personality showed sign...

BACKGROUND To assess the effectiveness of geriatric interventions, The Older Persons and Informal Caregivers Survey - Composite Endpoint (TOPICS-CEP) has been developed based on health valuations of older persons and informal caregivers. This study explored the influence of the raters' age on the preference weights of TOPICS-CEP's components. METHODS A vignette study was conducted with 200 ra...