OBJECTIVE To determine the length and position of a thyroidectomy scar that is cosmetically most appealing to naïve raters. METHODS Images of thyroidectomy scars were reproduced on male and female necks using digital imaging software. Surgical variables studied were scar position and length. Fifteen raters were presented with 56 scar pairings and asked to identify which was preferred cosmetic...

We analysed the peer review of grant proposals under Marie Curie Actions, a major EU research funding instrument, which involves two steps: an independent assessment (Individual Evaluation Report, IER) performed remotely by 3 raters, and a consensus opinion reached during a meeting by the same raters (Consensus Report, CR). For 24,897 proposals evaluated from 2007 to 2013, the association betwe...

This study sought to compare Iranian EFL novice and expert teachers regarding their professional knowledge at Iranian language institutes and universities. To achieve the aim of study, a questionnaire was developed based on the literature, the theoretical framework and the results of a qualitative study carried out by Yazdanpanah and Sahragard (2017). This questionnaire was administered to the ...

Feedback scores in an online marketplace have risen sharply over time, leading to substantial top-censoring. Some of the increase is explained by more satisfied raters, but at least 35-45% is attributable to raters applying lower standards. We show that this “reputation inflation” is the equilibrium outcome of a model in which (a) inferences made by future trading partners determine what consti...

The score reliability of language performance tests has attracted increasing interest. Classical Test Theory cannot examine multiple sources of measurement error. Generalizability theory extends Classical Test Theory to provide a practical framework to identify and estimate multiple factors contributing to the total variance of measurement. Generalizability theory by using analysis of variance ...

Despite the importance of interlanguage pragmatic rating (ILP) in the second language teaching and learning context, scant attention has been devoted to it. This study aims to investigate native EFL teachers’ major criteria in assessing the speech act of complaint produced by Iranian EFL learners. To fulfill this end, two groups of experienced native raters, including American (n=47) and Britis...

The use of centralized raters who are remotely linked to sites and interview patients via videoconferencing or teleconferencing has been suggested as a way to improve interrater reliability and interview quality. This study compared the effect of site-based and centralized ratings on patient selection and placebo response in subjects with major depressive disorder. Subjects in a 2-center placeb...