The same L2 speaking performance may be analysed and evaluated in very different ways by teachers or raters. We present a new, technology-assisted research design which opens up to investigation the trajectories of convergence divergence between tracked recorded what raters noticed when, whilst grading speaker’s videoed talk IELTS Speaking Test. devised graphs ‘noticing trajectories’ compare re...

BACKGROUND AND PURPOSE National Institutes of Health Stroke Scale certification is required for participation in modern stroke clinical trials and as part of good clinical care in stroke centers. A new training and demonstration DVD was produced to replace existing training and certification videotapes. Previously, this DVD, with 18 patients representing all possible scores on 15 scale items, w...

National income has a pervasive influence on the perception of ingroup stereotypes, with high status and wealthy targets perceived as more competent. In two studies we investigated the degree to which economic wealth of raters related to perceptions of outgroup competence. Raters' economic wealth predicted trait ratings when 1) raters in 48 other cultures rated Americans' competence and 2) Mexi...

Several studies have made positive claims regarding the validity, reliability, and utility of the Occupational Information Network (O*NET). In this first of three studies questioning such claims, I focused on the root cause of many problems regarding O*NET: i.e., the practice of rating overly abstract and heterogeneous occupational units (OUs), collecting ratings on OUs that exhibit substantial...

Study 1 expands upon previous research by looking at the ability of untrained raters to detect pathological traits within a normal population of college students. In Study 1, 30-s video clips of 81 target persons were shown to 42 raters. Ratings of traits of personality disorders made by thin slice raters reliably predicted scores on the personality pathology measures obtained from the targets ...

OBJECTIVE To determine the length and position of a thyroidectomy scar that is cosmetically most appealing to naïve raters. METHODS Images of thyroidectomy scars were reproduced on male and female necks using digital imaging software. Surgical variables studied were scar position and length. Fifteen raters were presented with 56 scar pairings and asked to identify which was preferred cosmetic...

We analysed the peer review of grant proposals under Marie Curie Actions, a major EU research funding instrument, which involves two steps: an independent assessment (Individual Evaluation Report, IER) performed remotely by 3 raters, and a consensus opinion reached during a meeting by the same raters (Consensus Report, CR). For 24,897 proposals evaluated from 2007 to 2013, the association betwe...

Feedback scores in an online marketplace have risen sharply over time, leading to substantial top-censoring. Some of the increase is explained by more satisfied raters, but at least 35-45% is attributable to raters applying lower standards. We show that this “reputation inflation” is the equilibrium outcome of a model in which (a) inferences made by future trading partners determine what consti...

INTRODUCTION 'Decompression stress' is commonly evaluated by scoring circulating bubble numbers post dive using Doppler or cardiac echography. This information may be used to develop safer decompression algorithms, assuming that the lower the numbers of venous gas emboli (VGE) observed post dive, the lower the statistical risk of decompression sickness (DCS). Current echocardiographic evaluatio...

BACKGROUND Successful hip reconstruction to restore the normal hip biomechanics requires precise placement of implants. Computer assisted navigation in total hip arthroplasty has been proposed to have the potential to help achieve a high accuracy in implant placement. The goal of the study was to evaluate the accuracy of an imageless computer navigation system on cadavers and to validate a non-...