Although the use of verbal protocols is growing in oral assessment, research on the use of raters’ verbal protocols is rather rare. Moreover, those few studies did not use a mixed-methods design. Therefore, this study investigated the possible impacts of rater training on novice and experienced raters’ application of a specified set of standards in rating. To meet this o...

The consideration of rater training is very important in construct validation of a writing test because it is through training that raters are adapted to the use of students’ writing ability instead of their own criteria for assessing compositions (Charney, 1984). However, although training has been discussed in the literature of writing assessment, there is little research regarding raters’ pe...

BACKGROUND The Cochrane risk of bias tool is commonly criticized for having a low reliability. We aimed to investigate whether training of raters, with objective and standardized instructions on how to assess risk of bias, can improve the reliability of the Cochrane risk of bias tool. METHODS In this pilot study, four raters inexperienced in risk of bias assessment were randomly allocated to ...

BACKGROUND In clinical practice, visual gait observation is often used to determine gait disorders and to evaluate treatment. Several reliability studies on observational gait analysis have been described in the literature and generally showed moderate reliability. However, patients with orthopedic disorders have received little attention. The objective of this study is to determine the reliabi...

Weaknesses in the nature of rater judgments are generally considered to compromise the utility of workplace-based assessment (WBA). In order to gain insight into the underpinnings of rater behaviours, we investigated how raters form impressions of and make judgments on trainee performance. Using theoretical frameworks of social cognition and person perception, we explored raters' implicit perfo...

This study investigated the inter-rater and intra-rater reliability of subjective judgements of cough in patients following inhalation of citric acid. Eleven speech-language pathologists (SLPs) currently using cough reflex testing in their clinical practice (experienced raters) and 34 SLPs with no experience using cough reflex testing (inexperienced raters) were recruited to the study. Particip...

Introduction. Cercospora leaf blight (CLB) caused by kikuchii is an endemic soybean crop disease in Argentina. Objective. To develop and validate a scale to quantify the severity of CLB leaflets (Glycine max) for evaluation foliar fungicides cultivar trials. Materials methods. A diagrammatic assess on was developed. The based 50 leaflets. different levels were determined according Weber-Fechner...

few, if any, studies have investigated the effect of professional experience as a rater variable and type of assessment as a task variable on raters’ criteria in the assessment of speech acts. this study aimed to explore the impact of nonnative teachers’ professional experience on the use of criteria in monologic and dialogic assessment of 12 role-plays of 3 apology speech acts. to this end, 60...