This study developed and evaluated a brief training program for grant reviewers that aimed to increase inter-rater reliability, rating scale knowledge, and effort to read the grant review criteria. Enhancing reviewer training may improve the reliability and accuracy of research grant proposal scoring and funding recommendations. Seventy-five Public Health professors from U.S. research universit...

English language practitioners have long relied on intuition-based scales for rating EFL/ESL writing. As these scales lack an empirical basis, the scores they generate tend to be unreliable, which results in invalid interpretations. Given the significance of the genre of description and the fact that the relevant literature does not introduce any data-based analytic scales for rating EFL descri...

The purpose of the present study is to develop appropriate scoring scales for each of the defined stages of the writing process, and also to determine to what extent these scoring scales can reliably and validly assess the performances of EFL learners in an academic writing task. Two hundred and two students’ writing samples were collected after a step-by-step process oriented essay writing ins...

The difficulty of assessing the writing skill is well known. Different testing facets seem to affect the result of assessing the writing skill. In addition to the writer’s ability, the topic of the writing task, and methods of rating may contribute to the writer’s score. In this study, 50 EFL learners wrote four different types of writing tasks (convincing, describing, instructing, and explaini...

Background The clinical practice examination (CPX) was introduced in 2010, and the Seoul-Gyeonggi CPX Consortium developed the patient-physician interaction (PPI) assessment tool in 2004. Both institutions use rating scales on classified sections of PPI but differ in their scoring of key components. This study investigated the accuracy of standardized patient scores across rating scales by comp...

the difficulty of assessing the writing skill is well known. different testing facets seem to affect the result of assessing the writing skill. in addition to the writer’s ability, the topic of the writing task, and methods of rating may contribute to the writer’s score. in this study, 50 efl learners wrote four different types of writing tasks (convincing, describing, instructing, and explaini...

this study aimed at investigating the reliability, predictive validity, and self-esteem and gender bias of confidence-based scoring. this is a method of scoring in which the test takers receive a positive or negative point based on their rating of their confidence in an answer. the participants, who were 49 english-major students taking their grammar course, were given 8 multiple-choice tests d...