Creativity assessment commonly uses open-ended divergent thinking tasks. The typical methods for scoring these tasks (uniqueness scoring and subjective ratings) are time-intensive, however, so it is impractical for researchers to include divergent thinking as an ancillary construct. The present research evaluated snapshot scoring of divergent thinking tasks, in which the set of responses receiv...

OBJECTIVE This study examined the confounding effect of treatment emergent physical or psychic symptoms on clinical global impression (CGI) ratings in CNS trials and examined the benefit of targeted scoring criteria on clarifying ratings and reducing scoring variance. METHODS Twenty-four raters participating in an investigator meeting training session scored a series of scripted CGI scenarios...

This study aimed at investigating the reliability, predictive validity, and self-esteem and gender bias of confidence-based scoring. This is a method of scoring in which the test takers receive a positive or negative point based on their rating of their confidence in an answer. The participants, who were 49 English-major students taking their grammar course, were given 8 multiple-choice tests d...

This paper proposes an approach to automatically evaluate the prosody of Chinese Mandarin speech for language learning. In this approach, we grade the appropriateness of prosody of speech units according to a model speech corpus from a teacher’s voice. To this end, we build two models, which are the prosody model and the scoring model. The prosody model that is built from the teacher’s speech p...

Background This technical report summarizes findings from a series of studies to measure focus, one of the two clinical constructs in the capability-focus treatment framework described in Kwiatkowski and Shriberg (in submission). Rationale and conceptual background for the capability-focus treatment framework (henceforth, the C-F framework) is reviewed elsewhere In this report we provide (a) co...

Formality is one of the most important dimensions of writing style variation. In this study we conducted an inter-rater reliability experiment for assessing sentence formality on a five-point Likert scale, and obtained good agreement results as well as different rating distributions for different sentence categories. We also performed a difficulty analysis to identify the bottlenecks of our rat...