Estimating and monitoring the construct-irrelevant variance (CIV) is of significant importance to validity, especially for constructed response assessments with rich contextualized information. To examine CIV in assessments, we developed a framework including model accounting measurement that could differentiate CIV. Specifically, includes due three factors: variability assessment item scenario...

Task-based listening tests such as IELTS require testees to listen to some information on a CD and simultaneously answer the related items. To answer such items, testees are expected to comprehend, analyze, compare and infer pieces of information while listening to the incoming audio material. The present research attempted to investigate whether the two major characteristics of question type a...

The many works devoted to the issue of fairness in language testing (e.g., Kunnan, 1999, 2000; Shohamy, 2001; Spolsky, 1981) testify to the field’s recognition of the importance of this issue. Brown (1996) defines fairness as “the degree to which a test treats every student the same or the degree to which it is impartial” (p. 31). The goal of language tests is to impartially measure individual ...

Utilities are numeric measurements that reflect an individual's beliefs about the desirableness of a health condition, willingness to take risks to gain health benefits, and preferences for time. This report discusses the approaches to assess and compare the validity of methods used to assign utilities for cost-utility analysis. Threats to validity include construct underrepresentation and cons...

Formula scoring (FS) is the use of a don't know option (DKO) with subtraction of points for wrong answers. Its effect on construct validity and reliability of progress test scores, is subject of discussion. Choosing a DKO may not only be affected by knowledge level, but also by risk taking tendency, and may thus introduce construct-irrelevant variance into the knowledge measurement. On the othe...

Non-verbal communication (NVC) plays a major role in various aspects of human life (Andersen, 2004; Cameron, 2001; Johnstone, 2008). Children learning their first language come to realize non-verbal communication as their socialization process takes place (Fletcher & German, 1990; Ingram, 1996; Owens, 2001). However, most EFL learners may have little exposure to these non-verbal aspects of comm...

The linguistic complexity of many text-based tests can be a source construct-irrelevant variance, as test-takers’ performance may affected by factors that are beyond the focus assessment itself, such reading comprehension skills. This experimental study examined extent to which use animated videos, opposed written text, could (i) reduce variance attributed language and skills (ii) impact reacti...