The application of Multifaceted Rasch Measurement (MFRM) in rating test takers’ oral language proficiency has been investigated in some previous studies (e.g., Winke, Gass, & Myford, 2012). However, little research so far has ever documented the effect of test takers’ genders on their oral performances and few studies have investigated the relationship between the impact of raters’ gender on th...

The Cognitive Science Research group at ETS conducts research and development at the forefront of educational assessment, using cognitive theory in the design of assessments, building cognitive models to guide interpretation of test-takers’ performance, and researching cognitive issues in the context of assessment. Moving beyond traditional (e.g., multiple-choice) tests, the group explores asse...

Differential item functioning (DIF) is when a test item favors or hinders a characteristic exhibited by group members of a test-taking population. DIF analyses are statistical procedures used to determine to what extent the content of an item affects the item endorsement of sub-groups of test-takers. If DIF is found for many items on the test, the final test scores do not represent the same mea...

In an attempt to assist proctors to prevent test takers from academic dishonesty in remotely administrated exams, this study investigated the ability of test takers’ behaviors during online assessments to predict their cheating decisions. Specifically, this experimental study focused on the role of students’ time delay and certainty rating during lab based online testing sessions. The analysis ...

In a randomized experiment with noncompliance, scientific interest is often in testing whether the treatment exposure X has an effect on the final outcome Y [2, 1]. We have proposed a finite-population significance test of the sharp null hypothesis that X has no effect on Y , within the principal stratum of compliers, using a generalized likelihood ratio test [4]. As both the null and alternati...

We develop a protocol for “distributed proctoring” which allows a network of graders to grade individual problems solved by a network of test takers. The mutual anonymity of the test takers and graders is ensured using anonymous MIXs, but an audit trail is provided in the event of a grading dispute. We discuss other applications for this protocol, more generally defined as “digital piecework.”

recent versions of international high-stakes tests like toefl and ielts have made use of integrated tasks in addition to the traditional independent tasks in a claim to provide a more realistic estimation of the test takers’ language abilities. the present study aimed to investigate how test takers’ performance may differ on such tasks. as such, the test takers’ performance was compared on ielt...

Dynamic assessment (DA) which is rooted in Vygotsky’s (1978) sociocultural theory involves the integration of instruction and assessment in a dialectical way to achieve two main purposes: enhancing learners' development and understanding about their learning potential. However, the feasibility and appropriateness of mediation are two main concerns of DA. The former is concerned with the applica...