Background The validity of inferences based on (average) test scores obtained from large-scale assessments depends heavily on test-takers’ engagement when taking the test, that is, the degree to which they were motivated to show what they actually know and can do, in other words, to deliver their maximum performance (Cronbach 1970). However, in Abstract Background: A potential problem of low-st...

We investigate whether questions generated automatically by two Natural Language Processing (NLP) based systems (one developed by the authors, the other a state-of-the-art system) can successfully be used to assist multimedia-based learning. We examine the feasibility of using a Question Generation (QG) system’s output as pre-questions; with different types of pre-questions used: text-based and...

The study aims to discover whether audio or video modality in a listening test is more beneficial to test takers. In this study, the posttest-only control group design was utilized and quantitative data were collected in order to measure participant performances concerning two types of modality (audio or video) in a listening test. The participants, first grade students from an ELT program, wer...

To date, the role of gender in speaking tests has received limited attention in language testing research. It is possible in oral interviews, for instance, that both interviewing and rating may be highly gendered processes. In tests like the IELTS interview, where the interviewer also acts as the rater, this poses the question of whether a gender effect, if it exists, stems from the interview i...

The response process of problem-solving items contains rich information about respondents' behaviours and cognitive in the digital tasks, while extraction is a big challenge. aim study to use data-driven approach explore latent states state transitions underlying reflect test-takers' behavioural patterns, investigate how these could be associated with performance. We employed Hidden Markov Mode...

This study investigates the impact of different types of electronic warnings during a simulated job application assessment. Results indicated that negatively worded warnings and accusations worked better than positively worded warnings at blunting faking. Although there was some evidence that test-takers engaging in higher levels of faking heeded warnings more, warnings tended to decrease score...

Researchers suggest that claims about the meaningfulness of test score interpretations and consequences use should be backed by evidence stakeholders understand definition construct assessed (meaningfulness) reports (consequences). Evaluation stakeholders' actual uses in large-scale standardized language proficiency tests, however, remains limited reporting literature. This study investigates h...