We introduce Item Response Theory (IRT) from psychometrics as an alternative to majority voting to create an IRT gold standard (GSIRT ). IRT describes characteristics of individual items in GSIRT their difficulty and discriminating power and is able to account for these characteristics in its estimation of human intelligence or ability for an NLP task. In this paper, we evaluated IRT’s model-fi...

The growing emphasis on patient-centered care has accelerated the demand for high-quality data from patient-reported outcome (PRO) measures. Traditionally, the development and validation of these measures has been guided by classical test theory. However, item response theory (IRT), an alternate measurement framework, offers promise for addressing practical measurement problems found in health-...

This study compares the psychometric utility of Classical Test Theory (CTT) and Item Response Theory (IRT) for scale construction with data from higher education student surveys. Using 2008 Your First College Year (YFCY) survey data from the Cooperative Institutional Research Program at the Higher Education Research Institute at UCLA, two scales are built and tested—one measuring social involve...

The maintenance of a good surgical view is mandatory in video-assisted thoracoscopic surgery (VATS). For routine segmentectomy, it is necessary to re-inflate the lung in order to identify the intersegmental borders. However, such re-inflation can occasionally obstruct the surgical view and can lead to prolongation of operation time, particularly in the context of VATS. Infrared thoracoscopy( IR...

Item response theory (IRT) has become a popular methodological framework for modeling response data from assessments in education and health; however, its use is not widespread among psychologists. This paper aims to provide a didactic application of IRT and to highlight some of these advantages for psychological test development. IRT was applied to two scales (a positive and a negative affect ...

Along with the increasing popularity of item response theory (IRT) in testing practices, it is important to check a fundamental assumption of most of the popular IRT models, which is unidimensionality. Nevertheless, it is hard for educational and psychological tests to be strictly unidimensional. The tests studied in this paper are from a standardized high-stake testing program. They feature po...

The Oligochaeta forms an important part of the macroinvertebrates inhabiting sediments of lotic ecosystems. It has an important role in the cycling of matter and energy transfer in these environments. The aim of this study is to analyse limnological variables, their influence on the structure and diversity of benthic oligochaete taxocenosis in a stretch of the Upper São Francisco River and its ...

When using functional brain imaging to study neuropsychiatric patients an important challenge is determining whether the imaging task assesses individual differences with equal precision in healthy control and impaired patient groups. Classical test theory (CTT) requires separate reliability studies of patients and controls to determine equivalent measurement precision with additional studies t...

Evaluation of NLP methods requires testing against a previously vetted gold-standard test set and reporting standard metrics (accuracy/precision/recall/F1). The current assumption is that all items in a given test set are equal with regards to difficulty and discriminating power. We propose Item Response Theory (IRT) from psychometrics as an alternative means for gold-standard test-set generati...