Helping Tools for Item Bank Calibration and Development of Computerized Adaptive Tests

Recently-developed testing theories and current technology make possible a fast and easy generation of computerized adaptive tests (CATs), which emulate the intelligent behaviour of human evaluators. In fact, they dynamically select and administer the most appropriate items depending on the previous responses given by the examinees. However, to choose the proper item from the bank, the CAT algorithm needs to know the values of some psychometric characteristics (called parameters) that feature the items. This means that the item bank must be calibrated according to some psychometric model. When the inexperienced user wants to design this kind of assessments, which are more efficient than traditional paper and pencil trials, three main problems arise: (1) the comprehension of statistical and psychometrical concepts, (2) the understanding of computers at a technical level, and (3) the need of many people, such as experts in the area and individuals to form the statistical samples needed during the calibration process. This paper presents a tandem of help tools, called CALLIE/GenTAI, which reduce the amount of knowledge required to effectively generate CATs. CALLIE guides the user through the different tasks of the item bank calibration process, helping her or him whenever making a decision is necessary, supporting the management of samples of individuals, and hiding, as much as possible, psychometrical concepts and statistical calculi at the same time. Once the item bank is calibrated, the user can employ GenTAI to administer dynamically generated CATs.