Evaluating an application of the CAREGen MCQ Creation Methodology

In 2011 senior managers at a UK electricity distribution company faced a training and assessment challenge that would have been very difficult to address using conventional methods. This paper describes the challenge they faced and the reasons for using the Construed Antonym Realization Exercise (CARE) generation methodology to create Multiple Alternative Choice (MAC) test items as part of the solution. An innovative design of an evaluation exercise is then described which includes some novel methods for calculating the efficacy of a MCQ generation methodology. The success of this intervention provides further evidence to support planned developments of the methodology and continued use of the proposed measures of efficacy.