Empirical Scoring System: A Cross-cultural Replication and Extension Study of Manual Scoring and Decision Policies

A cohort of 19 international polygraph examiner trainees at the Texas Department of Public Safety Polygraph School used the Empirical Scoring System (Blalock, Cushman & Nelson, 2009; Krapohl, 2010; Nelson, Krapohl & Handler, 2008) to evaluate 100 confirmed event-specific criminal investigation polygraph examinations. Bootstrap analytic procedures were used to calculate accuracy profiles and statistical confidence intervals for test results comparing decision rules, including; the Grand Total Rule, Two-Stage Rules (Senter, 2003; Senter & Dollins, 2002 & 2004), Spot Scoring Rules, and traditional ZCT decision rules (Department of Defense Polygraph Institute, 2006). Bootstrap analysis of the distribution of trainee scores with the Empirical Scoring System resulted in a mean accuracy rate of 90.1% (95% CI = 83.8% to 95.8%), excluding 3.3% inconclusives (95% CI = 1.0% to 7.0%). A second bootstrap analysis of decision agreement showed that these inexperienced examiners demonstrated an average rate of agreement of 85% (95% CI = 65 – 97%). Evaluation of the distribution of sub-total scores revealed that 61% (95% CI = 51% to 70%), of the sub-total scores of truthful cases produced a non-positive score (a zero or negative value). Results from this study are consistent with those from previous studies (Blalock, Cushman & Nelson, 2009; Krapohl, 2010; Nelson, Krapohl & Handler, 2008), and provide further support for the validity of the principles inherent to the ESS, including the bigger-is-better rule, three position scoring, electrodermal weighting, two-stage decision rules, and the use of optimal cut-scores. The authors recommend continued interest in and additional research on the ESS as an expedient, valid and reliable method for manually scoring PDD examination data using statistical decision theory.