Measuring the Usability of Paper Ballots: Efficiency, Effectiveness, and Satisfaction

The Help America Vote Act (HAVA) of 2002 secured funding for improvements to election administration. Improvements include upgrading older voting systems to meet new guidelines. To determine whether the new voting systems are improvements over existing voting systems, information is needed on the usability of the older, traditional systems. This study was designed as a first step in addressing the need for usability data on existing voting systems. Three traditional paper ballots were empirically evaluated to collect baseline data that can later be compared to newer, electronic voting systems. Usability was evaluated using the thee International Organization for Standardization (ISO) metrics suggested by the National Institute of Standards and Technology (NIST): effectiveness, efficiency, and satisfaction. All three ballot types (bubble, arrow, and open response) produced reasonable levels of efficiency. The three ballot types did not produce different levels of effectiveness, but the overall error rate was higher than would be expected. On satisfaction, voters were clearly more satisfied with their experience with the bubble ballot.