Ranking websites through prioritized web accessibility barriers

While accessibility can be tested with respect to guidelines (like WCAG 1.0, or Section 508) through a standards review method, other methods can be employed, like user testing (DRC, 2004) or usability inspection methods (Preece et al., 2002; Gray and Salzman, 1998; Nielsen, 1993) or those suggested by Henry and Grossnickle (2004). To be really useful, evaluation methods should constrain the way in which the evaluator identifies problems and how they are graded in terms of importance. Only when these two kinds of decisions can be standardized, then the results produced can be used to rank web sites and to prioritize their bugs. Prioritization of defects is of paramount importance as any web developer required to fix them works always in a scarce resource mode. This is not easy to achieve. As discussed by Brajnik (2006b) several definitions of accessibility exist: some refer to usability, some to effectiveness, and some to other abstract properties like perceivability, understandability and operability. The problem is that depending on the definition one chooses, different methods have to be used to investigate the website. Running a user test doesn’t make sense if we want to determine conformance to guidelines, and conversely conformance testing cannot be used to determine usability of the web site with respect to disabled users.