Evaluating accessibility-in-use

ABSTRACT

Evidence suggests that guidelines employed in conformance testing do not catch all the accessibility barriers encountered by users on the Web. Since accessibility is strongly tied to the users' experience there is a subjective perception of accessibility barriers and their severity. What is more, not only intangible qualities characterise the way in which these barriers are perceived, but also navigation styles, age, onset, expertise and abilities play a key role. In order to overcome the limitations of conformance testing and catch the problems that emerge during the interaction we propose a user-interaction-driven method to automatically report accessibility problems. To do so, we initially isolate the problematic situations faced by users and the tactics employed in such situations. These tactics are considered behavioural markers of cognitive processes that indicate problematic situations; the presence of tactics denotes the presence of problems. Then, we design and deploy algorithms to automatically detect the exhibition of these tactics and consequently detect problematic situations. WebTactics, a tool that unobtrusively detects and reports the problematic situations undergone by visually disabled users illustrates the method we propose.

References

1. P. Biswas and P. Robinson. Evaluating the design of inclusive interfaces by simulation. In Proceedings of the 15th international conference on Intelligent user interfaces, IUI '10, pages 277--280, 2010.
2. G. Brajnik. A comparative test of web accessibility evaluation methods. In Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility, Assets '08, pages 113--120, 2008.
3. M. Cooper, D. Sloan, B. Kelly, and S. Lewthwaite. A challenge to web accessibility metrics and guidelines: putting people and processes first. In Proceedings of the International Cross-Disciplinary Conference on Web Accessibility, W4A '12, pages 20:1--20:4, 2012.
4. J. Harty. Finding usability bugs with automated tests. Communications of the ACM, 54(2):44--49, 2011.
5. H. Petrie, F. Hamilton, N. King, and P. Pavan. Remote usability evaluations with disabled people. In Proceedings of the SIGCHI conference on Human Factors in computing systems, CHI '06, pages 1133--1141, 2006.
6. C. Power, A. Freire, H. Petrie, and D. Swallow. Guidelines are only half of the story: accessibility problems encountered by blind users on the web. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems, CHI '12, pages 433--442, 2012.
7. M. Schrepp. Goms analysis as a tool to investigate the usability of web units for disabled users. Universal Access the Information Society, 9(1):77--86, 2010.
8. K. Shinohara and J. Tenenberg. A blind person's interactions with technology. Communications of the ACM, 52(8):58--66, 2009.
9. F. Stefano, S. Borsci, and G. Stamerra. Web usability evaluation with screen reader users: implementation of the partial concurrent thinking aloud technique. Cognitive Processing, 11:263--272, 2010.
10. S. Trewin, B. E. John, J. Richards, C. Swart, J. Brezin, R. Bellamy, and J. Thomas. Towards a tool for keystroke level modeling of skilled screen reading. In Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility, Assets '10, pages 27--34, 2010.
11. M. Vigo and G. Brajnik. Automatic web accessibility metrics: Where we are and where we can go. Interacting with Computers, 23(2):137--155, 2011.
12. M. Vigo and S. Harper. Coping tactics employed by visually disabled users on the web (cope project, technical report), available at http://wel-eprints.cs.manchester.ac.uk/160/. In Transactions of the Web Ergonomics Lab, number 3 in COPE Project. University of Manchester, 2012.

Index Terms

• Applied computing

  • Document management and text processing

    • Document preparation

      • Multi / mixed media creation

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI design and evaluation methods

    • Interaction paradigms

      • Hypertext / hypermedia

• Software and its engineering

  • Software notations and tools

    • Development frameworks and environments

      • Integrated and visual development environments