Usage of Subjective Scales in Accessibility Research

Shari Trewin, Diogo Marques and Tiago Guerreiro

The 17th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2015)
Lisbon, Portugal, October 26-28, 2015

---

Abstract

Accessibility research studies often gather subjective responses to technology using Likert-type items, where participants respond to a prompt statement by selecting a position on a labeled response scale. We analyzed recent ASSETS papers, and found that participants in non-anonymous accessibility research studies gave more positive average ratings than typical usability studies, especially when responding to questions about a proposed innovation. We further explored potential positive response bias in a study of two telephone information systems, one more usable than the other. Although we couldn’t establish that a group of participants with visual impairment consistently rated systems higher than a group of students, because of a ceiling effect, we did find that their subjective ratings weren’t as sensitive to usability problems, and did not correlate as well with objective measures of performance. A deeper understanding of the mechanism behind this effect would help researchers to design better accessibility studies, and to interpret subjective ratings with more accuracy.

---

START Conference Manager (V2.61.0 - Rev. 3862)