START Conference Manager    

Usage of Subjective Scales in Accessibility Research

Shari Trewin, Diogo Marques and Tiago Guerreiro

The 17th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2015)
Lisbon, Portugal, October 26-28, 2015


Accessibility research studies often gather subjective responses to technology using Likert-type items, where participants respond to a prompt statement by selecting a position on a labeled response scale. We analyzed recent ASSETS papers, and found that participants in non-anonymous accessibility research studies gave more positive average ratings than typical usability studies, especially when responding to questions about a proposed innovation. We further explored potential positive response bias in a study of two telephone information systems, one more usable than the other. Although we couldn’t establish that a group of participants with visual impairment consistently rated systems higher than a group of students, because of a ceiling effect, we did find that their subjective ratings weren’t as sensitive to usability problems, and did not correlate as well with objective measures of performance. A deeper understanding of the mechanism behind this effect would help researchers to design better accessibility studies, and to interpret subjective ratings with more accuracy.

START Conference Manager (V2.61.0 - Rev. 3862)