START Conference Manager    

Tracked Speech-To-Text Display: Enhancing Accessibility and Readability of Real-Time Speech-To-Text

Raja Kushalnagar, Gary Behm, Aaron Kelstone and Shareef Ali

The 17th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2015)
Lisbon, Portugal, October 26-28, 2015


Deaf and Hard of Hearing students are under-served and under-represented in education in part because they miss spoken lecture information, even with aural-to-visual accommodations. Some deaf and hard of hearing students use a real-time Speech-To-Text Display (SD), in which a trained typist transcribes the lecture speech into text (speech-text) onto a display. Still, these students encounter significant but subtle barriers in following speech-text displays, especially when detailed visuals are used or when the speaker is fast or uses infrequent words. Hearing students can simultaneously watch the visuals and listen to the spoken explanation. On the other hand, DHH students have to constantly look away from the SD to search and observe details in the lecture visuals. As a result, they spend less time watching lecture visuals and gain less information than their hearing peers. They can also fall behind in reading the speech-text.

We discuss the implementation and evaluation of a real-time Tracked SpeechText Display (TSD), that addresses these subtle barriers presented by SD systems. The TSD system minimizes the student's viewing distance between the speech-text and the speaker. This is done through tracking the presenter and displaying the speech-text at a fixed distance above the presenter. Our evaluation showed that students significantly preferred TSD over SD and reported that it was easier to follow the lecture. They liked being able to see both the teacher and speech-to-text, and being able to set the number of displayed lines.

START Conference Manager (V2.61.0 - Rev. 3862)