Abstract
Captions are key tools for allowing deaf and hard-of-hearing audiences to obtain aural (audio) information. In a classroom setting, people with normal hearing can listen to the lecture while referencing the white/blackboard and documents to effectively combine the information, whereas those with hearing impairments cannot. During lectures, people with hearing impairments need to move their eyes (gaze direction) more than those with normal hearing. The purpose of this study is to investigate the tendency and characteristics of gaze behavior of deaf and hard-of-hearing persons during captioned lectures. In the experiment, we analyzed the gaze behavior, in terms of the gaze count and duration, in relation to the following gaze targets (points of gaze): a lecturer with slides (L), handouts (H), and captions (C). Furthermore, we created hybrid captions, wherein parts of the slide were cut out and inserted between chunks of text to reduce eye movement (EM). The study compared the gaze behavior of subjects viewing the hybrid captions to that of subjects viewing normal text-only captions. The gaze behavior for both captions was essentially similar. The analysis of the findings showed that the gaze behavior was not cyclic; instead, it was roundtrip-type behavior centered around L, the source of the lecture content. Additionally, the study suggested that hybrid captions were seamlessly accepted and used in a similar manner as normal captions, even though they did not reduce fatigue caused by EMs.
This study was partially supported by JSPS KAKENHI Grant Numbers JP15K01056 and JP19K02996 and was carried out with the permission of the Research Ethics Committee of the Tsukuba University of Technology (approval number: H28–10).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Preminger, J.E., Levitt, J.E.: Computer-Assisted Remote Transcription (CART): a tool to aid people who are deaf or hard of hearing in the workplace. Volta Rev. 99(4), 219–230 (1997)
Elliot, L.B., Stinson, M.S., McKee, B.G., Everhart, V.S., Francis, P.J.: College students’ perceptions of the C-print speech-to-text transcription system. J. Deaf Stud. Deaf Educ. 6(4), 285–298 (2001)
Marschark, M., et al.: Benefits of sign language interpreting and text alternatives to classroom learning by deaf students. J. Deaf Stud. Deaf Educ. 11(4), 421–437 (2006)
Elliot, L.B., Stinson, M.S., Easton, D., Bourgeois, J.: College students learning with C-print’s education software and automatic speech recognition. In: American Educational Research Association Annual Meeting (2008)
Lasecki, W.S., et al.: Real-time captioning by groups of non-experts. In: Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, pp. 23–34 (2012)
Takeuchi, Y., Ohta, H., Ohnishi, N., Wakatsuki, D., Minagawa, H.: Extraction of displayed objects corresponding to demonstrative words for use in remote transcription. In: Miesenberger, K., Klaus, J., Zagler, W., Karshmer, A. (eds.) ICCHP 2010. LNCS, vol. 6180, pp. 152–159. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14100-3_24
Rathbun, K., Berke, L., Caulfield, C., Stinson, M., Huenerfauth, M.: Eye movements of deaf and hard of hearing viewers of automatic captions. In: 32nd Annual International Technology and Persons with Disabilities Conference Scientific/Research Proceedings, San Diego, California, pp. 130–140 (2017)
Behm, G.W., Kushalnagar, R.S., Stanislow, J.S., Kelstone, A.W.: Enhancing accessibility of engineering lectures for Deaf & Hard of Hearing (DHH): Real-Time Tracking Text Displays (RTTD) in classrooms. Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington, June 2015. https://doi.org/10.18260/p.23995
Wakatsuki, D., Kato, N., Shionome, T., Kawano, S., Nishioka, T., Naito, I.: Development of web-based remote speech-to-text interpretation system captiOnline. JACIII 21(2), 310–320 (2017). https://doi.org/10.20965/jaciii.2017.p0310
Acknowledgements
We would like to thank Editage (www.editage.com) for English language editing.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Wakatsuki, D., Arai, T., Shionome, T. (2020). Analysis of the Gaze Behavior of Deaf and Hard-of-Hearing Students During a Captioned Lecture. In: Miesenberger, K., Manduchi, R., Covarrubias Rodriguez, M., Peňáz, P. (eds) Computers Helping People with Special Needs. ICCHP 2020. Lecture Notes in Computer Science(), vol 12377. Springer, Cham. https://doi.org/10.1007/978-3-030-58805-2_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-58805-2_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58804-5
Online ISBN: 978-3-030-58805-2
eBook Packages: Computer ScienceComputer Science (R0)