Jordan Aiko Deja
Alexczar Dela Torre
ABSTRACT
Visualizers are usually added into music players to augment the listening experiences of hearing users. However, for the case of most members of the Deaf and Hard of Hearing (DHH) community, they have partial deafness which may give them a "limited" listening experience as compared to their counterparts. In this paper, we present ViTune, a visualizer tool that enhances the musical experiences of the DHH through the use of an on-screen visualizer generating effects alongside music. We observed how members of the DHH community typically experienced music through an initial user study. We then iterated on developing a visualizer prototype where we did multiple usability tests involving at least 15 participants from the DHH community. We observed how they experienced music with the help of our prototype and noticed that certain music features and elements augmented these experiences. Visualization attributes, their matching qualitative descriptions and equivalent subjective attributes were identified with the help of music experts. Also, affects hypothesized to be induced and dissuaded were identified in improving these listening experiences.
- Alfonso Balandra, Hironori Mitake, and Shoichi Hasegawa. 2016. Haptic music player--synthetic audio-tactile stimuli generation based on the notes' pitch and instruments' envelope mapping. Proceedings of NIME 2016 (2016), 90--95.Google Scholar
- Jacob Cohen. 1988. Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates.Google Scholar
- Jordan Aiko Deja, Patrick Arceo, Darren Goldwin David, Patrick Lawrence Gan, and Ryan Christopher Roque. 2018. MyoSL: A Framework for Measuring Usability of Two-Arm Gestural Electromyography for Sign Language. In Universal Access in Human-Computer Interaction. Methods, Technologies, and Users, Margherita Antona and Constantine Stephanidis (Eds.). Springer International Publishing, Cham, 146--159.Google Scholar
- Brian Evans. 2005. Foundations of a Visual Music. Comput. Music J. 29, 4 (Dec. 2005), 11--24. DOI: http://dx.doi.org/10.1162/014892605775179955Google ScholarDigital Library
- Joyce Horn Fonteles, Maria Andréia Formico Rodrigues, and Victor Emanuel Dias Basso. 2013. Creating and Evaluating a Particle System for Music Visualization. J. Vis. Lang. Comput. 24, 6 (Dec. 2013), 472--482. DOI: http://dx.doi.org/10.1016/j.jvlc.2013.10.002Google ScholarDigital Library
- Kate Gfeller, Virginia Driscoll, Rachel See Smith, and Christina Scheperle. 2012. The Music Experiences and Attitudes of a First Cohort of Prelingually Deaf Adolescent and Young Adult Cochlear Implant Recipients. Seminars in Hearing 33, 4 (2012), 346--360.Google ScholarCross Ref
- Robert Jack, Andrew McPherson, and Tony Stockman. 2015. Designing tactile musical devices with and for deaf users: A case study. In Proceedings of the International Conference on the Multimodal Experience of Music. 23--25.Google Scholar
- Dhruv Jain, Leah Findlater, Jamie Gilkeson, Benjamin Holland, Ramani Duraiswami, Dmitry Zotkin, Christian Vogler, and Jon E. Froehlich. 2015. Head-Mounted Display Visualizations to Support Sound Awareness for the Deaf and Hard of Hearing. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 241--250. DOI: http://dx.doi.org/10.1145/2702123.2702393Google ScholarDigital Library
- Leigh. 2018. Audio Spectrum Explained. (Jun 2018). https://www.teachmeaudio.com/mixing/techniques/audio-spectrum/Google Scholar
- Brian McFee, Colin Raffel, Dawen Liang, Daniel Patrick Whittlesey Ellis, Matt McVicar, Eric Battenberg, and Oriol Nieto. 2015. librosa: Audio and Music Signal Analysis in Python.Google Scholar
- Suranga Nanayakkara, Elizabeth Taylor, Lonce Wyse, and S.H. Ong. 2007. Towards Building an Experiential Music Visualizer. Master's thesis. National University of Singapore, 14 Kent Ridge Road, Singapore 119223.Google Scholar
- Suranga Nanayakkara, Elizabeth Taylor, Lonce Wyse, and S H. Ong. 2009. An Enhanced Musical Experience for the Deaf: Design and Evaluation of a Music Display and a Haptic Chair. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, New York, NY, USA, 337--346. DOI: http://dx.doi.org/10.1145/1518701.1518756Google ScholarDigital Library
- Lauri H. Nelson, Whitney Wright, and Elizabeth W. Parker. 2016. Embedding Music Into Language and Literacy Instruction for Young Children Who Are Deaf or Hard of Hearing. Young Exceptional Children 19, 1 (2016), 27--38. DOI: http://dx.doi.org/10.1177/1096250614566539Google ScholarCross Ref
- Benjamin Petry, Thavishi Illandara, and Suranga Nanayakkara. 2016. MuSS-bits: Sensor-display Blocks for Deaf People to Explore Musical Sounds. In Proceedings of the 28th Australian Conference on Computer-Human Interaction (OzCHI '16). ACM, New York, NY, USA, 72--80. DOI: http://dx.doi.org/10.1145/3010915.3010939Google ScholarDigital Library
- Tatiane Ribiero Morais de Paula and Patricia Lima Martins Pederiva. 2017. Musical Experience in Deaf Culture. International Journal of Technology and Inclusive Education 6, 2 (2017), 1098--1107.Google Scholar
- James A. Russell. 1980. A circumplex model of affect. Journal of personality and social psychology 39, 6 (1980), 1161--1178.Google ScholarCross Ref
- Rob Walker. 2013. Child mental health and deafness. Paediatrics and Child Health 23, 10 (2013), 438--442.Google ScholarCross Ref
Index Terms
- ViTune: A Visualizer Tool to Allow the Deaf and Hard of Hearing to See Music With Their Eyes
Recommendations
Towards Inclusive Streaming: Building Multimodal Music Experiences for the Deaf and Hard of Hearing
CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing SystemsAs online streaming becomes a primary method for music consumption, the various modalities that many people with hearing loss rely on for their enjoyment need to be supported. While visual or tactile representations can be used to experience music in a ...
E-Learning Accessibility for the Deaf and Hard of Hearing - Practical Examples and Experiences
6th Symposium of the Workgroup Human-Computer Interaction and Usability Engineering on HCI in Work and Learning, Life and Leisure - Volume 6389Development of information and communication technology has offered new horizons to the deaf and hard of hearing for their integration into working, social and economic environment. Despite the positive attitude of international guidelines, the lack of ...
Accessibility for Deaf and Hard of Hearing Users: Sign Language Conversational User Interfaces
CUI '20: Proceedings of the 2nd Conference on Conversational User InterfacesWith the proliferation of voice-based conversational user interfaces (CUIs) comes accessibility barriers for Deaf and Hard of Hearing (DHH) users. There has not been significant prior research on sign-language conversational interactions with ...
Comments