Zihan Gao
Skip Abstract Section
Abstract
Spatial auditory cues are important for many tasks in immersive virtual environments, especially guidance tasks. However, due to the limited fidelity of spatial sounds rendered by generic Head-Related Transfer Functions (HRTFs), sound localization usually has a limited accuracy, especially in elevation, which can potentially impact the effectiveness of auditory guidance. To address this issue, we explored whether integrating sonification with spatial audio can enhance the perceptions of auditory guidance cues so user performance in auditory guidance tasks can be improved. Specifically, we investigated the effects of sonification mapping strategy using a controlled experiment that compared four elevation sonification mapping strategies: absolute elevation mapping, unsigned relative elevation mapping, signed relative elevation mapping, and binary relative elevation mapping. In addition, we examined whether azimuth sonification mapping can further benefit the perception of spatial sounds. The results demonstrate that spatial auditory cues can be effectively enhanced by integrating elevation and azimuth sonification, where the accuracy and speed of guidance tasks can be significantly improved. In particular, the overall results suggest that binary relative elevation mapping is generally the most effective strategy among four elevation sonification mapping strategies, which indicates that auditory cues with clear directional information are key to efficient auditory guidance.
- [1] . 2019. Sonification of rotation instructions to support navigation of people with visual impairment. In IEEE International Conference on Pervasive Computing and Communications (PerCom). IEEE, 1–10.
DOI: Google Scholar
Cross Ref
- [2] . 2019. Designing an interactive and collaborative experience in audio augmented reality. In International Conference on Virtual Reality and Augmented Reality. Springer, 305–311.Google ScholarDigital Library
- [3] . 1991. Challenges to the successful implementation of 3-D sound. J. Audio Eng. Societ. 39, 11 (1991), 864–870.Google Scholar
- [4] . 2000. 3-D Sound for Virtual Reality and Multimedia. Academic Press, San Diego, CA.Google ScholarDigital Library
- [5] . 1992. Techniques and applications for binaural sound manipulation. Int. J. Aviat. Psychol. 2, 1 (1992), 1–22.
DOI: Google ScholarCross Ref - [6] . 1993. Headphone localization of speech. Hum. Fact. 35, 2 (1993), 361–376.
DOI: Google ScholarCross Ref - [7] . 2018. Generic HRTFs may be good enough in virtual reality. Improving source localization through cross-modal plasticity. Front. Neurosci. 12 (2018), 21.
DOI: Google ScholarCross Ref - [8] . 1994. HRTF measurements of a KEMAR dummy-head microphone. MIT Media Lab. Percept. Comput.-Technic. Rep. 280 (1994), 1–7.Google Scholar
- [9] . 2017. Non-normal data: Is ANOVA still a valid option? Psicothema 29, 4 (2017), 552–557.
DOI: Google ScholarCross Ref - [10] . 2005. Presence and the utility of audio spatialization. Pres.: Teleop. Virt. Environ. 14, 3 (2005), 278–297.
DOI: Google ScholarDigital Library - [11] . 2003. Design guidelines for audio presentation of graphs and tables. In International Conference on Auditory Display.Google Scholar
- [12] . 1999. Can audio enhance visual perception and performance in a virtual environment? In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 43. Sage Publications, Los Angeles, CA, 1197–1201.
DOI: Google ScholarCross Ref - [13] . 2017. Designing a vibrotactile head-mounted display for spatial awareness in 3D spaces. IEEE Trans. Visualiz. Comput. Graph. 23, 4 (2017), 1409–1417.
DOI: Google ScholarDigital Library - [14] . 2007. Evaluation of a low-cost 3D sound system for immersive virtual reality training systems. IEEE Trans. Visualiz. Comput. Graph. 13, 2 (2007), 204–212.
DOI: Google ScholarDigital Library - [15] . 2013. A systematic review of mapping strategies for the sonification of physical quantities. PloS One 8, 12 (2013), e82491.
DOI: Google ScholarCross Ref - [16] . 2014. Headlock: A wearable navigation aid that helps blind cane users traverse large open spaces. In 16th International ACM SIGACCESS Conference on Computers & Accessibility. 19–26.
DOI: Google ScholarDigital Library - [17] . 2005. Comparison of auditory, visual, and audiovisual navigation in a 3D space. ACM Trans. Appl. Percept. 2, 4 (2005), 564–570.Google ScholarDigital Library
- [18] . 2004. Using 3D sound as a navigational aid in virtual environments. Behav. Inf. Technol. 23, 6 (2004), 435–446.
DOI: Google ScholarCross Ref - [19] . 2006. NASA-task load index (NASA-TLX): 20 years later. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 50. Sage Publications, Los Angeles, CA, 904–908.
DOI: Google ScholarCross Ref - [20] . 1974. Spectral cues used in the localization of sound sources on the median plane. J. Acoust. Societ. Amer. 56, 6 (1974), 1829–1834.
DOI: Google ScholarCross Ref - [21] . 2016. Where are we? Evaluating the current rendering fidelity of mobile audio augmented reality systems. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services. 278–282.Google ScholarDigital Library
- [22] . 2002. AudioGPS: Spatial audio navigation with a minimal attention interface. Person. Ubiq. Comput. 6, 4 (2002), 253–259.
DOI: Google ScholarDigital Library - [23] . 2020. The MIDI Manual: A Practical Guide to MIDI within Modern Music Production. Routledge.Google ScholarCross Ref
- [24] . 2017. Interactions between auditory elevation, auditory pitch and visual elevation during multisensory perception. Multisens. Res. 30, 3–5 (2017), 287–306.
DOI: Google ScholarCross Ref - [25] . 2012. NAVIG: Augmented reality guidance system for the visually impaired. Virt. Real. 16, 4 (2012), 253–269.
DOI: Google ScholarDigital Library - [26] . 2022. Partial visual loss disrupts the relationship between judged room size and sound source distance. Experim. Brain Res. (2022), 81–96.Google ScholarCross Ref
- [27] . 2020. The accuracy of auditory spatial judgments in the visually impaired is dependent on sound source distance. Sci. Rep. 10, 1 (2020), 1–9.Google ScholarCross Ref
- [28] . 2005. Navigation with auditory cues in a virtual environment. IEEE MultiMed. 12, 2 (2005), 80–86.
DOI: Google ScholarDigital Library - [29] . 1998. Navigation system for the blind: Auditory display modes and guidance. Presence 7, 2 (1998), 193–203.
DOI: Google ScholarDigital Library - [30] . 2019. Non-visual cues for view management in narrow field of view augmented reality displays. In IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 190–201.
DOI: Google ScholarCross Ref - [31] . 2020. Comparing non-visual and visual guidance methods for narrow field of view augmented reality displays. IEEE Trans. Visualiz. Comput. Graph. 26, 12 (2020), 3389–3401.
DOI: Google ScholarCross Ref - [32] . 2007. Nonvisual route following with guidance from a simple haptic or auditory display. J. Visual Impair. Blind. 101, 4 (2007), 203–211.
DOI: Google ScholarCross Ref - [33] . 2016. Sonification of guidance data during road crossing for people with visual impairments or blindness. Int. J. Hum.-comput. Stud. 85 (2016), 16–26.
DOI: Google ScholarDigital Library - [34] . 2003. Pedestrian navigation aids: Information requirements and design implications. Person. Ubiq. Comput. 7, 6 (2003), 331–338.
DOI: Google ScholarDigital Library - [35] . 1996. Binaural technique: Do we need individual recordings? J. Audio Eng. Societ. 44, 6 (1996), 451–469.Google Scholar
- [36] . 2011. Perception, cognition and action in auditory displays. In The Sonification Handbook, , , and (Eds.). Logos Verlag Berlin, 63–85.Google Scholar
- [37] . 2014. Natural auditory scene statistics shapes human spatial hearing. Proc. Nat. Acad. Sci. 111, 16 (2014), 6104–6108.
DOI: Google ScholarCross Ref - [38] . 2017. Sonification strategies for dynamic guidance tasks: Example with a driving game. In Proceedings of the 13th International Symposium on Computer Music Multidisciplinary Research (CMMR). 283–294.Google Scholar
- [39] . 2016. Comparison and evaluation of sonification strategies for guidance tasks. IEEE Trans. Multim. 18, 4 (2016), 674–686.
DOI: Google ScholarDigital Library - [40] . 2020. Do we need two ears to perceive the distance of a virtual frontal sound source? J. Acoust. Societ. Amer. 148, 3 (2020), 1614–1623.Google ScholarCross Ref
- [41] . 2010. Is it really robust? Methodology 6, 4 (2010), 147–151.
DOI: Google ScholarCross Ref - [42] . 2014. Sonification as a possible stroke rehabilitation strategy. Front. Neurosci. 8 (2014), 332.Google ScholarCross Ref
- [43] . 2002. Virtual auditory displays. In Handbook of Virtual Environments. CRC Press, 105–132.
DOI: Google ScholarCross Ref - [44] . 2018. Current use and future perspectives of spatial audio technologies in electronic travel aids. Wirel. Commun. Mob. Comput. (2018).
DOI: Google ScholarDigital Library - [45] . 2005. Mappings and metaphors in auditory displays: An experimental assessment. ACM Trans. Appl. Percept. 2, 4 (2005), 407–412.Google ScholarDigital Library
- [46] . 2006. Navigation performance with a virtual auditory display: Effects of beacon sound, capture radius, and practice. Hum. Fact. 48, 2 (2006), 265–278.
DOI: Google ScholarCross Ref - [47] . 2018. Newborns are sensitive to the correspondence between auditory pitch and visuospatial elevation. Devel. Psychobiol. 60, 2 (2018), 216–223.
DOI: Google ScholarCross Ref - [48] . 1940. The role of head movements and vestibular and visual cues in sound localization. J. Experim. Psychol. 27, 4 (1940), 339. Google ScholarCross Ref
- [49] . 1992. Localization in virtual acoustic displays. Pres.: Teleop. Virt. Environ. 1, 1 (1992), 80–107.
DOI: Google ScholarCross Ref - [50] . 1993. Localization using nonindividualized head-related transfer functions. J. Acoust. Societ. Amer. 94, 1 (1993), 111–123.
DOI: Google ScholarCross Ref - [51] . 2013. Head-related Transfer Function and Virtual Auditory Display. J. Ross Publishing.
DOI: Google ScholarCross Ref - [52] . 2009. Perceptually relevant parameters for virtual listening simulation of small room acoustics. J. Acoust. Societ. Amer. 126, 2 (2009), 776–791.Google ScholarCross Ref
- [53] . 2018. A psychoacoustic auditory display for navigation. In International Conference on Auditory Display (ICAD).Google ScholarCross Ref
- [54] . 2019. Psychoacoustical signal processing for three-dimensional sonification. In International Conference on Auditory Display (ICAD).Google ScholarCross Ref
- [55] . 2004. Rendering localized spatial audio in a virtual auditory space. IEEE Trans. Multim. 6, 4 (2004), 553–564.Google ScholarDigital Library
Index Terms
- Exploring Sonification Mapping Strategies for Spatial Auditory Guidance in Immersive Virtual Environments
Recommendations
Combining Spatial and Non-Spatial Sonification for Sound Localization in Virtual Environment: a Subjective Study
CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing SystemsSpatially sonified audio is frequently used in virtual environments to serve as an object localization cue. However, spatializers that are available to the general public typically have their limitations and lead to sound localization errors. By ...
Lyricon Lyrics + Earcons Improves Identification of Auditory Cues
Proceedings, Part II, of the 4th International Conference on Design, User Experience, and Usability: Users and Interactions - Volume 9187Auditory researchers have developed various non-speech cues in designing auditory user interfaces. A preliminary study of "lyricons" lyricsï ź+ï źearcons [1] has provided a novel approach to devising auditory cues in electronic products, by combining ...
Immersive auditory display system 'sound cask': three-dimensional sound field reproduction system based on the boundary surface control principle
VRST '18: Proceedings of the 24th ACM Symposium on Virtual Reality Software and TechnologySound cask was developed to realize the perfect 3D auditory display that creates 3D sound waves around the listener's head just the same as the primary sound field, based on the boundary surface control (BoSC) principle.
If we consider the sound ...
Comments