Abstract
Presence, the “perceptual illusion of non-mediation,” is often a central goal in mediated and mixed environments, and sound is believed to be crucial for inducing high-presence experiences. This chapter provides a review of the state of the art within presence research related to auditory environments. Various sound parameters such as externalization and spaciousness and consistency within and across modalities are discussed in relation to their presence-inducing effects. Moreover, these parameters are related to the use of audio in mixed realities and example applications are discussed. Finally, we give an account of the technological possibilities and challenges within the area of presence-inducing sound rendering and presentation for mixed realities and outline future research aims.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Loomis, J.M., Klatzky, R.L., Golledge, R.G.: Auditory distance perception in real, virtual and mixed environments. In Y. Othta & H. Tamura (Eds.), Mixed reality: Merging real and virtual worlds. Ohmsha, Tokyo (1999)
Kyota, H., Takanobu, H., Asako, K., Fumihisa, S., Hideyuki, T.: Mixed reality system using audio and visual senses: Implementation of simultaneous presentation in both audio and visual MR. Proceedings of the Virtual Reality Society of Japan Annual Conference. 11, 2A2–3 (2006)
Milgram, P., Kishino, F.: A taxonomy of mixed reality visual displays. IEICE Transactions on Information Systems, E77–D(2), 1321–1329 (1994)
Pörschmann, C.: One’s own voice in auditory virtual environments. Acta Acustica, 87, 378–388 (2001)
Nordahl, R.: Self-induced footsteps sounds in virtual reality: Latency, recognition, quality and presence. Proceedings of the Eight Annual International Workshop Presence, London, UK (2004)
Lombard, M., Ditton, T.B.: At the heart of it all: the concept of presence. Journal of Computer-Mediated Communication, 13(3) (1997)
Gilkey, R.H., Weisenberger, J.M.: The sense of presence for the suddenly deafened adult: implications for virtual environments. Presence: Teleoperators and Virtual Environments, 4(4), 357–363 (1995)
Ijsselsteijn, W.A.: Presence in depth. Doctoral dissertation, Eindhoven University of Technology, Eindhoven, The Netherlands (2004)
Slater, M: A note on presence terminology. Presence-Connect, 3(3), 1–5 (2003)
Pope, J., Chalmers, A.: Multi-sensory rendering: combining graphics and acoustics. Proceedings of the 7th International Conference in Central Europe on Computer Graphics, Czech Republic, 233–242 (1999)
Guttman, S.E., Gilroy, L.A., Blake, R.: Hearing what the eyes see: auditory encoding of visual temporal sequences. Psychological Science, 16(3), 228–235 (2005)
Kleiner, M., Dalenbäck, B-I., Svensson, P.: Auralization: an overview. Journal of the Audio Engineering Society, 41(11), 861–875 (1993)
Begault, D.R.: 3D sound for virtual reality and multimedia. Academic Press Professional, London (1994)
Rumsey, F.: Spatial audio. Focal Press Oxford; Boston (2001)
Shilling, R.D., Shinn-Cunningham, B.G.: Virtual auditory displays. In K.M. Stanney (Ed.), Handbook of virtual environments: Design, implementation, and applications (pp. 65–92). Lawrence Erlbaum Associates, London, Mahwah, NJ (2002)
Hughes, C.E., Stapleton, C.B., Hughes, D.E., Smith, E.M.: Mixed reality in education, entertainment, and training. IEEE Computer Graphics and Applications, 25(6), 24–30 (2005)
Jarlengrip, J.: Object detection system and method. International Patent no. WO/2007/049995. (2007)
Zimmermann, R., Kyriakakis, C., Shahabi, C., Papadopolous, C., Sawchuck, A.A., Neumann, U.: The remote media immersion system. IEEE MultiMedia, 11(2), 48–57 (2004)
Pulkki, V.: Virtual sound source positioning using vector base amplitude panning. Journal of the Audio Engineering Society, 45(6), 456–466 (1997)
Gerzon, M.A.: Ambisonics in multichannel broadcasting and video. Journal of the Audio Engineering Society, 33(11), 859–871 (1985)
Berkhout, A.J.: A holographic approach to acoustic control. Journal of the Audio Engineering Society, 36(12), 977–995 (1988)
Horbach, U., Corteel, E., Pellegrini, R.S., Hulsebos, E.: Real-time rendering of dynamic scenes using wave field synthesis. Proceedings of the IEEE International Conference on Multimedia and Expo (ICME ’02), Lausanne, Switzerland, 1, 517–520. (2002)
Horbach, U., Boone, M.M.: Future transmission and rendering formats for multichannel sound. Proceedings of the AES 16th Conference on Spatial Sound Reproduction, Rovaniemi, Finland (1999)
Murray, C.D., Arnold, P., Thornton, B.: Presence accompanying induced hearing loss: implications for immersive virtual environments. Presence: Teleoperators and Virtual Environments, 9(2), 137–148 (2000)
Blauert, J.: Spatial hearing, revised edition. The MIT Press, Cambridge, MA (1997)
Wenzel, E.M., Arruda, M., Kistler, D.J., Wightman, F.L.: Localization using nonindividualized head-related transfer functions. Journal of the Acoustical Society of America, 94(1), 111–123 (1993)
Zotkin, D.N., Duraiswami, R., Davis, L.S.: Rendering localized spatial audio in a virtual auditory space. IEEE Transactions on Multimedia, 6(4), 553–564 (2004)
Hofman, P.M., Riswick, J.G.A., van Opstal, A.J.: Relearning sound localization with new ears. Nature Neuroscience, 1(5), 417–421 (1998)
Durlach, N.I., Shinn-Cunningham, B.G., Held, R.M.: Supernormal auditory localization. I. General background. Presence: Teleoperators and Virtual Environments, 2(2), 89–103 (1993)
Gupta, N., Barreto, A., Ordonez, C.: Spectral modification of head-related transfer functions for improved virtual sound spatialization. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, Orlando, FL, 2, 1953–1956 (2002)
Härmä, A., Jakka, J., Tikander, M., Karjalainen, M., Lokki, T., Hiipakka, J., Lorho, G.: Augmented reality audio for mobile and wearable appliances. Journal of the Audio Engineering Society, 52(6), 618–639 (2004)
Evans, M.J., Tew, A.I., Angus, J.A.S.: Spatial audio teleconferencing: Which way is better?: Proceedings of the fourth International Conference on Auditory Display (ICAD ’97), Palo Alto, California, 29–37 (1997)
Loomis, J.M., Golledge, R.G., Klatzky, R.L.: Navigation system for the blind: auditory display modes and guidance. Presence: Teleoperators and Virtual Environments, 7(2), 193–203 (1998)
Tonndorf, J.: Bone conduction. In J. Tobias (Ed.), Foundations of modern auditory theory (Vol. 2, pp. 197–237). Academic Press, New York (1972)
Stenfelt, S., Goode, R.L.: Bone conducted sound: physiological and clinical aspects. Otology & Neurotology, 26(6), 1245–1261, (2005)
Pörschmann, C.: Influences of bone conduction and air conduction on the sound of one’s own voice. Acustica – Acta Acustica, 86(6), 1038–1045 (2000)
Snik, A.F.M., Bosman, A.J., Mylanus, E.A.M., Cremers, C.W.R.J.: Candidacy for the bone-anchored hearing aid. Audiology & Neurotology, 9(4), 190–196 (2004)
Väljamäe, A., Tajadura-Jiménez, A., Larsson, P., Västfjäll, D., Kleiner M.: Binaural bone-conducted sound in virtual environments: evaluation of a portable, multimodal motion simulator prototype. Journal of Acoustic Science and Technology, 29(2), 149–155 (2008)
Gaye, L., Mazé, R., Holmquist, L.E.: Sonic City: The urban environment as a musical interface. Proceedings of the 2003 Conference on New Interfaces for Musical Expression (NIME-03), Montreal, Canada, 109–115. (2003)
Sanders, R.D. Jr., Scorgie, M.A.: The effect of sound delivery methods on a user’s sense of presence in a virtual environment. Master thesis Naval Postgraduate School, Monterey, CA (2002)
Durlach, N.I., Mavor, A.S.: Virtual reality scientific and technological challenges. National Academy Press, Washington, DC (1995)
Savioja, L., Huopaniemi, J., Lokki, T., Väänänen, R.: Creating interactive virtual acoustic environments. Journal of the Audio Engineering Society, 47(9), 675–705 (1999)
Jot, J.-M., Warusfel, O.: A real-time spatial sound processor for music and virtual reality applications. Proceedings of the 1995 International Computer Music Conference, Banff, AB, Canada, 294–295 (1995)
Faller, C., Baumgarte, F.: Binaural cue coding – Part II: schemes and applications. IEEE Transactions on Speech and Audio Processing, 11(6), 520–531. (2003)
Breebaart, J., van de Par, V., Kohlrausch, A., Schuijers, K.: Parametric coding of stereo audio. EURASIP Journal on Applied Signal Processing, 9, 1305–1322 (2005)
Tsingos, N., Gallo, E., Drettakis, G.: Perceptual audio rendering of complex virtual environments. INRIA technical report RR-4734, Feb. 2003, REVES/INRIA Sophia-Antipolis (2003)
Ramsdell, D.A.: The psychology of the hard-of-hearing and deafened adult. In H. Davis & S.R. Silverman (Eds.), Hearing and deafness (4th ed., pp. 499–510). Holt, Rinehart & Winston, New York (1978)
Hendrix, C., Barfield, W.: The sense of presence within auditory virtual environments. Presence: Teleoperators and Virtual Environments, 5(3), 290–301 (1996)
Väljamäe, A., Larsson, P., Västfjäll, D., Kleiner, M.: Auditory presence, individualized head-related transfer functions and illusory ego-motion in virtual environments. Proceedings of the Seventh Annual International Workshop Presence, Valencia, Spain, 141–147 (2004)
Begault, D.R., Wenzel, E.M., Anderson, M.R.: Direct comparison of the impact of head-tracking, reverberation, and individualized head-related transfer functions on the spatial perception of a virtual speech source. Journal of the Audio Engineering Society, 49(10), 904–916 (2001)
Larsson, P., Västfjäll, D., Kleiner, M.: Auditory information consistency and room acoustic cues increase presence in virtual environments. Acoustical Science and Technology, 29(2), 191–194 (2008)
Larsson, P., Västfjäll, D., Kleiner, M.: On the quality of experience: A multi-modal approach to perceptual ego-motion and sensed presence in virtual environments. Proceedings of the First ITRW on Auditory Quality of Systems, Akademie Mont-Cenis, Germany. (2003)
Ozawa, K., Chujo, Y., Suzuki, Y., Sone, T.: Psychological factors involved in auditory presence. Acoustical Science and Technology, 24(1), 42–44 (2003)
Rumsey, F.: Spatial quality evaluation for reproduced sound: terminology, meaning, and a scene-based paradigm. Journal of the Audio Engineering Society, 50(9), 651–666 (2002)
Freeman, J., Lessiter, J.: Here there & everywhere: The effects of multi-channel audio on presence. Proceedings of The Seventh International Conference on Audio Display, Helsinki, Finland, 231–234 (2001)
Ozawa, K., Miyasaka, M.: Effects of reproduced sound pressure levels on auditory presence. Acoustical Science and Technology, 25(3), 207–209 (2004)
Serafin, G., Serafin, S.: Sound design to enhance presence in photorealistic virtual reality. Proceedings of the 2004 International Conference on Auditory Display (ICAD ’04), Sydney, Australia (2004)
Chueng, P., Marsden, P.: Designing auditory spaces to support sense of place: The role of expectation. Position paper for The Role of Place in On-line Communities Workshop, CSCW2002, New Orleans (2002)
Larsson, P., Västfjäll, D., Olsson, P., Kleiner, M.: When what you see is what you hear: Auditory-visual integration and presence in virtual environments. Proceedings of the 10th Annual International Workshop Presence, 11–18. Barcelona, Spain (2007)
Ozawa, K., Ohtake, S., Suzuki, Y., Sone, T.: Effects of visual information on auditory presence. Acoustical Science and Technology, 24(2), 97–99 (2003)
Knapp, J.M., Loomis, J.M.: Limited field of view of head-mounted displays is not the cause of distance underestimation in virtual environments. Presence: Teleoperators and Virtual Environments, 13(5), 572–577 (2004)
Thompson, W.B., Willemsen, P., Gooch, A.A., Creem-Regehr, S.H., Loomis, J.M., Beall, A.C.: Does the quality of the computer graphics matter when judging distances in visually immersive environments? Presence: Teleoperators and Virtual Environments, 13(5), 560–571 (2004)
37Murch, W.: Dense clarity – Clear density. Paper delivered at the “Volume: Bed of sound” program/exhibition, PS1/MoMA, New York, NY. http://www.ps1.org/cut/volume/murch.html (2000)
Neuhoff, J.G: Ecological psychoacoustics: Introduction and history. In J.G. Neuhoff (Ed.), Ecological psychoacoustics. (pp. 1–13). Elsevier Academic Press, Amsterdam, Boston (2004)
Hughes, D.E., Thropp, J., Holmquist, J., Moshell, J.M.: Spatial perception and expectation: factors in acoustical awareness for MOUT training. Proceedings of Army Science Conference (ASC) 2004, Orlando, FL (2004)
Weiss, E., Belton J.: Film sound: Theory and practice. Columbia University Press, New York (1985)
Väljamäe, A., Tajadura-Jiménez, A., Larsson P., Västfjäll D., Kleiner M.: Handheld Experiences: Using Audio To Enhance the Illusion of Self-Motion, IEEE MultiMedia, vol. 15, no.4, pp. 68–75, 2008
Riecke B.E., Väljamäe, A., Schulte-Pelkum, J.: Moving sounds enhance the visually-induced self-motion illusion (circular vection) in Virtual Reality ACM Transactions on Applied Perception (TAP) Volume 6, Issue 2, Article No. 7 (2009)
Väljamäe, A., Soto-Faraco, S.: Filling-in visual motion with sounds. Acta Psychologica, 129(2), 249–254. (2008)
Acknowledgments
The work presented in this chapter was supported by the EU FET Presence Research Initiative project POEMS (IST-2001-39223), the EU FET Integrated project PRESENCCIA (Project Number 27731), and the Swedish Science Council (VR). The first author thanks the Swedish Foundation for Strategic Research for its support. The second author thanks Alfred Ots’ scholarship foundation. We would also like to thank Armin Kohlrausch for commenting on earlier drafts of this chapter.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag London
About this chapter
Cite this chapter
Larsson, P., Väljamäe, A., Västfjäll, D., Tajadura-Jiménez, A., Kleiner, M. (2010). Auditory-Induced Presence in Mixed Reality Environments and Related Technology. In: Dubois, E., Gray , P., Nigay, L. (eds) The Engineering of Mixed Reality Systems. Human-Computer Interaction Series. Springer, London. https://doi.org/10.1007/978-1-84882-733-2_8
Download citation
DOI: https://doi.org/10.1007/978-1-84882-733-2_8
Published:
Publisher Name: Springer, London
Print ISBN: 978-1-84882-732-5
Online ISBN: 978-1-84882-733-2
eBook Packages: Computer ScienceComputer Science (R0)