research-article

Parametric time-frequency representation of spatial sound in virtual worlds

Authors:

Mikko-Ville Laitinen,

Tapani Pihlajamäki,

Ville PulkkiAuthors Info & Claims

ACM Transactions on Applied Perception (TAP), Volume 9, Issue 2

Article No.: 8, Pages 1 - 20

https://doi.org/10.1145/2207216.2207219

Published: 15 June 2012 Publication History

Abstract

Directional audio coding (DirAC) is a parametric time-frequency domain method for processing spatial audio based on psychophysical assumptions and on energetic analysis of the sound field. Methods to use DirAC in spatial sound synthesis for virtual worlds are presented in this article. Formal listening tests are used to show that DirAC can be used to position and to control the spatial extent of virtual sound sources with good audio quality. It is also shown that DirAC can be used to generate reverberation for N-channel horizontal listening with only two monophonic reverberators without a prominent loss in quality when compared with quality obtained with N-channel reverberators.

References

[1]

Allen, J. B. and Berkley, D. A. 1979. Image method for efficiently simulating small-room acoustic. J. Acoust. Soc. Amer. 65, 4, 943--950.

[2]

Baumgarte, F. and Faller, C. 2003. Binaural cue coding. Part I: Psychoacoustic fundamentals and design principles. IEEE Trans. Speech Audio Process. 11, 6, 509--519.

[3]

Begault, D. R., Wenzel, E. M., and Anderson, M. R. 2001. Direct comparison of the impact of head-tracking, reverberation, and individualized head-related transfer functions on the spatial perception of a virtual speech source. J. Audio Eng. Soc. 49, 10, 904--916.

[4]

Bennett, J. C., Barker, K., and Edeko, F. O. 1985. A new approach to the assessment of stereophonic sound system performance. J. Audio Eng. Soc. 33, 5, 314--321.

[5]

Berkhout, A. J., de Vries, D., and Vogel, P. 1993. Acoustics control by wave field synthesis. J. Acoust. Soc. Amer. 93, 5, 2764--2778.

[6]

Blauert, J. 1997. Spatial Hearing. The MIT Press, Cambridge, MA.

[7]

Blumlein, A. D. 1958. British patent specification 394,325, 1931. J. Audio Eng. Soc. 6, 2.

[8]

Del Galdo, G., Pulkki, V., Kuech, F., Laitinen, M.-V., Schultz-Amling, R., and Kallinger, M. 2009. Efficient methods for high quality merging of spatial audio streams in directional audio coding. In Proceedings of the AES 126th Convention.

[9]

Furness, R. K. 1990. Ambisonics - An overview. In Proceedings of the AES 8th International Conference.

[10]

Goodwin, M. M. and Jot, J.-M. 2006. A frequency-domain framework for spatial audio coding based on universal spatial cues. In Proceedings of the 120th AES Convention. Paper 6751.

[11]

Herre, J., Kjörling, K., Breebaart, J., Faller, C., Disch, S., Purnhagen, H., Koppens, J., Hilpert, J., Roden, J., Oomen, W., et al. 2008. MPEG surround-the ISO/MPEG standard for efficient and compatible multichannel audio coding. J. Audio Eng. Soc. 56, 932--955.

[12]

Hirvonen, T. 2007. Perceptual and modeling studies on spatial sound. Ph.D. dissertation, Helsinki University of Technology.

[13]

Hirvonen, T., Ahonen, J., and Pulkki, V. 2009. Perceptual compression methods for metadata in directional audio coding applied to audiovisual teleconference. In Proceedings of the AES 126th Convention.

[14]

ITU-R BS.1116-1. 1997. Methods for the subjective assessment of small impairments in audio systems including multichannel sound systems. ITU.

[15]

ITU.ITU-R BS.1534-1. 2003. Method for the subjective assessment of intermediate quality level of coding systems. ITU.

[16]

Jot, J.-M., Walsh, M., and Philp, A. 2006. Binaural simulation of complex acoustic scenes for interactive audio. In Proceedings of the AES 121st Convention.

[17]

Kirszenstein, J. 1984. An image source computer model for room acoustics analysis and electroacoustic simulation. Appl. Acoust. 17, 4, 275--290.

[18]

Laitinen, M.-V. and Pulkki, V. 2009. Binaural reproduction for directional audio coding. In Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, IEEE, Los Alamitos, CA.

[19]

Laitinen, M.-V. and Pulkki, V. 2011. Converting 5.1 audio recordings to B-format for directional audio coding reproduction. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing.

[20]

Lentz, T., Schröder, D., Vorlander, M., and Assenmacher, I. 2007. Virtual reality system with integrated sound field simulation and reproduction. EURASIP J. Appl. Signal Process. 1, 187--187.

Digital Library

[21]

Manocha, D., Calamia, P., Lin, M. C., Savioja, L., and Tsingos, N. 2009. Interactive sound rendering. In ACM SIGGRAPH'09 Courses. ACM, New York.

Digital Library

[22]

Moller, H., Sorensen, M. F., Hammershoi, D., and Jensen, C. B. 1995. Head-related transfer functions of human subjects. J. Audio Eng. Soc. 43, 5 (May), 300--321.

[23]

Moore, B. C. J. 1982. An Introduction to the Psychology of Hearing. Academic Press.

[24]

Moore, B. C. J. 1995. Hearing. Academic Press.

[25]

OpenAL. 2000. OpenAL: An open source 3D sound library. http://www.openal.org.

[26]

Potard, G. and Burnett, I. 2004. Decorrelation techniques for the rendering of apparent sound source width in 3D audio displays. In Proceedings of the 7th International Conference on Digital Audio Effects.

[27]

Pulkki, V. 1997. Virtual source positioning using vector base amplitude panning. J. Audio Eng. Soc. 45, 6, 456--466.

[28]

Pulkki, V. 2001. Spatial sound generation and perception by amplitude panning techniques. Ph.D. dissertation, Helsinki University of Technology.

[29]

Pulkki, V. 2007. Spatial sound reproduction with directional audio coding. J. Audio Eng. Soc. 55, 6, 503--516.

[30]

Pulkki, V., Laitinen, M.-V., and Erkut, C. 2009. Efficient spatial sound synthesis for virtual worlds. In Proceedings of the AES 35th International Conference.

[31]

Pulkki, V., Lokki, T., and Rocchesso, D. 2011. Spatial effects. In DAFX: Digital Audio Effects 2nd Ed., U. Zoelzer Ed., Wiley.

[32]

Riecke, B. E., Valjamae, A., and Schulte-Pelkum, J. 2009. Moving sounds enhance the visually-induced self-motion illusion (circular vection) in virtual reality. ACM Trans. Appl. Percept. 6, 2, 7--27.

Digital Library

[33]

Santala, O. and Pulkki, V. 2011. Directional perception of distributed sound sources. J. Acoust. Soc. Am. 129, 3, 1522--1530.

[34]

Savioja, L., Huopaniemi, J., Lokki, T., and Vaananen, R. 1999. Creating interactive virtual acoustic environments. J. Audio Eng. Soc. 47, 9, 675--705.

[35]

Seeber, B., Kerber, S., and Hafter, E. 2010. A system to simulate and reproduce audio-visual environments for spatial hearing research. Hearing Res. 260, 1--2, 1--10.

[36]

Tsingos, N., Gallo, E., and Drettakis, G. 2004. Perceptual audio rendering of complex virtual environments. In Proceedings of the SIGGRAPH 31st International Conference on Computer Graphics and Interactive Techniques.

Digital Library

[37]

Verron, C., Aramaki, M., Kronland-Martinet, R., and Pallone, G. 2010. A 3-D immersive synthesizer for environmental sounds. IEEE Trans. Audio, Speech, Lang. Process. 18, 6, 1550--1561.

Digital Library

[38]

Vilkamo, J., Lokki, T., and Pulkki, V. 2009. Directional audio coding: Virtual microphone based synthesis and subjective evaluation. J. Audio Eng. Soc. 57, 9, 709--724.

Cited By

Zuo HBirnie LSamarasinghe PAbhayapala TTourbabin V(2023)Particle-Velocity-Based Mixed-Source Sound Field Translation for Binaural ReproductionApplied Sciences10.3390/app1311644913:11(6449)Online publication date: 25-May-2023
https://doi.org/10.3390/app13116449
Frank MZotter F(2023)AmbisonicsHandbuch der Audiotechnik10.1007/978-3-662-60357-4_25-1(1-20)Online publication date: 5-May-2023
https://doi.org/10.1007/978-3-662-60357-4_25-1
Zotter FFrank MKim J(2019)Signal Flow and Effects in Ambisonic ProductionsGenome Data Analysis10.1007/978-3-030-17207-7_5(99-129)Online publication date: 1-May-2019
https://doi.org/10.1007/978-3-030-17207-7_5
Show More Cited By

Index Terms

Parametric time-frequency representation of spatial sound in virtual worlds

Recommendations

Spatial sound localization in an augmented reality environment
OZCHI '06: Proceedings of the 18th Australia conference on Computer-Human Interaction: Design: Activities, Artefacts and Environments

Augmented Reality (AR), the overlay of virtual images onto the real world, is an increasingly popular technique for developing new human-computer interfaces. As human navigation and orientation in different environments depend on both visual and ...
Spatial Sound and Multimodal Interaction in Immersive Environments
AM '15: Proceedings of the Audio Mostly 2015 on Interaction With Sound

Spatial sound and interactivity are key elements of investigation at the Sound And Music Computing master program at Aalborg University Copenhagen.

We present a collection of research directions and recent results from work in these areas, with the ...
An experimental study of spatial sound usefulness in searching and navigating through AR environments

This paper presents an experimental study of spatial sound usefulness in searching and navigating through augmented reality environments. Participants were asked to find three objects hidden within no-sound and spatial sound AR environments. The ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Applied Perception

ACM Transactions on Applied Perception Volume 9, Issue 2

June 2012

73 pages

ISSN:1544-3558

EISSN:1544-3965

DOI:10.1145/2207216

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 June 2012

Accepted: 01 November 2011

Revised: 01 October 2011

Received: 01 April 2011

Published in TAP Volume 9, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Seventh Framework Programme

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
661
Total Downloads

Downloads (Last 12 months)29
Downloads (Last 6 weeks)6

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zuo HBirnie LSamarasinghe PAbhayapala TTourbabin V(2023)Particle-Velocity-Based Mixed-Source Sound Field Translation for Binaural ReproductionApplied Sciences10.3390/app1311644913:11(6449)Online publication date: 25-May-2023
https://doi.org/10.3390/app13116449
Frank MZotter F(2023)AmbisonicsHandbuch der Audiotechnik10.1007/978-3-662-60357-4_25-1(1-20)Online publication date: 5-May-2023
https://doi.org/10.1007/978-3-662-60357-4_25-1
Zotter FFrank MKim J(2019)Signal Flow and Effects in Ambisonic ProductionsGenome Data Analysis10.1007/978-3-030-17207-7_5(99-129)Online publication date: 1-May-2019
https://doi.org/10.1007/978-3-030-17207-7_5
Zhang ZRaghuvanshi NSnyder JMarschner S(2018)Ambient sound propagationACM Transactions on Graphics10.1145/3272127.327510037:6(1-10)Online publication date: 4-Dec-2018
https://dl.acm.org/doi/10.1145/3272127.3275100
Raghuvanshi NSnyder J(2018)Parametric directional coding for precomputed sound propagationACM Transactions on Graphics10.1145/3197517.320133937:4(1-14)Online publication date: 30-Jul-2018
https://dl.acm.org/doi/10.1145/3197517.3201339
Çamcı ALee KRoberts CForbes AGajos KMankoff JHarrison C(2017)INVISOProceedings of the 30th Annual ACM Symposium on User Interface Software and Technology10.1145/3126594.3126644(507-518)Online publication date: 20-Oct-2017
https://dl.acm.org/doi/10.1145/3126594.3126644
Pulkki VPolitis APihlajamäki TLaitinen M(2017)Spatial Sound Scene Synthesis and Manipulation for Virtual Reality and Audio EffectsParametric Time‐Frequency Domain Spatial Audio10.1002/9781119252634.ch14(347-361)Online publication date: 13-Oct-2017
https://doi.org/10.1002/9781119252634.ch14
Ziemer T(2016)Source Width in Music Production. Methods in Stereo, Ambisonics, and Wave Field SynthesisStudies in Musical Acoustics and Psychoacoustics10.1007/978-3-319-47292-8_10(299-340)Online publication date: 27-Dec-2016
https://doi.org/10.1007/978-3-319-47292-8_10
Grani FOverholt DErkut CGelineck STriantafyllidis GNordahl RSerafin SKalliris GDimoulas C(2015)Spatial Sound and Multimodal Interaction in Immersive EnvironmentsProceedings of the Audio Mostly 2015 on Interaction With Sound10.1145/2814895.2814919(1-5)Online publication date: 7-Oct-2015
https://dl.acm.org/doi/10.1145/2814895.2814919
Pugliese RPolitis ATakala TVerplank BJu WAntle AMazalek AMueller F(2015)ATSIProceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/2677199.2680550(97-104)Online publication date: 15-Jan-2015
https://dl.acm.org/doi/10.1145/2677199.2680550
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents