Matthias Frank
ABSTRACT
Matching visual and auditory cues in virtual reality (VR) is important to provide plausibility and create the impression of presence in a scene. This paper presents an experiment in VR, in which participants match acoustic and visual room size, distance, and orientation of a directive sound source in a simulated concert hall. The simulation is fully interactive and allows the participants to move with 6 degrees of freedom. For all three parameters, the experiment was done in both directions: adjusting the acoustic parameters to given visual settings and adjusting the visual parameters to given acoustic settings. The results show that the adjustment generally works in both directions. However, for distance the auditory adjustment works better and does not reveal the typical compression. Regarding room size, results agree with just noticeable differences in reverberation time known from real-world experiments.
- Niels W Adelman-Larsen and Jens Jørgen Dammerud. 2011. A survey of reverberation times in 50 European venues presenting pop & rock concerts. In Proceedings of Forum Acusticum.Google Scholar
- Leo L Beranek. 1992. Concert hall acoustics—1992. The Journal of the Acoustical Society of America 92, 1 (1992), 1–39.Google ScholarCross Ref
- Changan Chen, Ruohan Gao, Paul Calamia, and Kristen Grauman. 2022. Visual Acoustic Matching. arXiv preprint arXiv:2202.06875(2022).Google Scholar
- EBU. 2008. EBU SQAM CD: Sound Quality Assessment Material recordings for subjective tests. https://tech.ebu.ch/publications/sqamcdGoogle Scholar
- Kajetan Enge, Matthias Frank, and Robert Höldrich. 2020. Listening experiment on the plausibility of acoustic modeling in virtual reality. In Fortschritte der Akustik, DAGA.Google Scholar
- Matthias Frank and Manuel Brandner. 2019. Perceptual Evaluation of Spatial Resolution in Directivity Patterns. In Fortschritte der Akustik, DAGA. Rostock.Google Scholar
- Matthias Frank, Manuel Brandner, and Franz Zotter. 2022. Perceptual Evaluation of Spatial Resolution in Early Reflections. In Fortschritte der Akustik, DAGA.Google Scholar
- Jean-Marc Jot and Antoine Chaigne. 1991. Digital delay networks for designing artificial reverberators. In 90th AES Conv., prepr. 3030. Paris.Google Scholar
- Andrew J Kolarik, Brian CJ Moore, Pavel Zahorik, Silvia Cirstea, and Shahina Pardhan. 2016. Auditory distance perception in humans: a review of cues, development, neuronal bases, and effects of sensory loss. Attention, Perception, & Psychophysics 78, 2 (2016), 373–395.Google ScholarCross Ref
- Antti Kuusinen and Tapio Lokki. 2015. Investigation of auditory distance perception and preferences in concert halls by using virtual acoustics. The Journal of the Acoustical Society of America 138, 5 (2015), 3148–3159.Google ScholarCross Ref
- Erik Larsen, Nandini Iyer, Charissa R Lansing, and Albert S Feng. 2008. On the minimum audible difference in direct-to-reverberant energy ratio. The Journal of the Acoustical Society of America 124, 1 (2008), 450–461.Google ScholarCross Ref
- Pontus Larsson, Daniel Västfjäll, Pierre Olsson, Mendel Kleiner, 2007. When what you hear is what you see: Presence and auditory-visual integration in virtual environments. In Proceedings of the 10th annual international workshop on presence. 11–18.Google Scholar
- Zihou Meng, Fengjie Zhao, and Mu He. 2006. The just noticeable difference of noise length and reverberation perception. In 2006 International Symposium on Communications and Information Technologies. IEEE, 418–421.Google ScholarCross Ref
- Djordje Perinovic and Matthias Frank. 2021. Spatial Resolution of Diffuse Reverberation in Binaural Ambisonic Playback. In Fortschritte der Akustik, DAGA. Munich.Google Scholar
- Christian Schörkhuber, Markus Zaunschirm, and Robert Höldrich. 2018. Binaural Rendering of Ambisonic Signals via Magnitude Least Squares. In Fortschritte der Akustik, DAGA. Munich.Google Scholar
- Magne Skålevik. 2010. Reverberation Time–The mother of all room acoustic parameters. In Proceedings of 20th International Congress on Acoustic, ICA, Vol. 10.Google Scholar
- John Stautner and Miller Puckette. 1982. Designing Multi-Channel Reverberators. Computer Music Journal 6, 1 (1982), 52–65.Google ScholarCross Ref
- Stephan Werner, Florian Klein, Thomas Mayenfels, and Karlheinz Brandenburg. 2016. A summary on acoustic room divergence and its effect on externalization of auditory events. In 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX). IEEE, 1–6.Google ScholarCross Ref
- Pavel Zahorik and Frederic L Wightman. 2001. Loudness constancy with varying sound source distance. Nature neuroscience 4, 1 (2001), 78–83.Google Scholar
- Markus Zaunschirm, Christian Schörkhuber, and Robert Höldrich. 2018. Binaural rendering of Ambisonic signals by head-related impulse response time alignment and a diffuseness constraint. J. Acoust. Soc. Am. 143, 6 (2018), 3616–3627.Google ScholarCross Ref
- Franz Zotter and Matthias Frank. 2015. Investigation of auditory objects caused by directional sound sources in rooms. Acta Physica Polonica A 128, 1 (2015), A5–A10.Google ScholarCross Ref
- Franz Zotter and Matthias Frank. 2019. Ambisonics - A Practical 3D Audio Theory for Recording, Studio Production, Sound Reinforcement, and Virtual Reality. Springer.Google Scholar
- Franz Zotter, Matthias Frank, Andreas Fuchs, and Daniel Rudrich. 2014. Preliminary study on the perception of orientation-changing directional sound sources in rooms. In Proc. of forum acusticum, Kraków.Google Scholar
Index Terms
- Matching auditory and visual room size, distance, and source orientation in virtual reality
Recommendations
Virtual Room Re-Creation: A New Measure of Room Size Perception
SAP '21: ACM Symposium on Applied Perception 2021The perception of the size of indoor rooms is traditionally measured with spaciousness ratings or verbal estimations of width and length, but these measures inherently focus on specific aspects of rooms that may not accurately capture a holistic ...
The impact of sound control room acoustics on the perceived acoustics of a diffuse field recording
Live recordings of music and speech in concert halls have acoustical properties, such as reverberation, definition, clarity and spaciousness. Sound engineers play back these recordings through loudspeakers in sound control rooms for audio CD or film. ...
Multichannel Room Impulse Response Generation With Coherence Control
A method is proposed by which an arbitrary number of room impulse responses (RIRs) is generated from one input RIR. The method works by convolving the input RIR with a multitude of filters, one for each desired output RIR. The filters are designed in ...
Comments