skip to main content
10.1145/3595916.3626354acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
demonstration

Directional Sound Source Representation Using Paired Microphone Array with Different Characteristics Suitable for Volumetric Video Capture

Published: 01 January 2024 Publication History

Abstract

In this research, we propose a directional sound source representation technique for 3D contents such as volumetric video in metaverse and digital twin. Our proposed technique enables us to have a novel 3D audio-visual experience which is derived from immersive audio presentation expressing the radiation characteristics of sound source. To realize such an experience, we configure the spaced placement of paired microphone array and capture sound source signals completely without obstacles for volumetric video capture. Then, we synthesize the directional sound source signal using our technique which conducts signal processing to capture sound signals based on the positional and directional information of an object relative to a user. We developed and demonstrated a VR application using this technique to evaluate the change of sound with the object or user's movement in accordance with visual rendering. In our user study, we received lots of positive feedback for a novel audio-visual experience.

Supplementary Material

MP4 File (ACM MM camera ready.mp4)
Video explaining the demonstration

References

[1]
Greg Slabaugh, Bruce Culbertson, Tom Malzbender, and Ron Schafer. 2001. A survey of methods for volumetric scene reconstruction from photographs. In Proceedings of the 2001 Eurographics Conference on Volume Graphics (VG'01). Eurographics Association, Goslar, DEU, 81–101.
[2]
Adrien Maglo, Guillaume Lavoué, Florent Dupont, and Céline Hudelot. 2015. 3D Mesh Compression: Survey, Comparisons, and Emerging Trends. ACM Comput. Surv. 47, 3, Article 44 (April 2015), 41 pages.
[3]
Huseyin. Hacihabiboglu, Enzo De Sena, Zoran Cvetkovic, James Johnston and Julius O. Smith III. 2017. Perceptual Spatial Audio Recording, Simulation, and Rendering: An Overview of Spatial-audio Techniques Based on Psychoacoustics. IEEE Signal Processing Magazine. 34, 3, (May 2017), 36-54.
[4]
Jean-Marc Jot, Rémi Audfray, Mark Hertensteiner and Brian Schmidt. 2021. Rendering Spatial Sound for Interoperable Experiences in the Audio Metaverse. 2021 Immersive and 3D Audio: from Architecture to Automotive (I3DA). 1-15.
[5]
F. Pedersini, A. Sarti and S. Tubaro. 2000. Object-based Sound Synthesis for Virtual Environments-using Musical Acoustics. IEEE Signal Processing Magazine. 17, 6, (Nov. 2000), 37-51.
[6]
Jens Blauert. 1996. Spatial hearing: The psychophysics of human sound localization. MIT Press, Cambridge, MA, USA.
[7]
Roger K. Furness. 1990. Ambisonics - An overview. In Proc. AES InternationalConference: The Sound of Audio. AES, New York, NY, USA, 181–189.
[8]
Katuhiro Maki, Toshiyuki Kimura, and Michiaki Katsumoto. 2010. Reproduction of sound radiation directivities of musical instruments by a spherical loudspeaker with multiple transducers. In Proceedings of the 9th ACM SIGGRAPH Conference on Virtual-Reality Continuum and its Applications in Industry (VRCAI '10). Association for Computing Machinery, New York, NY, USA, 85–88. https://doi.org/10.1145/1900179.1900197
[9]
Camilla H. Larsen, David S. Lauritsen, Jacob J. Larsen, Marc Pilgaard, and Jacob B. Madsen. 2013. Differences in human audio localization performance between a HRTF- and a non-HRTF audio system. In Proceedings of the 8th Audio Mostly Conference (AM '13). Association for Computing Machinery, New York, NY, USA, Article 5, 1–8.

Index Terms

  1. Directional Sound Source Representation Using Paired Microphone Array with Different Characteristics Suitable for Volumetric Video Capture

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    MMAsia '23: Proceedings of the 5th ACM International Conference on Multimedia in Asia
    December 2023
    745 pages
    ISBN:9798400702051
    DOI:10.1145/3595916
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 01 January 2024

    Check for updates

    Author Tags

    1. Sound radiation
    2. spaced array of microphones
    3. volumetric video

    Qualifiers

    • Demonstration
    • Research
    • Refereed limited

    Funding Sources

    • National Institute of Information and Communications Technology (NICT)

    Conference

    MMAsia '23
    Sponsor:
    MMAsia '23: ACM Multimedia Asia
    December 6 - 8, 2023
    Tainan, Taiwan

    Acceptance Rates

    Overall Acceptance Rate 59 of 204 submissions, 29%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 54
      Total Downloads
    • Downloads (Last 12 months)28
    • Downloads (Last 6 weeks)3
    Reflects downloads up to 28 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media