research-article

INVISO: A Cross-platform User Interface for Creating Virtual Sonic Environments

Authors:

Cody J. Roberts,

Angus G. ForbesAuthors Info & Claims

UIST '17: Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology

Pages 507 - 518

https://doi.org/10.1145/3126594.3126644

Published: 20 October 2017 Publication History

Abstract

The predominant interaction paradigm of current audio spatialization tools, which are primarily geared towards expert users, imposes a design process in which users are characterized as stationary, limiting the application domain of these tools. Navigable 3D sonic virtual realities, on the other hand, can support many applications ranging from soundscape prototyping to spatial data representation. Although modern game engines provide a limited set of audio features to create such sonic environments, the interaction methods are inherited from the graphical design features of such systems, and are not specific to the auditory modality. To address such limitations, we introduce INVISO, a novel web-based user interface for designing and experiencing rich and dynamic sonic virtual realities. Our interface enables both novice and expert users to construct complex immersive sonic environments with 3D dynamic sound components. INVISO is platform-independent and facilitates a variety of mixed reality applications, such as those where users can simultaneously experience and manipulate a virtual sonic environment. In this paper, we detail the interface design considerations for our audio-specific VR tool. To evaluate the usability of INVISO, we conduct two user studies: The first demonstrates that our visual interface effectively facilitates the generation of creative audio environments; the second demonstrates that both expert and non-expert users are able to use our software to accurately recreate complex 3D audio scenes.

Supplementary Material

suppl.mov (uistf3835.mp4)

Supplemental video

Download
106.79 MB

References

[1]

Paul Adenot and Chris Wilson. 2015. Web Audio API. (2015). Available: http://webaudio.github.io/web-audio-api/.

[2]

Michel Beaudouin-Lafon and William W Gaver. 1994. ENO: synthesizing structured sound spaces. In Proceedings of the 7th Annual ACM Symposium on User Interface Software and Technology (UIST). 49--57.

Digital Library

[3]

Benjamin B. Bederson, James D. Hollan, Ken Perlin, Jonatham Meyer, David Bacon, and George Furnas. 1995. PAD++: A Zoomable Graphical Sketchpad for Exploring Alternate Interface Physics. Journal of Visual Languages and Computing 7 (1995), 3--31.

[4]

Doug A. Bowman, Chris North, Jian Chen, Nicholas F. Polys, Pardha S. Pyla, and Umur Yilmaz. 2003. Information-rich Virtual Environments: Theory, Tools, and Research Agenda. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST). 81--90.

Digital Library

[5]

Thibaut Carpentier. 2015. Binaural synthesis with the Web Audio API. In Proceedings of the 1st Web Audio Conference.

[6]

Anıl Çamcı, Paul Murray, and Angus G. Forbes. 2016. Designing and Controlling Virtual Sonic Environments Using a Browser-based 3DUI. In Proceedings of the IEEE Symposium on 3D User Interfaces (3DUI). 235--236.

[7]

Anıl Çamcı, Paul Murray, and Angus G. Forbes. 2016. A Web-based System for Designing Interactive Virtual Soundscapes. In Proceedings of the 42nd International Computer Music Conference (ICMC). 579--584.

[8]

Anıl Çamcı, Zeynep Özcan, and Damla Pehlevan. 2015. Interactive Virtual Soundscapes: a Research Report. In Proceedings of the 41st International Computer Music Conference (ICMC). 163--169.

[9]

Thomas Funkhouser, Jean Marc Jot, and Nicolas Tsingos. 2002. "Sounds good to me!"--Computational sound for graphics, virtual reality, and interactive systems. ACM SIGGRAPH Course Notes (2002), 1--43.

[10]

Bo Gehring. 1988. Attitude Indicator. (27 September 1988). US Patent #4,774,515.

[11]

Matthias Geier and Sascha Spors. 2012. Spatial Audio with the SoundScape Renderer. In 27th Tonmeistertagung--VDT International Convention.

[12]

Morton L. Heilig. 1960. Stereoscopic-television apparatus for individual use. (4 October 1960). US Patent #2,955,156.

[13]

Jyri Huopaniemi, Lauri Savioja, and Tapio Takala. 1996. DIVA virtual audio reality system. In Proceedings of International Conference on Auditory Display (ICAD).

[14]

Jacek Jankowski and Stefan Decker. 2012. A Dual-mode User Interface for Accessing 3D Content on the World Wide Web. In Proceedings of the 21st International Conference on World Wide Web (WWW). 1047--1056.

Digital Library

[15]

Mikko-Ville Laitinen, Tapani Pihlajamäki, Cumhur Erkut, and Ville Pulkki. 2012. Parametric time-frequency representation of spatial sound in virtual worlds. ACM Transactions on Applied Perception (TAP) 9, 2 (2012), 8:1--20.

Digital Library

[16]

Kyla McMullen. 2014. The potentials for spatial audio to convey information in virtual environments. In IEEE VR Workshop on Sonic Interaction in Virtual Environments (SIVE). 31--34.

[17]

Ravish Mehra, Atul Rungta, Abhinav Golas, Ming Lin, and Dinesh Manocha. 2015. WAVE: Interactive Wave-based Sound Propagation for Virtual Environments. IEEE Transactions on Visualization and Computer Graphics 21, 4 (2015), 434--442.

Digital Library

[18]

Elizabeth D Mynatt, Maribeth Back, Roy Want, and Ron Frederick. 1997. Audio Aura: Light-weight audio augmented reality. In Proceedings of the 10th Annual ACM Symposium on User interface Software and Technology (UIST). 211--212.

Digital Library

[19]

Ji-Young Oh and Wolfgang Stuerzlinger. 2005. Moving Objects with 2D Input Devices in CAD Systems and Desktop Virtual Environments. In Proceedings of Graphics Interface. 195--202.

Digital Library

[20]

Rui Penha and J Oliveira. 2013. Spatium, tools for sound spatialization. In Proceedings of the Sound and Music Computing Conference. 660--667.

[21]

Chris Pike, Peter Taylour, and Frank Melchior. 2015. Delivering Object-Based 3D Audio Using The Web Audio API And The Audio Definition Model. In Proceedings of the 1st Web Audio Conference.

[22]

Khairi Reda, Alessandro Febretti, Aaron Knoll, Jillian Aurisano, Jason Leigh, Andrew Johnson, Michael E Papka, and Mark Hereld. 2013. Visualizing large, heterogeneous data in hybrid-reality environments. IEEE Computer Graphics and Applications 4 (2013), 38--48.

Digital Library

[23]

Mathias Rossignol, Gregoire Lafay, Mathieu Lagrange, and Nicolas Misdarris. 2015. SimScene: A web-based acoustic scenes simulator. In Proceedings of the 1st Web Audio Conference.

[24]

Joseph Rozier, Karrie Karahalios, and Judith Donath. 2000. Hear&There: An Augmented Reality System of Linked Audio. In Proceedings of International Conference on Auditory Display (ICAD).

[25]

Lauri Savioja, Jyri Huopaniemi, Tapio Lokki, and Ritta Väänänen. 1999. Creating Interactive Virtual Acoustic Environments. J. Audio Eng. Soc 47, 9 (1999), 675--705.

[26]

Chris Schmandt. 1998. Audio hallway: A virtual acoustic environment for browsing. In Proceedings of the 11th Annual ACM Symposium on User Interface Software and Technology (UIST). 163--170.

Digital Library

[27]

B Shinn-Cunningham. 1998. Applications of virtual auditory displays. In Proceedings of the International Conference of the IEEE Engineering in Biology and Medicine Society, Vol. 3. 1105--1108.

[28]

Ben Shneiderman. 2007. Creativity support tools: Accelerating discovery and innovation. Commun. ACM 50, 12 (2007), 20--32.

Digital Library

[29]

Ken Shoemake. 1992. ARCBALL: A User Interface for Specifying Three-dimensional Orientation Using a Mouse. In Proceedings of the Conference on Graphics Interface. 151--156.

Digital Library

[30]

Tapio Takala and James Hahn. 1992. Sound rendering. Computer Graphics 26, 2 (1992), 211--220.

Digital Library

[31]

Tan Yiyu, Yasushi Inoguchi, Eiko Sugawara, Makoto Otani, Yukio Iwaya, Yukinori Sato, Hiroshi Matsuoka, and Takao Tsuchiya. 2011. A real-time sound field renderer based on digital Huygens' model. Journal of Sound and Vibration 330, 17 (2011), 4302--4312.

[32]

Shengkui Zhao, Ryan Rogowski, Reece Johnson, and Douglas L Jones. 2012. 3D binaural audio capture and reproduction using a miniature microphone array. In Proceedings of the 15th International Conference on Digital Audio Effects (DAFx). 151--154.

Cited By

Ferreira CCavaco SRodrigues ACorreia N(2023)Immerscape: Supporting the Creation of Immersive Soundscapes by Users in Cultural Heritage ContextsAdvances in Mobile Computing and Multimedia Intelligence10.1007/978-3-031-48348-6_15(179-186)Online publication date: 4-Dec-2023
https://dl.acm.org/doi/10.1007/978-3-031-48348-6_15
Comunità MGerino APicinali L(2022)PlugSonic: a web- and mobile-based platform for dynamic and navigable binaural audioEURASIP Journal on Audio, Speech, and Music Processing10.1186/s13636-022-00250-x2022:1Online publication date: 15-Jul-2022
https://dl.acm.org/doi/10.1186/s13636-022-00250-x
Johansen Svan Berkel NFritsch J(2022)Characterising Soundscape Research in Human-Computer InteractionProceedings of the 2022 ACM Designing Interactive Systems Conference10.1145/3532106.3533458(1394-1417)Online publication date: 13-Jun-2022
https://dl.acm.org/doi/10.1145/3532106.3533458
Show More Cited By

Index Terms

INVISO: A Cross-platform User Interface for Creating Virtual Sonic Environments
1. Human-centered computing
  1. Human computer interaction (HCI)
2. Information systems
  1. Information systems applications
    1. Multimedia information systems

Recommendations

A true spatial sound system for CAVE-like displays using four loudspeakers
ICVR'07: Proceedings of the 2nd international conference on Virtual reality

The paper introduces an audio rendering system based on the binaural approach, which allows a real-time simulation of spatially distributed sound sources and, in addition to that, near-to-head sources in room-mounted virtual environments. We have been ...
Spatialized audio rendering for immersive virtual environments
VRST '02: Proceedings of the ACM symposium on Virtual reality software and technology

We present a spatialized audio rendering system for the use in immersive virtual environments. The system is optimized for rendering a sufficient number of dynamically moving sound sources in multi-speaker environments using off-the-shelf audio ...
A framework for immersive spatial audio performance
NIME '06: Proceedings of the 2006 conference on New interfaces for musical expression

Traditional uses of virtual audio environments tend to focus on perceptually accurate acoustic representations. Though spatialization of sound sources is important, it is necessary to leverage control of the sonic representation when considering musical ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '17: Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology

October 2017

870 pages

ISBN:9781450349819

DOI:10.1145/3126594

General Chair:
Krzysztof Gajos
Harvard University
,
Program Chairs:
Jennifer Mankoff
Carnegie Mellon University
,
Chris Harrison
Carnegie Mellon University

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 October 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UIST '17

Sponsor:

UIST '17: The 30th Annual ACM Symposium on User Interface Software and Technology

October 22 - 25, 2017

QC, Québec City, Canada

Acceptance Rates

UIST '17 Paper Acceptance Rate 73 of 324 submissions, 23%;

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
469
Total Downloads

Downloads (Last 12 months)60
Downloads (Last 6 weeks)8

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ferreira CCavaco SRodrigues ACorreia N(2023)Immerscape: Supporting the Creation of Immersive Soundscapes by Users in Cultural Heritage ContextsAdvances in Mobile Computing and Multimedia Intelligence10.1007/978-3-031-48348-6_15(179-186)Online publication date: 4-Dec-2023
https://dl.acm.org/doi/10.1007/978-3-031-48348-6_15
Comunità MGerino APicinali L(2022)PlugSonic: a web- and mobile-based platform for dynamic and navigable binaural audioEURASIP Journal on Audio, Speech, and Music Processing10.1186/s13636-022-00250-x2022:1Online publication date: 15-Jul-2022
https://dl.acm.org/doi/10.1186/s13636-022-00250-x
Johansen Svan Berkel NFritsch J(2022)Characterising Soundscape Research in Human-Computer InteractionProceedings of the 2022 ACM Designing Interactive Systems Conference10.1145/3532106.3533458(1394-1417)Online publication date: 13-Jun-2022
https://dl.acm.org/doi/10.1145/3532106.3533458
Comunità MGerino ALim VPicinali L(2021)Design and Evaluation of a Web- and Mobile-Based Binaural Audio Platform for Cultural HeritageApplied Sciences10.3390/app1104154011:4(1540)Online publication date: 8-Feb-2021
https://doi.org/10.3390/app11041540
McArthur ATonder CGaston-Bird LKnight-Hill A(2021)A survey of 3D audio through the browser: practitioner perspectives2021 Immersive and 3D Audio: from Architecture to Automotive (I3DA)10.1109/I3DA48870.2021.9610839(1-10)Online publication date: 8-Sep-2021
https://doi.org/10.1109/I3DA48870.2021.9610839
Hirankerd KKittisunthonphisarn N(2020)E-Learning Management System Based on Reality Technology with AIInternational Journal of Information and Education Technology10.18178/ijiet.2020.10.4.137310:4(259-264)Online publication date: 2020
https://doi.org/10.18178/ijiet.2020.10.4.1373
Çamcı AGroß-Vogt KHöldrich R(2020)Teaching immersive media at the "dawn of the new everything"Proceedings of the 15th International Audio Mostly Conference10.1145/3411109.3411121(229-232)Online publication date: 15-Sep-2020
https://dl.acm.org/doi/10.1145/3411109.3411121
Li HLau S(2020)A review of audio-visual interaction on soundscape assessment in urban built environmentsApplied Acoustics10.1016/j.apacoust.2020.107372166(107372)Online publication date: Sep-2020
https://doi.org/10.1016/j.apacoust.2020.107372
Speicher MHall BNebeling MBrewster SFitzpatrick GCox AKostakos V(2019)What is Mixed Reality?Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300767(1-15)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300767
Çamcı AÇakmak CForbes A(2019)Applying Game Mechanics to Networked Music HCI SystemsMethionine Dependence of Cancer and Aging10.1007/978-3-319-92069-6_14(223-241)Online publication date: 7-Feb-2019
https://doi.org/10.1007/978-3-319-92069-6_14

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten