skip to main content
10.1145/3341162.3349302acmconferencesArticle/Chapter ViewAbstractPublication PagesubicompConference Proceedingsconference-collections
research-article

Audio augmented reality for human-object interactions

Published: 09 September 2019 Publication History

Abstract

In the past, augmented reality (AR) research focused mostly on visual augmentation, which requires a visual rendering device like head-mounted displays that are usually obtrusive, expensive, and socially unaccepted. In contrast, wearable audio headsets are already popularized and the auditory sense also plays an important role in everyday interactions with the environment. In this PhD project, we explore audio augmented reality (AAR) that augments objects with 3D sounds, which are spatialized virtually but are perceived as originating from real locations in the space. We intend to design, implement, and evaluate such AAR systems that enhance people's intuitive and immersive interactions with objects in various consumer and industrial scenarios. By exploring AAR using pervasive and wearable devices, we hope to contribute to the vision of ubiquitous AR.

References

[1]
Robert Albrecht, Riitta Väänänen, and Tapio Lokki. (2016). Guided by Music: Pedestrian and Cyclist Navigation with Route and Beacon Guidance. Personal and Ubiquitous Computing 20, 20:1, 20(1), 121--145.
[2]
Youssef El Baba, Andreas Walther, and Emanuel A. P. Habets. 2018. 3D Room Geometry Inference Based on Room Impulse Response Stacks. IEEE/ACM Transactions on Audio, Speech, and Language Processing 26, 5 (2018), 857--872.
[3]
Amit Barde, Matt Ward, William S Helton, Mark Billinghurst, and Gun Lee. 2016. Attention Redirection Using Binaurally Spatialised Cues Delivered Over a Bone Conduction Headset. In Human Factors and Ergonomics Society Annual Meeting (2016). SAGE Publications, 1534--1538.
[4]
Jens Blauert. 1997. Spatial Hearing: The Psychophysics of Human Sound Localization. MIT press.
[5]
Simon Blessenohl, Cecily Morrison, Antonio Criminisi, and Jamie Shotton. 2015. Improving Indoor Mobility of The Visually Impaired with Depth-based Spatial Sound. In IEEE International Conference on Computer Vision Workshops (ICCV'15). IEEE, 26--34.
[6]
Cesar Cadena, Luca Carlone, Henry Carrillo, Yasir Latif, Davide Scaramuzza, José Neira, Ian Reid, and John J Leonard. (2016). Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age. IEEE Transactions on Robotics 32, 6, 32(6), 1309--1332.
[7]
Alessio Del Bue and Marco Crocco. 2016. 3D room Reconstruction with Sound. https://vgm.iit.it/tutorials/3d-room-reconstruction-with-sound
[8]
Arindam Dey, Mark Billinghurst, Robert W Lindeman, and J Swan. 2018. A Systematic Review of 10 Years of Augmented Reality Usability Studies: 2005 to 2014. Frontiers in Robotics and AI 5 (2018), 37.
[9]
Florian Heller and Jan Borchers. 2014. AudioTorch: Using A Smartphone as Directional Microphone in Virtual Audio Spaces. In The 16th ACM International Conference on Human-Computer Interaction with Mobile Devices & Services (MobileHCI'14). ACM.
[10]
Florian Heller and Johannes Schöning. 2018. NavigaTone: Seamlessly Embedding Navigation Cues in Mobile Music Listening. In The ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'18). ACM.
[11]
Kangsoo Kim, Mark Billinghurst, Gerd Bruder, Henry Been-Lirn Duh, and Gregory F Welch. 2018. Revisiting Trends in Augmented Reality Research: A Review of the 2nd Decade of ISMAR (2008--2017). IEEE Transactions on Visualization and Computer Graphics (2018).
[12]
Kent Lyons, Maribeth Gandy, and Thad Starner. 2000. Guided by Voices: An Audio Augmented Reality System. In Conference on Auditory Display (ICAD'00).
[13]
Jörg Müller, Matthias Geier, Christina Dicke, and Sascha Spors. 2014. The BoomRoom: Mid-air Direct Interaction with Virtual Sound Sources. In The ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'14). ACM, 247--256.
[14]
Mirco Rossi, Julia Seiter, Oliver Amft, Seraina Buchmeier, and Gerhard Tröster. 2013. RoomSense. In Proceedings of the 4th Augmented Human International Conference, Albrecht Schmidt (Ed.). ACM, New York, NY, 89--95.
[15]
Spencer Russell, Gershon Dublon, and Joseph A Paradiso. 2016. HearThere: Networked Sensory ProsThetics through Auditory Augmented Reality. In The 7th Augmented Human International Conference (AH'16). ACM, 20.
[16]
Carl Schissler, Christian Loftin, and Dinesh Manocha. (2018). Acoustic Classification and Optimization for Multi-modal Rendering of Real-World Scenes. IEEE Transactions on Visualization and Computer Graphics 24, 3, 24(3), 1246--1259.
[17]
Carl Schissler and Dinesh Manocha. 2017. Interactive sound propagation and rendering for large multi-source scenes. ACM Transactions on Graphics 36, 1 (2017), 2.
[18]
Eldon Schoop, James Smith, and Bjoern Hartmann. 2018. HindSight: Enhancing Spatial Awareness by Sonifying Detected Objects in Real-Time 360-Degree Video. In ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'18). ACM, 143.
[19]
Jaka Sodnik, Saso Tomazic, Raphael Grasset, Andreas Duenser, and Mark Billinghurst. 2006. Spatial sound localization in an augmented reality environment. In Proceedings of the 18th Australia conference on computer-human interaction: design: activities, artefacts and environments. 111--118.
[20]
Titus JJ Tang and Wai Ho Li. 2014. An Assistive Eyewear Prototype That Interactively Converts 3D Object Locations into Spatial Audio. In The 2014 ACM International Symposium on Wearable Computers (ISWC'14). ACM, 119--126.
[21]
Jing Yang, Yves Frank, and Gábor Sörös. 2019. Hearing Is Believing: Synthesizing Spatial Audio from Everyday Objects to Users. In The 10th Augmented Human International Conference (AH'19). ACM.
[22]
Jing Yang and Gábor Sörös. 2018. Augmenting Smart Object Interactions with Smart Audio. In AH'18 Proceedings of the 9th Augmented Human International Conference. ACM, ACM, Seoul, Republic of Korea, 28:1 -- 28:3.

Cited By

View all
  • (2024)Hicclip: Sonification of Augmented Eating Sounds to Intervene Snacking BehaviorsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661532(1372-1384)Online publication date: 1-Jul-2024
  • (2024)MIMOSA: Human-AI Co-Creation of Computational Spatial Audio Effects on VideosProceedings of the 16th Conference on Creativity & Cognition10.1145/3635636.3656189(156-169)Online publication date: 23-Jun-2024
  • (2024)Audio augmented reality using sonification to enhance visual art experiencesInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103329191:COnline publication date: 1-Nov-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
UbiComp/ISWC '19 Adjunct: Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers
September 2019
1234 pages
ISBN:9781450368698
DOI:10.1145/3341162
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 September 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. audio augmented reality
  2. human-object interaction
  3. spatial audio
  4. wearable

Qualifiers

  • Research-article

Conference

UbiComp '19

Acceptance Rates

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)77
  • Downloads (Last 6 weeks)4
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Hicclip: Sonification of Augmented Eating Sounds to Intervene Snacking BehaviorsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661532(1372-1384)Online publication date: 1-Jul-2024
  • (2024)MIMOSA: Human-AI Co-Creation of Computational Spatial Audio Effects on VideosProceedings of the 16th Conference on Creativity & Cognition10.1145/3635636.3656189(156-169)Online publication date: 23-Jun-2024
  • (2024)Audio augmented reality using sonification to enhance visual art experiencesInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103329191:COnline publication date: 1-Nov-2024
  • (2024)Taxonomy and definition of audio augmented reality (AAR)International Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103179182:COnline publication date: 1-Feb-2024
  • (2024)Workplace Aspects of Knowledge and Expertise Sharing Practices Supported by Augmented Reality Systems: Findings from a Design Case StudyComputer Supported Cooperative Work (CSCW)10.1007/s10606-024-09508-8Online publication date: 17-Dec-2024
  • (2022)Characterising Soundscape Research in Human-Computer InteractionProceedings of the 2022 ACM Designing Interactive Systems Conference10.1145/3532106.3533458(1394-1417)Online publication date: 13-Jun-2022
  • (2022)Deceiving Audio Design in Augmented Environments : A Systematic Review of Audio Effects in Augmented Reality2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct57072.2022.00018(36-43)Online publication date: Oct-2022
  • (2021)Rapid prototyping approach for audio augmented reality user experiencesACM SIGGRAPH 2021 Labs10.1145/3450616.3464981(1-2)Online publication date: 31-Jul-2021
  • (2021)Multi-stream Network for Human-object Interaction DetectionInternational Journal of Pattern Recognition and Artificial Intelligence10.1142/S021800142150025735:08(2150025)Online publication date: 12-Mar-2021
  • (2021)Lifelogging meets alternate and cross-realities: an investigation into broadcasting personal visual realities to remote audiencesMultimedia Tools and Applications10.1007/s11042-021-11310-383:15(46707-46730)Online publication date: 14-Aug-2021
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media