skip to main content
10.1145/3384657.3384792acmotherconferencesArticle/Chapter ViewAbstractPublication PagesahsConference Proceedingsconference-collections
research-article

Manipulatable Auditory Perception in Wearable Computing

Published: 06 June 2020 Publication History

Abstract

We proposed a framework to manipulate auditory perception. Since auditory perception is passive sense, we often do not notice important information and acquire unimportant information. In this study, we focused on earphone-type wearable computers (hearable devices) that not only have speakers but also microphones. In a hearable computing environment, we always attach microphones and speakers to the ears. Therefore, we can manipulate our auditory perception using a hearable device. We manipulated the frequency of the input sound from the microphones and transmitted the converted sound from the speakers. Thus, we could acquire the sound that is not heard with our normal auditory perception and eliminate the unwanted sound according to the user's requirements. We devised five types of frequency-manipulating techniques and implemented a prototype device. Moreover, we proposed seven assumed applications that can be realized by the proposed method.

References

[1]
3DSignals. 2019. Sound-based Predictive Maintenance. https://www.3dsig.com/.
[2]
Robert Albrecht, Tapio Lokki, and Lauri Savioja. 2011. A Mobile Augmented Reality Audio System with Binaural Microphones. In Proceedings of Interacting with Sound Workshop: Exploring Context-Aware, Local and Social Audio Applications (IwS '11). Association for Computing Machinery, New York, NY, USA, 7--11. https://doi.org/10.1145/2019335.2019337
[3]
Joshua Alexander. 2016. Hearing Aid Delay and Current Drain in Modern Digital Devices. Canadian Audiologist 3, 4 (2016).
[4]
Takashi Amesaka, Hiroki Watanabe, and Masanori Sugimoto. 2019. Facial Expression Recognition Using Ear Canal Transfer Function. In Proceedings of the 23rd International Symposium on Wearable Computers (ISWC '19). ACM, New York, NY, USA, 1--9. https://doi.org/10.1145/3341163.3347747
[5]
Toshiyuki Ando, Yuki Kubo, Buntarou Shizuki, and Shin Takahashi. 2017. CanalSense: Face-Related Movement Recognition System Based on Sensing Air Pressure in Ear Canals. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (UIST '17). ACM, New York, NY, USA, 679--689. https://doi.org/10.1145/3126594.3126649
[6]
Takayuki Arakawa, Takafumi Koshinaka, Shohei Yano, Hideki Irisawa, Ryoji Miyahara, and Hitoshi Imaoka. 2016. Fast and accurate personal authentication using ear acoustics. In Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA). 1--4. https://doi.org/10.1109/APSIPA.2016.7820886
[7]
Jérôme Ardouin, Anatole Lécuyer, Maud Marchal, Clément Riant, and Eric Marchand. 2012. FlyVIZ: A Novel Display Device to Provide Humans with 360 Vision by Coupling Catadioptric Camera with Hmd. In Proceedings of the 18th ACM Symposium on Virtual Reality Software and Technology (VRST '12). ACM, New York, NY, USA, 41--44. https://doi.org/10.1145/2407336.2407344
[8]
Halim Cagri Ates, Alexander Fiannaca, and Eelke Folmer. 2015. Immersive Simulation of Visual Impairments Using a Wearable See-through Display. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '15). ACM, New York, NY, USA, 225--228. https://doi.org/10.1145/2677199.2680551
[9]
Abdelkareem Bedri, David Byrd, Peter Presti, Himanshu Sahni, Zehua Gue, and Thad Starner. 2015. Stick It in Your Ear: Building an In-ear Jaw Movement Sensor. In Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers (UbiComp/ISWC'15 Adjunct). ACM, New York, NY, USA, 1333--1338. https://doi.org/10.1145/2800835.2807933
[10]
Abdelkareem Bedri, Apoorva Verlekar, Edison Thomaz, Valerie Avva, and Thad Starner. 2015. Detecting Mastication: A Wearable Approach. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (ICMI '15). ACM, New York, NY, USA, 247--250. https://doi.org/10.1145/2818346.2820767
[11]
Jens Blauert. 1996. Spatial Hearing: The Psychophysics of Human Sound Localization. MIT Press.
[12]
Bose. 2019. Bose Hearphones. https://www.bose.com/en_us/products/wellness/conversation_enhancing_headphones/hearphones.html.
[13]
Pettersson Elektronik. 2019. Bat detectors and sound analysis software. http://www.batsound.com/.
[14]
Emerson. 2019. About Ultrasonic Gas Leak Detectors. https://www.emerson.com/en-sg/automation/measurement-instrumentation/flame-gas-detection/about-ultrasonic-gas-leak-detectors.
[15]
Kevin Fan, Jochen Huber, Suranga Nanayakkara, and Masahiko Inami. 2014. SpiderVision: Extending the Human Field of View for Augmented Awareness. In Proceedings of the 5th Augmented Human International Conference (AH '14). ACM, New York, NY, USA, Article 49, 8 pages. https://doi.org/10.1145/2582051.2582100
[16]
Mark D. Fletcher, Sian Lloyd Jones, Paul R. White, Craig N. Dolder, Benjamin Lineton, and Timothy G. Leighton. 2018. Public exposure to ultrasound and very high-frequency sound in air. The Journal of the Acoustical Society of America 144, 4 (2018), 2554--2564. https://doi.org/10.1121/1.5063817 arXiv:https://doi.org/10.1121/1.5063817
[17]
Hiromitsu Fujii, Atsushi Yamashita, and Hajime Asama. 2016. Defect Detection with Estimation of Material Condition using Ensemble Learning for Hammering Test. In Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA '16). 3847--3854.
[18]
Francisco Kiss, Romina Poguntke, Albrecht Schmidt, and Pawel W. Woundefinedniak. 2018. S5: Selective Sensing of Single Sound Sources. In Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI '18). Association for Computing Machinery, New York, NY, USA, 39--46. https://doi.org/10.1145/3236112.3236118
[19]
Doppler Labs. 2019. Here One. https://hereplus.me/products/here-one/.
[20]
Tobias Langlotz, Jonathan Sutton, Stefanie Zollmann, Yuta Itoh, and Holger Regenbrecht. 2018. ChromaGlasses: Computational Glasses for Compensating Colour Blindness. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 390, 12 pages. https://doi.org/10.1145/3173574.3173964
[21]
Gierad Laput, Xiang 'Anthony' Chen, and Chris Harrison. 2016. SweepSense: Ad Hoc Configuration Sensing Using Reflected Swept-Frequency Ultrasonics. In Proceedings of the 21st International Conference on Intelligent User Interfaces (IUI '16). ACM, New York, NY, USA, 332--335. https://doi.org/10.1145/2856767.2856812
[22]
Jean Laroche and Mark Dolson. 1999. New phase-vocoder techniques for pitch-shifting, harmonizing and other exotic effects. In Proceedings of the 1999 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. WASPAA '99 (Cat. No.99TH8452). 91--94. https://doi.org/10.1109/ASPAA.1999.810857
[23]
Hiroyuki Manabe, Masaaki Fukumoto, and Tohru Yagi. 2013. Conductive Rubber Electrodes for Earphone-based Eye Gesture Input Interface. In Proceedings of the 2013 International Symposium on Wearable Computers (ISWC '13). ACM, New York, NY, USA, 33--40. https://doi.org/10.1145/2493988.2494329
[24]
Jörg Müller, Matthias Geier, Christina Dicke, and Sascha Spors. 2014. The boom-Room: Mid-air Direct Interaction with Virtual Sound Sources. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 247--256. https://doi.org/10.1145/2556288.2557000
[25]
NEC. 2019. Human-Oriented IoT Solutions Using Hearable Technology from NEC. https://www.nec.com/en/global/techrep/journal/g17/n01/170110.html.
[26]
Oticon. 2019. Inium Sense - Speech Rescue™ technology. https://www.oticon.com/professionals/brainhearing-technology/brainhearing-technology/speech-rescue.
[27]
ReSound. 2019. Sound Shaper - Newest hearing aids tailored to your individual hearing environment and preference. http://www.gnhearing.dk/en/professionals/technology-and-innovation/surround-sound-by-resound/sound-shaper.
[28]
Nirupam Roy, Haitham Hassanieh, and Romit Roy Choudhury. 2017. BackDoor: Making Microphones Hear Inaudible Sounds. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys '17). ACM, New York, NY, USA, 2--14. https://doi.org/10.1145/3081333.3081366
[29]
Eldon Schoop, James Smith, and Bjoern Hartmann. 2018. HindSight: Enhancing Spatial Awareness by Sonifying Detected Objects in Real-Time 360-Degree Video. InProceedingsofthe 2018CHIConference on Human Factorsin ComputingSystems (CHI '18). ACM, New York, NY, USA, Article 143, 12 pages. https://doi.org/10.1145/3173574.3173717
[30]
Kenichiro Shirota, Roshan Lalintha Peiris, and Kouta Minamizawa. 2019. Altered Pinna: Exploring Shape Change of Pinna for Perception and Illusion of Sound Direction Change. In Proceedings of the 23rd International Symposium on Wearable Computers (ISWC '19). Association for Computing Machinery, New York, NY, USA, 220--224. https://doi.org/10.1145/3341163.3347725
[31]
Jascha Sohl-Dickstein, Santani Teng, Benjamin M Gaub, Chris C Rodgers, Crystal Li, Michael R DeWeese, and Nicol S Harper. 2015. A device for human ultrasonic echolocation. IEEE Transactions on Biomedical Engineering 62, 6 (2015), 1526--1534.
[32]
Dominik Storek, Jan Stuchlik, and Frantisek Rund. 2015. Modifications of the surrounding auditory space by augmented reality audio: Introduction to warped acoustic reality. In International Conference on Auditory Display (ICAD '15). 225--230.
[33]
Miwa Sumiya, Kaoru Ashihara, Kazuki Yoshino, Masaki Gogami, Yoshiki Nagatani, Kohta I Kobayasi, Yoshiaki Watanabe, and Shizuko Hiryu. 2019. Batinspired signal design for target discrimination in human echolocation. The JournaloftheAcoustical SocietyofAmerica 145, 4 (2019), 2221--2236.
[34]
Ana Tajadura-Jiménez, Maria Basia, Ophelia Deroy, Merle Fairhurst, Nicolai Marquardt, and Nadia Bianchi-Berthouze. 2015. As Light As Your Footsteps: Altering Walking Sounds to Change Perceived Body Weight, Emotional State and Gait. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 2943--2952. https://doi.org/10.1145/2702123.2702374
[35]
Ana Tajadura-Jiménez, Joseph Newbold, Linge Zhang, Patricia Rick, and Nadia Bianchi-Berthouze. 2019. As Light As You Aspire to Be: Changing Body Perception with Sound to Support Physical Activity. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, New York, NY, USA, Article 658, 14 pages. https://doi.org/10.1145/3290605.3300888
[36]
Masaru Takagi, Kosuke Fujimoto, Yoshihiro Kawahara, and Tohru Asami. 2014. Detecting Hybrid and Electric Vehicles Using a Smartphone. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '14). ACM, New York, NY, USA, 267--275. https://doi.org/10.1145/2632048.2632088
[37]
Enrico Tanuwidjaja, Derek Huynh, Kirsten Koa, Calvin Nguyen, Churen Shao, Patrick Torbett, Colleen Emmenegger, and Nadir Weibel. 2014. Chroma: A Wearable Augmented-reality Solution for Color Blindness. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '14). ACM, New York, NY, USA, 799--810. https://doi.org/10.1145/2632048.2632091
[38]
Shunsuke Uchibori, Yuki Sarumaru, Kaoru Ashihara, Tetsuo Ohta, and Shizuko Hiryu. 2015. Experimental evaluation of binaural recording system using a miniature dummy head. Acoustical Science and Technology 36, 1 (2015), 42--45. https://doi.org/10.1250/ast.36.42
[39]
International Telecommunication Union. 2019. P.800: Methods for subjective determination of transmission quality. https://www.itu.int/rec/T-REC-P.800-199608-I/en.
[40]
Marian Weger, Thomas Hermann, and Robert Höldrich. 2019. Real-time auditory contrast enhancement. In International Conference on Auditory Display (ICAD '19). Georgia Institute of Technology, 254--261.
[41]
Atsushi Yamashita, Takahiro Hara, and Toru Kaneko. 2006. Inspection of Visible and Invisible Features of Objects with Image and Sound Signal Processing. In Proceedings of the 2006 IEEE International Conference on Intelligent Robots and Systems. 3837--3842.
[42]
Jing Yang, Yves Frank, and Gábor Sörös. 2019. Hearing Is Believing: Synthesizing Spatial Audio from Everyday Objects to Users. In Proceedings of the 10th Augmented Human International Conference 2019 (AH2019). ACM, New York, NY, USA, Article 28, 9 pages. https://doi.org/10.1145/3311823.3311872
[43]
Guoming Zhang, Chen Yan, Xiaoyu Ji, Tianchen Zhang, Taimin Zhang, and Wenyuan Xu. 2017. DolphinAttack: Inaudible Voice Commands. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS '17). ACM, New York, NY, USA, 103--117. https://doi.org/10.1145/3133956.3134052
[44]
Yuhang Zhao, Sarit Szpiro, Jonathan Knighten, and Shiri Azenkot. 2016. CueSee: Exploring Visual Cues for People with Low Vision to Facilitate a Visual Search Task. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '16). ACM, New York, NY, USA, 73--84. https://doi.org/10.1145/2971648.2971730

Cited By

View all
  • (2024)Everyday Life Challenges and Augmented Realities: Exploring Use Cases For, and User Perspectives on, an Augmented Everyday LifeProceedings of the Augmented Humans International Conference 202410.1145/3652920.3652921(52-62)Online publication date: 4-Apr-2024
  • (2024)Investigation of Frequency-Selective Loudness Reduction and Its Recovery Method in HearablesIEEE Access10.1109/ACCESS.2024.338512312(49916-49926)Online publication date: 2024
  • (2023)Transparency Mode of Hearable Reduces Your Spatial Hearing: Evaluation and Cancelling Method to Restore Spatial HearingIEEE Access10.1109/ACCESS.2023.331271311(97952-97960)Online publication date: 2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
AHs '20: Proceedings of the Augmented Humans International Conference
March 2020
296 pages
ISBN:9781450376037
DOI:10.1145/3384657
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 June 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Wearable computing
  2. auditory perception
  3. hearable device
  4. ultrasound

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

AHs '20
AHs '20: Augmented Humans International Conference
March 16 - 17, 2020
Kaiserslautern, Germany

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)23
  • Downloads (Last 6 weeks)2
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Everyday Life Challenges and Augmented Realities: Exploring Use Cases For, and User Perspectives on, an Augmented Everyday LifeProceedings of the Augmented Humans International Conference 202410.1145/3652920.3652921(52-62)Online publication date: 4-Apr-2024
  • (2024)Investigation of Frequency-Selective Loudness Reduction and Its Recovery Method in HearablesIEEE Access10.1109/ACCESS.2024.338512312(49916-49926)Online publication date: 2024
  • (2023)Transparency Mode of Hearable Reduces Your Spatial Hearing: Evaluation and Cancelling Method to Restore Spatial HearingIEEE Access10.1109/ACCESS.2023.331271311(97952-97960)Online publication date: 2023
  • (2022)Otonona: a prototype for experiencing absolute pitch using multimodal augmented reality2022 IEEE 11th Global Conference on Consumer Electronics (GCCE)10.1109/GCCE56475.2022.10014312(453-456)Online publication date: 18-Oct-2022
  • (2022)Human-Machine Cooperative Echolocation Using UltrasoundIEEE Access10.1109/ACCESS.2022.322446810(125264-125278)Online publication date: 2022
  • (2021)MIXIQProceedings of the 27th Annual International Conference on Mobile Computing and Networking10.1145/3447993.3483270(364-377)Online publication date: 25-Oct-2021

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media