Work in Progress

VisaudiBlow: Fine-grained Blowing Interaction Based on Visual and Auditory Detection

Authors:

Guihuan FengAuthors Info & Claims

CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems

Article No.: 311, Pages 1 - 6

https://doi.org/10.1145/3544549.3585829

Published: 19 April 2023 Publication History

Abstract

More and more studies regard blowing as a means of reflecting the user’s physiological conditions. Its potential as an independent input channel has not been fully explored. In this paper, we analyze the design space of blowing as an independent interactive channel and propose a novel blowing recognition approach, VisaudiBlow, based on visual and audio detection. VisaudiBlow uses Facial Tracker to capture the mouth expression when doing specific blowing actions, and audio information generated by blowing is obtained by a microphone beside the mouth, enabling identification and detection of the input parameters such as blowing type, direction, and intensity. In order to verify the validity of blowing as an independent input channel, a prototype system BlowUI is implemented. The experimental results demonstrate that blow interaction can effectively complete the menu control tasks and achieve the same interaction efficiency as controllers in specific scenarios.

Supplementary Material

MP4 File (3544549.3585829-video-preview.mp4)

Video Preview

Download
24.26 MB

MP4 File (3544549.3585829-talk-video.mp4)

Pre-recorded Video Presentation

Download
93.27 MB

References

[1]

Dan Adler, Bruno Herbelin, Thomas Similowski, and Olaf Blanke. 2014. Breathing and sense of self: Visuo–respiratory conflicts alter body self-consciousness. Respiratory Physiology & Neurobiology 203 (2014), 68–74. https://doi.org/10.1016/j.resp.2014.08.003

[2]

Phil Corbishley and Esther Rodriguez-Villegas. 2008. Breathing Detection: Towards a Miniaturized, Wearable, Battery-Operated Monitoring System. IEEE Transactions on Biomedical Engineering 55, 1 (2008), 196–204. https://doi.org/10.1109/TBME.2007.910679

[3]

John Desnoyers-Stewart, Ekaterina R. Stepanova, Philippe Pasquier, and Bernhard E. Riecke. 2019. JeL: Connecting Through Breath in Virtual Reality. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA ’19). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3290607.3312845

Digital Library

[4]

Sevinc Eroglu, Sascha Gebhardt, Patric Schmitz, Dominik Rausch, and Torsten Wolfgang Kuhlen. 2018. Fluid Sketching―Immersive Sketching Based on Fluid Flow. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 475–482. https://doi.org/10.1109/VR.2018.8446595

[5]

E.F. Greneker. 1997. Radar sensing of heartbeat and respiration at a distance with applications of the technology. In Radar 97 (Conf. Publ. No. 449). 150–154. https://doi.org/10.1049/cp:19971650

[6]

Alberto Hernando, Hugo Posada-Quintero, María Dolores Peláez-Coca, Eduardo Gil, and Ki H. Chon. 2022. Autonomic Nervous System characterization in hyperbaric environments considering respiratory component and non-linear analysis of Heart Rate Variability. Computer Methods and Programs in Biomedicine 214 (2022), 106527. https://doi.org/10.1016/j.cmpb.2021.106527

Digital Library

[7]

Aizhan Issatayeva, Aidana Beisenova, Daniele Tosi, and Carlo Molardi. 2020. Fiber-Optic Based Smart Textiles for Real-Time Monitoring of Breathing Rate. Sensors 20, 12 (2020). https://doi.org/10.3390/s20123408

[8]

Jong-Hyun Kim and Jung Lee. 2020. Controlling your contents with the breath: Interactive breath interface for VR, games, and animations. PloS one 15, 10 (2020), e0241498. https://doi.org/10.1371/journal.pone.0241498

[9]

Martijn J.L. Kors, Gabriele Ferri, Erik D. van der Spek, Cas Ketel, and Ben A.M. Schouten. 2016. A Breathtaking Journey. On the Design of an Empathy-Arousing Mixed-Reality Game. In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play (Austin, Texas, USA) (CHI PLAY ’16). Association for Computing Machinery, New York, NY, USA, 91–104. https://doi.org/10.1145/2967934.2968110

Digital Library

[10]

D Kranzlmuller, Bernhard Reitinger, Ingo Hackl, and Jens Volkert. 2001. Voice controlled virtual reality and its perspectives for everyday life. Itg Fachbericht (2001), 101–108.

[11]

Amit Krishan Kumar, M. Ritam, Lina Han, Shuli Guo, and Rohitash Chandra. 2022. Deep learning for predicting respiratory rate from biosignals. Computers in Biology and Medicine 144 (2022), 105338. https://doi.org/10.1016/j.compbiomed.2022.105338

Digital Library

[12]

Koichi Kuzume. 2010. Input Device for Disabled Persons Using Expiration and Tooth-Touch Sound Signals. In Proceedings of the 2010 ACM Symposium on Applied Computing (Sierre, Switzerland) (SAC ’10). Association for Computing Machinery, New York, NY, USA, 1159–1164. https://doi.org/10.1145/1774088.1774335

Digital Library

[13]

Yu-Ching Lee, Abdan Syakura, Muhammad Adil Khalil, Ching-Ho Wu, Yi-Fang Ding, and Ching-Wei Wang. 2021. A real-time camera-based adaptive breathing monitoring system. Medical & biological engineering & computing 59, 6 (June 2021), 1285—1298. https://doi.org/10.1007/s11517-021-02371-5

[14]

Fan Li, Ching-Hung Lee, Shanshan Feng, Amy Trappey, and Fazal Gilani. 2021. Prospective on Eye-Tracking-based Studies in Immersive Virtual Reality. In 2021 IEEE 24th International Conference on Computer Supported Cooperative Work in Design (CSCWD). 861–866. https://doi.org/10.1109/CSCWD49262.2021.9437692

[15]

Joe Marshall, Duncan Rowland, Stefan Rennick Egglestone, Steve Benford, Brendan Walker, and Derek McAuley. 2011. Breath Control of Amusement Rides. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI ’11). Association for Computing Machinery, New York, NY, USA, 73–82. https://doi.org/10.1145/1978942.1978955

Digital Library

[16]

Shwetak N. Patel and Gregory D. Abowd. 2007. Blui: Low-Cost Localized Blowable User Interfaces. In Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology (Newport, Rhode Island, USA) (UIST ’07). Association for Computing Machinery, New York, NY, USA, 217–220. https://doi.org/10.1145/1294211.1294250

Digital Library

[17]

Rakesh Patibanda, Florian ’Floyd’ Mueller, Matevz Leskovsek, and Jonathan Duckworth. 2017. Life Tree: Understanding the Design of Breathing Exercise Games. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (Amsterdam, The Netherlands) (CHI PLAY ’17). Association for Computing Machinery, New York, NY, USA, 19–31. https://doi.org/10.1145/3116595.3116621

Digital Library

[18]

Sudha M R, Sriraghav K, Sudar Abisheck S, Shomona Gracia Jacob, and Manisha S. 2017. Approaches and Applications of Virtual Reality and Gesture Recognition: A Review. Int. J. Ambient Comput. Intell. 8, 4 (oct 2017), 1–18. https://doi.org/10.4018/IJACI.2017100101

Digital Library

[19]

Natasha Anne Rappa, Susan Ledger, Timothy Teo, Kok Wai Wong, Brad Power, and Bruce Hilliard. 2019. The use of eye tracking technology to explore learning and performance within virtual reality and mixed reality settings: a scoping review. Interactive Learning Environments (2019), 1–13.

[20]

Yanzhi Ren, Zhourong Zheng, Hongbo Liu, Yingying Chen, Hongwei Li, and Chen Wang. 2021. Breathing Sound-based Exercise Intensity Monitoring via Smartphones. In 2021 International Conference on Computer Communications and Networks (ICCCN). 1–10. https://doi.org/10.1109/ICCCN52240.2021.9522176

[21]

Joan Sol Roo, Renaud Gervais, Jeremy Frey, and Martin Hachet. 2017. Inner Garden: Connecting Inner States to a Mixed Reality Sandbox for Mindfulness. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 1459–1470. https://doi.org/10.1145/3025453.3025743

Digital Library

[22]

Dangdang Shao, Yuting Yang, Chenbin Liu, Francis Tsow, Hui Yu, and Nongjian Tao. 2014. Noncontact Monitoring Breathing Pattern, Exhalation Flow Rate and Pulse Transit Time. IEEE Transactions on Biomedical Engineering 61, 11 (2014), 2760–2767. https://doi.org/10.1109/TBME.2014.2327024

[23]

Florian Soyka, Markus Leyrer, Joe Smallwood, Chris Ferguson, Bernhard E. Riecke, and Betty J. Mohler. 2016. Enhancing Stress Management Techniques Using Virtual Reality. In Proceedings of the ACM Symposium on Applied Perception (Anaheim, California) (SAP ’16). Association for Computing Machinery, New York, NY, USA, 85–88. https://doi.org/10.1145/2931002.2931017

Digital Library

[24]

Misha Sra, Xuhai Xu, and Pattie Maes. 2018. BreathVR: Leveraging Breathing as a Directly Controlled Interface for Virtual Reality Games. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3173914

Digital Library

[25]

Markus Tatzgern, Michael Domhardt, Martin Wolf, Michael Cenger, Gerlinde Emsenhuber, Radomir Dinic, Nathalie Gerner, and Arnulf Hartl. 2022. AirRes Mask: A Precise and Robust Virtual Reality Breathing Interface Utilizing Breathing Resistance as Output Modality. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 274, 14 pages. https://doi.org/10.1145/3491102.3502090

Digital Library

[26]

Marieke van Rooij, Adam Lobel, Owen Harris, Niki Smit, and Isabela Granic. 2016. DEEP: A Biofeedback Virtual Reality Game for Children At-Risk for Anxiety(CHI EA ’16). Association for Computing Machinery, New York, NY, USA, 1989–1997. https://doi.org/10.1145/2851581.2892452

Digital Library

[27]

Daniel Zielasko, Neha Neha, Benjamin Weyers, and Torsten W. Kuhlen. 2017. A reliable non-verbal vocal input metaphor for clicking. In 2017 IEEE Symposium on 3D User Interfaces (3DUI). 40–49. https://doi.org/10.1109/3DUI.2017.7893316

[28]

Daniel Zielasko, Dominik Rausch, Yuen C. Law, Thomas C. Knott, Sebastian Pick, Sven Porsche, Joachim Herber, Johannes Hummel, and Torsten W. Kuhlen. 2015. Cirque des bouteilles: The art of blowing on bottles. In 2015 IEEE Symposium on 3D User Interfaces (3DUI). 209–210. https://doi.org/10.1109/3DUI.2015.7131774

Cited By

Zielasko DJaquez Lora J(2024)Ubiquitous BlowClick: Non-verbal Vocal Input for Confirmation with Hand-held Mobile Devices in the FieldProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682101(1-8)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682101
Brooks JMazursky AHixon JLopes P(2024)Augmented Breathing via Thermal Feedback in the NoseProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676438(1-11)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676438
Min TXia CYamamoto TSugiura Y(2023)Seeing the Wind: An Interactive Mist Interface for Airflow InputProceedings of the ACM on Human-Computer Interaction10.1145/36264807:ISS(398-419)Online publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1145/3626480

Index Terms

VisaudiBlow: Fine-grained Blowing Interaction Based on Visual and Auditory Detection
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Virtual reality
  2. Interaction design
    1. Interaction design process and methods
      1. Interface design prototyping

Recommendations

Fine-grained warm water cooling for improving datacenter economy
ISCA '19: Proceedings of the 46th International Symposium on Computer Architecture

Driven by the increasing power consumption of datacenters, the industry is focusing more on water cooling for improving the energy efficiency. Using warm water to cool servers has been considered as an efficient method to reduce the cooling energy. ...
Fine-grained face verification

As performance on some aspects of the Labeled Faces in the Wild (LFW) benchmark approaches 100% accuracy, there is an intense debate on whether unconstrained face verification problem has already been solved. In this paper, we study a new face ...
CFD study of using a tube insert and an air blowing system to prevent tube failure in the Bisotoun power plant boiler
SMO'05: Proceedings of the 5th WSEAS international conference on Simulation, modelling and optimization

In this study attempts to find a theoretical solution to prevent tube explosion of the tubes placed in the superheater region of the Bisotoun Power Plant boiler, Iran. The boiler has three types of tube and the tube failure was occurred in the long ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems

April 2023

3914 pages

ISBN:9781450394222

DOI:10.1145/3544549

Editors:
Albrecht Schmidt
LMU Munich, Germany
,
Kaisa Väänänen
Tampere University, Finland
,
Tesh Goyal
Google Research, USA
,
Per Ola Kristensson
University of Cambridge, UK
,
Anicia Peters
University of Namibia, Namibia

Copyright © 2023 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 April 2023

Check for updates

Author Tags

Qualifiers

Work in progress
Research
Refereed limited

Conference

CHI '23

Sponsor:

SIGCHI

CHI '23: CHI Conference on Human Factors in Computing Systems

April 23 - 28, 2023

Hamburg, Germany

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
241
Total Downloads

Downloads (Last 12 months)134
Downloads (Last 6 weeks)31

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zielasko DJaquez Lora J(2024)Ubiquitous BlowClick: Non-verbal Vocal Input for Confirmation with Hand-held Mobile Devices in the FieldProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682101(1-8)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682101
Brooks JMazursky AHixon JLopes P(2024)Augmented Breathing via Thermal Feedback in the NoseProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676438(1-11)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676438
Min TXia CYamamoto TSugiura Y(2023)Seeing the Wind: An Interactive Mist Interface for Airflow InputProceedings of the ACM on Human-Computer Interaction10.1145/36264807:ISS(398-419)Online publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1145/3626480

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View full text|Download PDF

View Table of Conten