skip to main content
research-article

LoEar: Push the Range Limit of Acoustic Sensing for Vital Sign Monitoring

Published: 07 September 2022 Publication History

Abstract

Acoustic sensing has been explored in numerous applications leveraging the wide deployment of acoustic-enabled devices. However, most of the existing acoustic sensing systems work in a very short range only due to fast attenuation of ultrasonic signals, hindering their real-world deployment. In this paper, we present a novel acoustic sensing system using only a single microphone and speaker, named LoEar, to detect vital signs (respiration and heartbeat) with a significantly increased sensing range. We first develop a model, namely Carrierforming, to enhance the signal-to-noise ratio (SNR) via coherent superposition across multiple subcarriers on the target path. We then propose a novel technique called Continuous-MUSIC (Continuous-MUltiple SIgnal Classification) to detect a dynamic reflections, containing subtle motion, and further identify the target user based on the frequency distribution to enable Carrierforming. Finally, we adopt an adaptive Infinite Impulse Response (IIR) comb notch filter to recover the heartbeat pattern from the Channel Frequency Response (CFR) measurements which are dominated by respiration and further develop a peak-based scheme to estimate respiration rate and heart rate. We conduct extensive experiments to evaluate our system, and results show that our system outperforms the state-of-the-art using commercial devices, i.e., the range of respiration sensing is increased from 2 m to 7 m, and the range of heartbeat sensing is increased from 1.2 m to 6.5 m.

References

[1]
Chao Cai, Zhe Chen, Henglin Pu, Liyuan Ye, Menglan Hu, and Jun Luo. Acute: Acoustic thermometer empowered by a single smartphone. In Proceedings of ACM SenSys, 2020.
[2]
Rajalakshmi Nandakumar, Shyamnath Gollakota, and Nathaniel Watson. Contactless sleep apnea detection on smartphones. In Proceedings of ACM MobiSys, 2015.
[3]
Anran Wang, Jacob E Sunshine, and Shyamnath Gollakota. Contactless infant monitoring using white noise. In Proceedings of MobiCom, 2019.
[4]
Tianben Wang, Daqing Zhang, Yuanqing Zheng, Tao Gu, Xingshe Zhou, and Bernadette Dorizzi. C-fmcw based contactless respiration detection using acoustic signal. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 1(4):1--20, 2018.
[5]
Xingzhe Song, Boyuan Yang, Ge Yang, Ruirong Chen, Erick Forno, Wei Chen, and Wei Gao. Spirosonic: monitoring human lung function via acoustic sensing on commodity smartphones. In Proceedings of ACM MobiCom, 2020.
[6]
Kun Qian, Chenshu Wu, Fu Xiao, Yue Zheng, Yi Zhang, Zheng Yang, and Yunhao Liu. Acousticcardiogram: Monitoring heartbeats using acoustic signals on smart devices. In Proceedings of IEEE INFOCOM, 2018.
[7]
Fusang Zhang, Zhi Wang, Beihong Jin, Jie Xiong, and Daqing Zhang. Your smart speaker can" hear" your heartbeat! Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 4(4):1--24, 2020.
[8]
Youwei Zeng, Dan Wu, Jie Xiong, Enze Yi, Ruiyang Gao, and Daqing Zhang. Farsense: Pushing the range limit of wifi-based respiration sensing with csi ratio of two antennas. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 3(3):1--26, 2019.
[9]
Mingmin Zhao, Fadel Adib, and Dina Katabi. Emotion recognition using wireless signals. In Proceedings of ACM MobiCom, 2016.
[10]
Wenguang Mao, Mei Wang, Wei Sun, Lili Qiu, Swadhin Pradhan, and Yi-Chao Chen. Rnn-based room scale hand motion tracking. In Proceedings of ACM MobiCom, 2019.
[11]
Dong Li, Jialin Liu, Sunghoon Ivan Lee, and Jie Xiong. Fm-track: pushing the limits of contactless multi-target tracking using acoustic signals. In Proceedings of ACM SenSys, 2020.
[12]
Anup Agarwal, Mohit Jain, Pratyush Kumar, and Shwetak Patel. Opportunistic sensing with mic arrays on smart speakers for distal interaction and exercise tracking. In Proceedings of IEEE ICASSP, 2018.
[13]
Wenguang Mao, Wei Sun, Mei Wang, and Lili Qiu. Deeprange: Acoustic ranging via deep learning. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 4(4):1--23, 2020.
[14]
Rajalakshmi Nandakumar, Vikram Iyer, Desney Tan, and Shyamnath Gollakota. Fingerio: Using active sonar for fine-grained finger tracking. In Proceedings of ACM CHI, 2016.
[15]
Haoran Wan, Shuyu Shi, Wenyu Cao, Wei Wang, and Guihai Chen. Resptracker: Multi-user room-scale respiration tracking with commercial acoustic devices. In Proceedings of IEEE INFOCOM, 2021.
[16]
Lei Wang, Xiang Zhang, Yuanshuang Jiang, Yong Zhang, Chenren Xu, Ruiyang Gao, and Daqing Zhang. Watching your phone's back: Gesture recognition by sensing acoustical structure-borne propagation. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 5(2):1--26, 2021.
[17]
Wei Wang, Alex X. Liu, and Ke Sun. Device-free gesture tracking using acoustic signals. In Proceedings of ACM MobiCom, 2016.
[18]
Kanji Ono. A comprehensive report on ultrasonic attenuation of engineering materials, including metals, ceramics, polymers, fiber-reinforced composites, wood, and rocks. Applied Sciences, 10(7):2230, 2020.
[19]
Manikanta Kotaru, Kiran Joshi, Dinesh Bharadia, and Sachin Katti. Spotfi: Decimeter level localization using wifi. In Proceedings of ACM SIGCOMM, 2015.
[20]
Jie Xiong and Kyle Jamieson. Arraytrack: A fine-grained indoor location system. In Proceedings of USENIX NSDI, 2013.
[21]
Xiang Li, Shengjie Li, Daqing Zhang, Jie Xiong, Yasha Wang, and Hong Mei. Dynamic-music: Accurate device-free indoor localization. In Proceedings of ACM UbiComp, 2016.
[22]
C Lowanichkiattikul, M Dhanachai, C Sitathanee, S Khachonkham, and P Khaothong. Impact of chest wall motion caused by respiration in adjuvant radiotherapy for postoperative breast cancer patients. SpringerPlus, 5(1):1--8, 2016.
[23]
Young K Jang and Joe F Chicharo. Adaptive iir comb filter for harmonic signal cancellation. International Journal of Electronics Theoretical and Experimental, 75(2):241--250, 1993.
[24]
Gary G Berntson, J Thomas Bigger Jr, Dwain L Eckberg, Paul Grossman, Peter G Kaufmann, Marek Malik, Haikady N Nagaraja, Stephen W Porges, J Philip Saul, Peter H Stone, et al. Heart rate variability: origins, methods, and interpretive caveats. Psychophysiology, 34(6):623--648, 1997.
[25]
Reham Mohamed and Moustafa Youssef. Heartsense: Ubiquitous accurate multi-modal fusion-based heart rate estimation using smartphones. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 1(3):1--18, 2017.
[26]
Federica Landreani, Mattia Morri, Alba Martin-Yebra, Claudia Casellato, Esteban Pavan, Carlo Frigo, and Enrico G Caiani. Ultra-short-term heart rate variability analysis on accelerometric signals from mobile phone. In Proceedings of IEEE EHB, 2011.
[27]
Carolyn Jarvis. Physical Examination and Health Assessment E-Book. Elsevier Health Sciences, 2019.
[28]
A Rodríguez Valiente, A Trinidad, JR García Berrocal, C Górriz, and R Ramírez Camacho. Extended high-frequency (9-20 khz) audiometry reference thresholds in 645 healthy subjects. International journal of audiology, 53(8):531--545, 2014.
[29]
https://www.vernier.com/product/go-direct-respiration-belt/.
[30]
http://www.healforce.com/en/html/products/portableecgmonitors/healthcare-equipment-portable-ECG-monitors-PC-80B.html.
[31]
https://www.pdf-manuals.com/pdf/jbl-jembe-wireless-speakers-jbljembebtam-b-h-photo-video-240865-user-manual.pdf.
[32]
https://www.cdc.gov/niosh/docs/98-126/pdfs/98-126.pdf?id=10.26616/NIOSHPUB98126.
[33]
Cora E Lewis, Kathleen M McTigue, Lora E Burke, Paul Poirier, Robert H Eckel, Barbara V Howard, David B Allison, Shiriki Kumanyika, and F Xavier Pi-Sunyer. Mortality, health outcomes, and body mass index in the overweight range: a science advisory from the american heart association. Circulation, 119(25):3263--3271, 2009.
[34]
Dong Li, Jialin Liu, Sunghoon Ivan Lee, and Jie Xiong. Lasense: Pushing the limits of fine-grained activity sensing using acoustic signals. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 6(1):1--27, 2022.
[35]
Patrick Lazik and Anthony Rowe. Indoor pseudo-ranging of mobile devices using ultrasonic chirps. In Proceedings of ACM SenSys, 2012.
[36]
Sangki Yun, Yi-Chao Chen, and Lili Qiu. Turning a mobile device into a mouse in the air. In Proceedings of ACM MobiSys, 2015.
[37]
Corey R Pittman and Joseph J LaViola Jr. Multiwave: Complex hand gesture recognition using the doppler effect. In Proceedings of ACM GI, 2017.
[38]
Wenjie Ruan, Quan Z Sheng, Lei Yang, Tao Gu, Peipei Xu, and Longfei Shangguan. Audiogest: enabling fine-grained hand gesture detection by decoding echo signal. In Proceedings of ACM UbiComp, 2016.
[39]
Sidhant Gupta, Daniel Morris, Shwetak Patel, and Desney Tan. Soundwave: Using the doppler effect to sense gestures. In Proceedings of ACM CHI, 2012.
[40]
Kang Ling, Haipeng Dai, Yuntang Liu, Alex X Liu, Wei Wang, and Qing Gu. Ultragesture: Fine-grained gesture sensing and recognition. IEEE Transactions on Mobile Computing, 2020.
[41]
Yanwen Wang, Jiaxing Shen, and Yuanqing Zheng. Push the limit of acoustic gesture recognition. IEEE Transactions on Mobile Computing, 2020.
[42]
Rajalakshmi Nandakumar, Alex Takakuwa, Tadayoshi Kohno, and Shyamnath Gollakota. Covertband: Activity information leakage using music. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 1(3):1--24, 2017.
[43]
Wei Xu, ZhiWen Yu, Zhu Wang, Bin Guo, and Qi Han. Acousticid: gait-based human identification using acoustic signal. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 3(3):1--25, 2019.
[44]
Wenguang Mao, Jian He, and Lili Qiu. Cat: High-precision acoustic motion tracking. In Proceedings of MobiCom, 2016.
[45]
Ke Sun, Wei Wang, Alex X Liu, and Haipeng Dai. Depth aware finger tapping on virtual displays. In Proceedings of ACM MobiSys, 2018.
[46]
Sangki Yun, Yichao Chen, Huihuang Zhang, Lili Qiu, and Wenguang Mao. Strata: Fined-grained device-free tracking using acoustic signals. In Proceedings of ACM MobiSys, 2017.
[47]
Zhe Chen, Tianyue Zheng, Chao Cai, and Jun Luo. Movi-fi: motion-robust vital signs waveform recovery via deep interpreted rf sensing. In Proceedings of SCM MobiCom, 2021.
[48]
Tianyue Zheng, Zhe Chen, Shujie Zhang, Chao Cai, and Jun Luo. More-fi: Motion-robust and fine-grained respiration monitoring via deep-learning uwb radar. In Proceedings of ACM MobiCom, 2021.

Cited By

View all
  • (2025) mmTAA : A Contact-Less Thoracoabdominal Asynchrony Measurement System Based on mmWave Sensing IEEE Transactions on Mobile Computing10.1109/TMC.2024.346178424:2(627-641)Online publication date: Feb-2025
  • (2024)Remote Multi-Person Heart Rate Monitoring with Smart Speakers: Overcoming Separation ConstraintSensors10.3390/s2402038224:2(382)Online publication date: 8-Jan-2024
  • (2024)Heart Rate Variability Estimation based on RFID Tag-Pair in Dynamic EnvironmentsACM Transactions on Computing for Healthcare10.1145/3691355Online publication date: 9-Sep-2024
  • Show More Cited By

Index Terms

  1. LoEar: Push the Range Limit of Acoustic Sensing for Vital Sign Monitoring

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
    Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 6, Issue 3
    September 2022
    1612 pages
    EISSN:2474-9567
    DOI:10.1145/3563014
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 07 September 2022
    Published in IMWUT Volume 6, Issue 3

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Acoustic sensing
    2. Vital sign

    Qualifiers

    • Research-article
    • Research
    • Refereed

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)260
    • Downloads (Last 6 weeks)18
    Reflects downloads up to 20 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025) mmTAA : A Contact-Less Thoracoabdominal Asynchrony Measurement System Based on mmWave Sensing IEEE Transactions on Mobile Computing10.1109/TMC.2024.346178424:2(627-641)Online publication date: Feb-2025
    • (2024)Remote Multi-Person Heart Rate Monitoring with Smart Speakers: Overcoming Separation ConstraintSensors10.3390/s2402038224:2(382)Online publication date: 8-Jan-2024
    • (2024)Heart Rate Variability Estimation based on RFID Tag-Pair in Dynamic EnvironmentsACM Transactions on Computing for Healthcare10.1145/3691355Online publication date: 9-Sep-2024
    • (2024)Collecting Self-reported Physical Activity and Posture Data Using Audio-based Ecological Momentary AssessmentProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785848:3(1-35)Online publication date: 9-Sep-2024
    • (2024)BreathPro: Monitoring Breathing Mode during Running with EarablesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596078:2(1-25)Online publication date: 15-May-2024
    • (2024)ICY Interfaces: Exploration of Ice’s Ephemeral Features for Digital Game User ExperienceProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660725(2107-2124)Online publication date: 1-Jul-2024
    • (2024)RFSpy: Eavesdropping on Online Conversations with Out-of-Vocabulary Words by Sensing Metal Coil Vibration of Headsets Leveraging RFIDProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661887(169-182)Online publication date: 3-Jun-2024
    • (2024)F2Key: Dynamically Converting Your Face into a Private Key Based on COTS Headphones for Reliable Voice InteractionProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661860(127-140)Online publication date: 3-Jun-2024
    • (2024)EarSlideProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435158:1(1-29)Online publication date: 6-Mar-2024
    • (2024)Combating Islamophobia: Compromise, Community, and Harmony in Mitigating Harmful Online ContentACM Transactions on Social Computing10.1145/36415107:1(1-32)Online publication date: 27-Apr-2024
    • Show More Cited By

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media