skip to main content
10.1145/3329189.3329204acmotherconferencesArticle/Chapter ViewAbstractPublication PagespervasivehealthConference Proceedingsconference-collections
research-article

Towards Reliable Data Collection and Annotation to Extract Pulmonary Digital Biomarkers Using Mobile Sensors

Published: 20 May 2019 Publication History

Abstract

Proliferation of sensors embedded in smartphones and smartwatches helps capture rich dataset for machine learning algorithms to extract meaningful digital bio-markers on consumer devices for monitoring disease progression and treatment response. However, development and validation of machine learning algorithms depend on gathering high fidelity sensor data and reliable ground-truth. We conduct a study, called mLungStudy, with 131 subjects with varying pulmonary conditions to collect mobile sensor data including audio, accelerometer, gyroscope using a smartphone and a smartwatch, in order to extract pulmonary biomarkers such as breathing, coughs, spirometry, and breathlessness. Our study shows that commonly used breathing ground-truth data from chestband may not always be reliable as a gold-standard. Our analysis shows that breathlessness biomarkers such as pause time and pause frequency from 2.15 minutes of audio can be as reliable as those extracted from 5 minutes' worth of speech data. This finding can be useful for future studies to trade-off between the reliability of breathlessness data and patient comfort in generating continuous speech data. Furthermore, we use crowdsourcing techniques to annotate pulmonary sound events for developing signal processing and machine learning algorithms. In this paper, we highlight several practical challenges to collect and annotate physiological data and acoustic symptoms from chronic pulmonary patients and ways to improve data quality. We show that the waveform visualization of the audio signal improves annotation quality which leads to a 6.59% increase in cough classification accuracy and a 6% increase in spirometry event classification accuracy. Findings from this study inform future studies focusing on developing explainable machine learning models to extract pulmonary digital bio-markers using mobile sensors.

References

[1]
Mohsin Ahmed, Md Mahbubur Rahman, Viswam Nathan, Ebrahim Nemati, Korosh Vatanparvar, and Jilong Kuang. 2019. mLung: Privacy-Preserving Naturally Windowed Lung Activity Detection for Pulmonary Patients. In IEEE BSN.
[2]
ALA 2018. American Lung Association. Retrieved December 12, 2018 from http://www.lung.org/lung-health-and-diseases/lung-disease-lookup/copd/learn-about-copd/how-serious-is-copd.html
[3]
Rummana Bari, Roy J Adams, Md Mahbubur Rahman, Megan Battles Parsons, Eugene H Buder, and Santosh Kumar. 2018. rConverse: Moment by Moment Conversation Detection Using a Mobile Respiration Sensor. ACM IMWUT (2018).
[4]
Andrew Bates, Martin J Ling, Janek Mann, and DK Arvind. 2010. Respiratory Rate and Flow Waveform Estimation from Tri-axial Accelerometer Data. In IEEE BSN.
[5]
Matthias Budde, Andrea Schankin, Julien Hoffmann, Marcel Danz, Till Riedel, and Michael Beigl. 2017. Participatory Sensing or Participatory Nonsense?: Mitigating the Effect of Human Error on Data Quality in Citizen Science. ACM IMWUT (2017).
[6]
S Burge and JA Wedzicha. 2003. COPD Exacerbations: Definitions and Classifications. European Respiratory Journal (2003).
[7]
Pierre-Régis Burgel, Pascale Nesme-Meyer, Pascal Chanez, Denis Caillaud, Philippe Carré, Thierry Perez, Nicolas Roche, et al. 2009. Cough and Sputum Production are Associated with Frequent Exacerbations and Hospitalizations in COPD Subjects. Elsevier Chest (2009).
[8]
Soujanya Chatterjee, Md Mahbubur Rahman, Ebrahim Nemati, and Jilong Kunag. 2019. WheezeD: Respiration Phase Based Wheeze Detection Using Acoustic Data From Pulmonary Patients Under Attack. In ACM Pervasive Computing Technologies for Healthcare (PervasiveHealth).
[9]
Emil Chiauzzi, Carlos Rodarte, and Pronabesh DasMahapatra. 2015. Patient-centered activity monitoring in the self-management of chronic health conditions. BMC medicine (2015).
[10]
Alvaro A Cruz. 2007. Global surveillance, prevention and control of chronic respiratory diseases: a comprehensive approach. World Health Organization.
[11]
Emre Ertin, Nathan Stohs, Santosh Kumar, Andrew Raij, Mustafa al'Absi, and Siddharth Shah. 2011. AutoSense: Unobtrusively Wearable Sensor Suite for Inferring the Onset, Causality, and Consequences of Stress in the Field. In ACM SenSys.
[12]
Florian Eyben, Martin Wöllmer, and Björn Schuller. 2010. Opensmile: the Munich Versatile and Fast Open-source Audio Feature Extractor. In ACM conference on Multimedia.
[13]
Grant Fairbanks. 1960. Voice and Articulation Drillbook. Harper New York.
[14]
FigureEight 2019. Figure Eight. Retrieved April 15, 2019 from https://www.figure-eight.com/
[15]
Kevin E Forkheim, David Scuse, and Hans Pasterkamp. 1995. A comparison of neural network models for wheeze detection. In IEEE WESCANEX Communications, Power, and Computing.
[16]
Javier Hernandez, Daniel McDuff, and Rosalind W Picard. 2015. Biowatch: estimation of heart and breathing rates from wrist motions. In IEEE Pervasive Computing Technologies for Healthcare (PervasiveHealth).
[17]
Javier Hernandez, Daniel J McDuff, and Rosalind W Picard. 2015. Biophone: Physiology monitoring from peripheral smartphone motions. (2015).
[18]
Eric C Larson, Mayank Goel, Gaetano Boriello, Sonya Heltshe, Margaret Rosenfeld, and Shwetak N Patel. 2012. SpiroSmart: using a microphone to measure lung function on a mobile phone. In ACM UbiComp.
[19]
Daniyal Liaqat, Robert Wu, Andrea Gershon, Hisham Alshaer, Frank Rudzicz, and Eyal de Lara. 2018. Challenges with Real-world Smartwatch based Audio Monitoring. In ACM Workshop on Wearable Systems and Applications.
[20]
LungInstitute 2018. Lung Institute. Retrieved December 12, 2018 from https://lunginstitute.com/blog/the-cost-of-lung-disease/
[21]
Marc Miravitlles. 2011. Cough and Sputum Production as Risk Factors for Poor Outcomes in Patients with COPD. Elsevier Respiratory Medicine (2011).
[22]
Jimmy Moore, Pascal Goffin, Miriah Meyer, Philip Lundrigan, Neal Patwari, Katherine Sward, and Jason Wiese. 2018. Managing In-home Environments through Sensing, Annotating, and Visualizing Air Quality Data. ACM IMWUT (2018).
[23]
Bernard Munos, Pamela C Baker, Brian M Bot, Michelle Crouthamel, Glen Vries, Ian Ferguson, John D Hixson, Linda A Malek, John J Mastrototaro, Veena Misra, Aydogan Ozcan, Leonard Sacks, and Pei Wang. 2016. Mobile Health: the Power of Wearables, Sensors, and Apps to Transform Clinical Trials. Wiley Online Library Annals of the New York Academy of Sciences (2016).
[24]
Viswam Nathan, Korosh Vatanparvar, Md Mahbubur Rahman, Ebrahim Nemati, and Jilong Kuang. 2019. Assessment of Chronic Pulmonary Disease Patients Using Biomarkers from Natural Speech Recorded by Mobile Devices. In IEEE BSN.
[25]
Ebrahim Nemati, Christina Batteate, and Michael Jerrett. 2017. Opportunistic Environmental Sensing with Smartphones: a Critical Review of Current Literature and Applications. Springer Current environmental health reports (2017).
[26]
Ebrahim Nemati, Md Mahbubur Rahman, Viswam Nathan, and Jilong Kuang. 2018. Private Audio-Based Cough Sensing for In-Home Pulmonary Assessment using Mobile Devices. In EAI International Conference on Body Area Networks.
[27]
Ebrahim Nemati, Young Soo Suh, Babak Motamed, and Majid Sarrafzadeh. 2016. Gait Velocity Estimation for a Smartwatch Platform using Kalman Filter Peak Recovery. In IEEE BSN.
[28]
Nuance 2019. Nuance PowerMic III. Retrieved January 15, 2019 from https://www.nuance.com/dragon/dragon-accessories.html
[29]
M Pourhomayoun, E Nemati, B Mortazavi, and M Sarrafzadeh. 2015. Context-aware data analytics for activity recognition. In International Conference on Data Analytics.
[30]
Public Good 2018. Private Data for Public Good. Retrieved December 18, 2018 from http://hdexplore.calit2.net/wp-content/uploads/2015/08/hdx_final_report_small.pdf
[31]
Mahbubur Rahman, Nasir Ali, Rummana Bari, Nazir Saleheen, Mustafa al'Absi, Emre Ertin, Ashley Kennedy, Kenzie L Preston, and Santosh Kumar. 2017. mDebugger: Assessing and Diagnosing the Fidelity and Yield of Mobile Sensor Data. In Springer Mobile Health.
[32]
Md Mahbubur Rahman, Rummana Bari, Amin Ahsan Ali, Moushumi Sharmin, Andrew Raij, Karen Hovsepian, Syed Monowar Hossain, Emre Ertin, Ashley Kennedy, David H Epstein, Kenzie L Preston, Michelle Jobes, Kenneth D Ward, Mustafa al'Absi, and Santosh Kumar. 2014. Are We There Yet? Feasibility of Continuous Stress Assessment via Wireless Physiological Sensors. In ACM Conference on Bioinformatics, Computational Biology, and Health Informatics.
[33]
Md Mahbubur Rahman, Ebrahim Nemati, Viswam Nathan, and Jilong Kuang. 2018. InstantRR: Instantaneous Respiratory Rate Estimation on Context-aware Mobile Devices. In EAI International Conference on Body Area Networks.
[34]
Javier Ramırez, José C Segura, Carmen Benıtez, Angel De La Torre, and Antonio Rubio. 2004. Efficient Voice Activity Detection Algorithms Using Long-term Speech Information. Elsevier Speech communication (2004).
[35]
Ronald W Schafer. 2011. What is a Savitzky-Golay Filter?{lecture notes}. IEEE Signal processing magazine (2011).
[36]
sHealth 2018. Samsung Health. Retrieved December 18, 2018 from https://www.samsung.com/us/support/answer/ANS00062448/
[37]
Germán D Sosa, Angel Cruz-Roa, and Fabio A González. 2015. Automatic Detection of Wheezes by Evaluation of Multiple Acoustic Feature Extraction Methods and C-weighted SVM. In Society for Optics and Photonics Symposium on Medical Information Processing and Analysis.
[38]
Xiao Sun, Li Qiu, Yibo Wu, Yeming Tang, and Guohong Cao. 2017. Sleepmonitor: Monitoring respiratory rate and body position during sleep using smartwatch. ACM IMWUT (2017).
[39]
R Tehrany. 2015. Speech Breathing Patterns in Health and Chronic Respiratory Disease. Ph.D. Dissertation. University of Southampton.
[40]
Korosh Vatanparvar, Viswam Nathan, Ebrahim Nemati, Md Mahbubur Rahman, and Jilong Kuang. 2019. A Generative Model for Speech Segmentation and Obfuscation for Remote Health Monitoring. In IEEE BSN.
[41]
Varun Viswanath, Jake Garrison, and Shwetak Patel. 2018. SpiroConfidence: Determining the Validity of Smartphone Based Spirometry Using Machine Learning. In IEEE EMBC.
[42]
Rui Wang, Fanglin Chen, Zhenyu Chen, Tianxing Li, Gabriella Harari, Stefanie Tignor, Xia Zhou, Dror Ben-Zeev, and Andrew T Campbell. 2014. StudentLife: Assessing Mental Health, Academic Performance and Behavioral Trends of College Students using Smartphones. In ACMUbiComp.
[43]
Shichao Yue, Hao He, Hao Wang, Hariharan Rahul, and Dina Katabi. 2018. Extracting Multi-Person Respiration from Entangled RF Signals. ACM IMWUT (2018).
[44]
Youwei Zeng, Dan Wu, Ruiyang Gao, Tao Gu, and Daqing Zhang. 2018. Full-Breathe: Full Human Respiration Detection Exploiting Complementarity of CSI Phase and Amplitude of WiFi Signals. ACM IMWUT (2018).
[45]
zephyr 2019. Zephyr Bioharness. Retrieved January 15, 2019 from https://www.zephyranywhere.com/

Cited By

View all
  • (2024)Aiding Clinical Decision-Making at the Individual and Community Level Using Mobile Sensor Data: Study Protocol for an Experimental DesignCureus10.7759/cureus.71398Online publication date: 13-Oct-2024
  • (2024)Smart City Solutions: Freemove App Enables Efficient Management of Loja’s Bike LanesProceedings of the International Conference on Computer Science, Electronics and Industrial Engineering (CSEI 2023)10.1007/978-3-031-69228-4_50(764-784)Online publication date: 23-Dec-2024
  • (2023)SpiroMask: Measuring Lung Function Using Consumer-Grade MasksACM Transactions on Computing for Healthcare10.1145/35701674:1(1-34)Online publication date: 27-Feb-2023
  • Show More Cited By

Index Terms

  1. Towards Reliable Data Collection and Annotation to Extract Pulmonary Digital Biomarkers Using Mobile Sensors

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      PervasiveHealth'19: Proceedings of the 13th EAI International Conference on Pervasive Computing Technologies for Healthcare
      May 2019
      475 pages
      ISBN:9781450361262
      DOI:10.1145/3329189
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      In-Cooperation

      • EAI: The European Alliance for Innovation

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 20 May 2019

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Breathing
      2. Breathlessness
      3. Cough
      4. Crowdsourced Annotation
      5. Data Quality
      6. Digital Biomarkers
      7. mHealth

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Conference

      PervasiveHealth'19

      Acceptance Rates

      Overall Acceptance Rate 55 of 116 submissions, 47%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)53
      • Downloads (Last 6 weeks)5
      Reflects downloads up to 20 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Aiding Clinical Decision-Making at the Individual and Community Level Using Mobile Sensor Data: Study Protocol for an Experimental DesignCureus10.7759/cureus.71398Online publication date: 13-Oct-2024
      • (2024)Smart City Solutions: Freemove App Enables Efficient Management of Loja’s Bike LanesProceedings of the International Conference on Computer Science, Electronics and Industrial Engineering (CSEI 2023)10.1007/978-3-031-69228-4_50(764-784)Online publication date: 23-Dec-2024
      • (2023)SpiroMask: Measuring Lung Function Using Consumer-Grade MasksACM Transactions on Computing for Healthcare10.1145/35701674:1(1-34)Online publication date: 27-Feb-2023
      • (2023)Engineering Large Wearable Sensor Data towards Digital Measures2023 IEEE 8th International Conference on Big Data Analytics (ICBDA)10.1109/ICBDA57405.2023.10104923(14-23)Online publication date: 3-Mar-2023
      • (2022)Aiding clinical decision-making at the individual and community level using mobile sensor data – A study protocol for an experimental design (Preprint)JMIR Research Protocols10.2196/39442Online publication date: 10-May-2022
      • (2022)BreatheBuddyProceedings of the ACM on Human-Computer Interaction10.1145/35467486:MHCI(1-18)Online publication date: 20-Sep-2022
      • (2022)From Personalized Medicine to Population Health: A Survey of mHealth Sensing TechniquesIEEE Internet of Things Journal10.1109/JIOT.2022.31610469:17(15413-15434)Online publication date: 1-Sep-2022
      • (2021)BreathTrackProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34781235:3(1-22)Online publication date: 14-Sep-2021
      • (2021)Determinants of Longitudinal Adherence in Smartphone-Based Self-Tracking for Chronic Health ConditionsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34480935:1(1-24)Online publication date: 30-Mar-2021
      • (2021)Frugal development and deployment of an innovative mobile health platform for COVID-19 in Sri Lanka: the case of SelfShield appBMJ Innovations10.1136/bmjinnov-2021-0008367:4(604-608)Online publication date: 28-Sep-2021
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media