Skip to main content
SpringerLink
Log in

Remote Respiration Monitoring of Moving Person Using Radio Signals

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13697))

Included in the following conference series:

Abstract

Non-contact respiration rate measurement (nRRM), which aims to monitor one’s breathing status without any contact with the skin, can be utilized in various remote applications (e.g., telehealth or emergency detection). The existing nRRM approaches mainly analyze fine details from videos to extract minute respiration signals; however, they have practical limitations in that the head or body of a subject must be quasi-stationary. In this study, we examine the task of estimating the respiration signal of a non-stationary subject (a person with large body movements or even walking around) based on radio signals. The key idea is that the received radio signals retain both the reflections from human global motion (GM) and respiration in a mixed form, while preserving the GM-only components at the same time. During training, our model leverages a novel multi-task adversarial learning (MTAL) framework to capture the mapping from radio signals to respiration while excluding the GM components in a self-supervised manner. We test the proposed model based on the newly collected and released datasets under real-world conditions. This study is the first realization of the nRRM task for moving/occluded scenarios, and also outperforms the state-of-the-art baselines even when the person sits still.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ali, M., Elsayed, A., Mendez, A., Savaria, Y., Sawan, M.: Contact and remote breathing rate monitoring techniques: a review. IEEE Sens. J. 21(13), 14569–14586 (2021)

    Article  Google Scholar 

  2. Bobbia, S., Macwan, R., Benezeth, Y., Mansouri, A., Dubois, J.: Unsupervised skin tissue segmentation for remote photoplethysmography. Pattern Recognit. Lett. 124(1), 82–90 (2019)

    Article  Google Scholar 

  3. Chen, W., McDuff, D.: Deepphys: video-based physiological measurement using convolutional attention networks. In: European Conference on Computer Vision (ECCV), pp. 349–365, September 2018

    Google Scholar 

  4. Chintalapudi, K., Padmanabha Iyer, A., Padmanabhan, V.N.: Indoor localization without the pain. In: ACM Annual International Conference on Mobile Computing and Networking (MobiCom), pp. 173–184 (2010)

    Google Scholar 

  5. Choi, I.O., Kim, M., Choi, J.H., Park, J.K., Park, S.H., Kim, K.T.: Robust cardiac rate estimation of an individual. IEEE Sens. J. 21(13), 15053–15064 (2021)

    Article  Google Scholar 

  6. Choi, J.H., Kim, J.E., Jeong, N.H., Kim, K.T., Jin, S.H.: Accurate people counting based on radar: deep learning approach. In: IEEE Radar Conference (RadarConf), pp. 1–5 (2020)

    Google Scholar 

  7. Choi, J.H., Kim, J.E., Kim, K.T.: Deep learning approach for radar-based people counting. IEEE Internet Things J. 9, 1–16 (2021)

    Google Scholar 

  8. Choi, J.H., Kim, J.E., Kim, K.T.: People counting using IR-UWB radar sensor in a wide area. IEEE Internet Things J. 8(7), 5806–5821 (2021)

    Article  Google Scholar 

  9. Ding, C., Yan, J., Zhang, L., Zhao, H., Hong, H., Zhu, X.: Noncontact multiple targets vital sign detection based on VMD algorithm. In: IEEE Radar Conference (RadarConf), pp. 0727–0730 (2017)

    Google Scholar 

  10. Estepp, J.R., Blackford, E.B., Meier, C.M.: Recovering pulse rate during motion artifact with a multi-imager array for non-contact imaging photoplethysmography. In: IEEE Conference on Systems, Man, and Cybernetics (SMC), pp. 1462–1469 (2014)

    Google Scholar 

  11. Fan, L., Li, T., Fang, R., Hristov, R., Yuan, Y., Katabi, D.: Learning longterm representations for person re-identification using radio signals. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10699–10709 (2020)

    Google Scholar 

  12. Fan, L., Li, T., Yuan, Y., Katabi, D.: In-home daily-life captioning using radio signals. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12347, pp. 105–123. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58536-5_7

    Chapter  Google Scholar 

  13. Guan, J., Madani, S., Jog, S., Gupta, S., Hassanieh, H.: Through fog high-resolution imaging using Millimeter wave radar. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11461–11470 (2020)

    Google Scholar 

  14. Ha, U., Assana, S., Adib, F.: Contactless seismocardiography via deep learning radars. In: ACM Annual International Conference on Mobile Computing and Networking (MobiCom), pp. 1–14 (2020)

    Google Scholar 

  15. de Haan, G., Jeanne, V.: Robust pulse rate from chrominance-based rPPG. IEEE Trans. Biomed. Eng. 60(10), 2878–2886 (2013)

    Article  Google Scholar 

  16. de Haan, G., Van Leest, A.: Improved motion robustness of remote-PPG by using the blood volume pulse signature. Physiol. Meas. 35(9), 1913–1926 (2014)

    Article  Google Scholar 

  17. He, M., Nian, Y., Liu, B.: Noncontact heart beat signal extraction based on wavelet transform. In: International Conference on Biomedical Engineering and Informatics (BMEI), pp. 209–213 (2015)

    Google Scholar 

  18. Iovescu, C., Rao, S.: The fundamentals of millimeter wave sensors. Texas Instrum. 1–8 (2017)

    Google Scholar 

  19. Jiang, C., Guo, J., He, Y., Jin, M., Li, S., Liu, Y.: mmVib: micrometer-level vibration measurement with mmWave radar. In: ACM Annual International Conference on Mobile Computing and Networking (MobiCom), pp. 1–13 (2020)

    Google Scholar 

  20. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (ICLR), pp. 1–15 (2015)

    Google Scholar 

  21. Kumar, S., Gil, S., Katabi, D., Rus, D.: Accurate indoor localization with zero start-up cost. In: ACM Annual International Conference on Mobile Computing and Networking (MobiCom), pp. 483–494 (2014)

    Google Scholar 

  22. Lam, A., Kuno, Y.: Robust heart rate measurement from video using select random patches. In: International Conference on Computer Vision (ICCV), pp. 3640–3648 (2015)

    Google Scholar 

  23. Lee, E., Chen, E., Lee, C.-Y.: Meta-rPPG: remote heart rate estimation using a transductive meta-learner. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12372, pp. 392–409. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58583-9_24

    Chapter  Google Scholar 

  24. Lewandowska, M., Rumiński, J., Kocejko, T., Nowak, J.: Measuring pulse rate with a webcam - a non-contact method for evaluating cardiac activity. In: Federated Conference on Computer Science and Information Systems (FedCSIS), pp. 405–410 (2011)

    Google Scholar 

  25. Li, J., Stoica, P.: MIMO Radar Signal Processing. Wiley, Hoboken (2008)

    Book  Google Scholar 

  26. Li, J., Liu, L., Zeng, Z., Liu, F.: Advanced signal processing for vital sign extraction with applications in UWB radar detection of trapped victims in complex environments. IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. 7(3), 783–791 (2014)

    Google Scholar 

  27. Li, X., Chen, J., Zhao, G., Pietikäinen, M.: Remote heart rate measurement from face videos under realistic situations. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4264–4271 (2014)

    Google Scholar 

  28. Liu, X., Fromm, J., Patel, S., McDuff, D.: Multi-task temporal shift attention networks for on-device contactless vitals measurement. In: Advances in Neural Information Processing Systems (NIPS), pp. 1–23 (2020)

    Google Scholar 

  29. McDuff, D.: Deep super resolution for recovering physiological information from videos. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1480–1487 (2018)

    Google Scholar 

  30. McDuff, D.J., Sarah, G., Picard, R.W.: Improvements in remote cardiopulmonary measurement using a five band digital camera. IEEE Trans. Biomed. Eng. 61(10), 2593–2601 (2014)

    Article  Google Scholar 

  31. Mercuri, M., Lorato, I., Liu, Y.H., Wieringa, F., Van Hoof, C., Torfs, T.: Vital-sign monitoring and spatial tracking of multiple people using a contactless radar-based sensor. Nat. Electron. 2, 252–262 (2019)

    Article  Google Scholar 

  32. Niu, X., Han, H., Shan, S., Chen, X.: SynRhythm: learning a deep heart rate estimator from general to specific. In: International Conference on Pattern Recognition (ICPR), pp. 3580–3585 (2018)

    Google Scholar 

  33. Niu, X., Han, H., Shan, S., Chen, X.: VIPL-HR: a multi-modal database for pulse estimation from less-constrained face video. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) ACCV 2018. LNCS, vol. 11365, pp. 562–576. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20873-8_36

    Chapter  Google Scholar 

  34. Niu, X., Shan, S., Han, H., Chen, X.: RhythmNet: end-to-end heart rate estimation from face via spatial-temporal representation. IEEE Trans. Image Process. 29, 2409–2423 (2020)

    Article  Google Scholar 

  35. Nowara, E.M., McDuff, D., Veeraraghavan, A.: The benefit of distraction: Denoising camera-based physiological measurements using inverse attention. In: International Conference on Computer Vision (ICCV), pp. 4955–4964 (2021)

    Google Scholar 

  36. Pan, J.J., Pan, S.J., Yin, J., Ni, L.M., Yang, Q.: Tracking mobile users in wireless networks via semi-supervised colocalization. IEEE Trans. Pattern Anal. Mach. Intell. 34(3), 587–600 (2012)

    Article  Google Scholar 

  37. Poh, M.Z., McDuff, D.J., Picard, R.W.: Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt. Express 18(10), 10762–10774 (2010)

    Article  Google Scholar 

  38. Poh, M.Z., McDuff, D.J., Picard, R.W.: Advancements in noncontact, multiparameter physiological measurements using a webcam. IEEE Trans. Biomed. Eng. 58(1), 7–11 (2011)

    Article  Google Scholar 

  39. Ren, W., et al.: Vital sign detection in any orientation using a distributed radar network via modified independent component analysis. IEEE Trans. Microw. Theory Techn. 69(11), 4774–4790 (2021)

    Article  Google Scholar 

  40. Revanur, A., Li, Z., Ciftci, U.A., Yin, L., Jeni, L.A.: The first vision for vitals (V4V) challenge for non-contact video-based physiological estimation. In: International Conference on Computer Vision Workshop (ICCVW), pp. 2760–2767 (2021)

    Google Scholar 

  41. Rohling, H.: Radar CFAR thresholding in clutter and multiple target situations. IEEE Trans. Aerosp. Electron. Syst. AES-19(4), 608–621 (1983)

    Google Scholar 

  42. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  43. Scheiner, N., et al.: Seeing around street corners: non-line-of-sight detection and tracking in-the-wild using Doppler radar. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2068–2077 (2020)

    Google Scholar 

  44. Tariq, A., Ghafouri-Shiraz, H.: Vital signs detection using Doppler radar and continuous wavelet transform. In: European Conference on Antennas and Propagation (EUCAP), pp. 285–288 (2011)

    Google Scholar 

  45. Tu, J., Hwang, T., Lin, J.: Respiration rate measurement under 1-D body motion using single continuous-wave doppler radar vital sign detection system. IEEE Trans. Microw. Theory Techn. 64(6), 1937–1946 (2016)

    Article  Google Scholar 

  46. Verkruysse, W., Othar Svaasand, L., Stuart Nelson, J.: Remote plethysmographic imaging using ambient light. Opt. Express 16(26), 21434–21445 (2008)

    Article  Google Scholar 

  47. Wang, W., den Brinker, A.C., Stuijk, S., de Haan, G.: Amplitude-selective filtering for remote-PPG. Biomed. Opt. Express 8(3), 1965–1980 (2017)

    Google Scholar 

  48. Wang, W., den Brinker, A.C., Stuijk, S., de Haan, G.: Algorithmic principles of remote PPG. IEEE Trans. Biomed. Eng. 64(7), 1479–1491 (2017)

    Article  Google Scholar 

  49. Wang, W., Stuijk, S., de Haan, G.: Exploiting spatial redundancy of image sensor for motion robust rPPG. IEEE Trans. Biomed. Eng. 62(2), 415–425 (2015)

    Article  Google Scholar 

  50. Xiong, J., Sundaresan, K., Jamieson, K.: ToneTrack: leveraging frequency-agile radios for time-based indoor wireless localization. In: ACM Annual International Conference on Mobile Computing and Networking (MobiCom), pp. 537–549 (2015)

    Google Scholar 

  51. Yu, Z., Peng, W., Li, X., Hong, X., Zhao, G.: Remote heart rate measurement from highly compressed facial videos: an end-to-end deep learning solution with video enhancement. In: International Conference on Computer Vision (ICCV), pp. 151–160 (2019)

    Google Scholar 

  52. Zhan, Q., Wang, W., de Haan, G.: Analysis of CNN-based remote-PPG to understand limitations and sensitivities. Biomed. Opt. Express 11(3), 1268–1283 (2020)

    Article  Google Scholar 

  53. Zhao, M., et al.: Through-wall human pose estimation using radio signals. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7356–7365 (2018)

    Google Scholar 

  54. Zhao, M., et al.: Through-wall human mesh recovery using radio signals. In: International Conference on Computer Vision (ICCV), pp. 10112–10121 (2019)

    Google Scholar 

  55. Zhao, M., et al.: RF-based 3D skeletons. In: Conference of the ACM Special Interest Group Data Communication (SIGCOMM), pp. 267–281 (2018)

    Google Scholar 

  56. Zheng, T., Chen, Z., Zhang, S., Cai, C., Luo, J.: MoRe-Fi: motion-robust and fine-grained respiration monitoring via deep-learning UWB radar. In: ACM Conference on Embedded Networked Sensor Systems (SenSys), New York, NY, USA, pp. 111–124 (2021)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. POSTECH, Pohang, Republic of Korea

    Jae-Ho Choi, Ki-Bong Kang & Kyung-Tae Kim

  2. Samsung Electronics, Suwon, Republic of Korea

    Ki-Bong Kang

Authors
  1. Jae-Ho Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ki-Bong Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kyung-Tae Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kyung-Tae Kim .

Editor information

Editors and Affiliations

  1. Tel Aviv University, Tel Aviv, Israel

    Shai Avidan

  2. University College London, London, UK

    Gabriel Brostow

  3. Google AI, Accra, Ghana

    Moustapha Cissé

  4. University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Facebook (United States), Menlo Park, CA, USA

    Tal Hassner

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 1040 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Choi, JH., Kang, KB., Kim, KT. (2022). Remote Respiration Monitoring of Moving Person Using Radio Signals. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13697. Springer, Cham. https://doi.org/10.1007/978-3-031-19836-6_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19836-6_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19835-9

  • Online ISBN: 978-3-031-19836-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation