Skip to main content
SpringerLink
Log in

MobiEye: turning your smartphones into a ubiquitous unobtrusive vital sign monitoring system

  • Regular Paper
  • Published:
CCF Transactions on Pervasive Computing and Interaction Aims and scope Submit manuscript

A Correction to this article was published on 12 January 2021

This article has been updated

Abstract

Recent advances in mobile and wearable technologies have stimulated significantly growing demands for more affordable, user-friendly, pervasive healthcare solutions that can be adopted by the public to proactively manage their health conditions and alleviate the burdens of hospitalization. This study seeks to propose a personalized ubiquitous health monitoring system that can unobtrusively monitor individuals’ vital signs, anywhere at any time. The proposed MobiEye framework makes use of the regular camera available on any smartphones or tablets to record various most important physiological signals without the need for acquiring extra specialized medical devices or attaching any sensor to the body. Through recording the reflected light intensities corresponding to the subtle blood flow changes with blood volume pulses, the proposed technique accurately extracts blood volume pulses from the facial videos recorded in real-world scenarios with the designed protocol. Experiments show that the proposed system achieved \(96 \, \% \,\) accuracy on average (with the standard deviations of \(\pm 1.2\)) for the heart rate estimation and higher correlations between the pulse transit time and the reference systolic blood pressure \((mean \, r = 0.89,\, SE= 0.05)\).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Change history

References

  • Abuella, H., Ekin, S.: Wireless vital signs monitoring system using visible light sensing (vls). arXiv preprint arXiv:1807.05408 (2018)

  • Angelopoulou, E.: Understanding the color of human skin. In: Human vision and electronic imaging VI, International Society for Optics and Photonics, vol. 4299, pp. 243–252 (2001)

  • Aoyagi, T.: Improvement of the earpiece oximeter. Abstr. Jpn. Soc. Med. Electron. Biol. Eng. 1974, 90–91 (1974)

    Google Scholar 

  • Baranoski, G.V.G., Krishnaswamy, A.: Light interaction with human skin: From believable images to predictable models. In: ACM SIGGRAPH ASIA 2008 Courses, SIGGRAPH Asia’08. ACM (2008)

  • Barun, V.V., Ivanov, A., Volotovskaya, A., Ulashchik, V.: Absorption spectra and light penetration depth of normal and pathologically altered human skin. J. Appl. Spectrosc. 74(3), 430–439 (2007)

    Article  Google Scholar 

  • Burt, P., Adelson, E.: The Laplacian pyramid as a compact image code. IEEE Trans. Commun. 31(4), 532–540 (1983)

    Article  Google Scholar 

  • Chandrasekhar, A., Natarajan, K., Yavarimanesh, M., Mukkamala, R.: An iPhone application for blood pressure monitoring via the oscillometric finger pressing method. Sci. Rep. 8(1), 13136 (2018)

    Article  Google Scholar 

  • Chekmenev, S.Y., Rara, H., Farag, A.A.: Non-contact, wavelet-based measurement of vital signs using thermal imaging. In: The first international conference on graphics, vision, and image processing (GVIP), pp. 107–112 (2005)

  • Chen, K.M., Misra, D., Wang, H., Chuang, H.R., Postow, E.: An X-band microwave life-detection system. IEEE Trans. Biomed. Eng. 7, 697–701 (1986)

    Article  Google Scholar 

  • Downey, C., Chapman, S., Randell, R., Brown, J., Jayne, D.G.: The impact of continuous versus intermittent vital signs monitoring in hospitals: a systematic review and narrative synthesis. Int. J. Nurs. Stud. 84, 19–27 (2018)

    Article  Google Scholar 

  • Fuchs, M., Chen, T., Wang, O., Raskar, R., Seidel, H.P., Lensch, H.P.: Real-time temporal shaping of high-speed video streams. Comput. Graph. 34(5), 575–584 (2010)

    Article  Google Scholar 

  • Gonzalez, R.C., Woods, R.E.: Image processing. Dig. Image Proces. 2, 1 (2007)

    Google Scholar 

  • Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis, vol. 46. Wiley, Hoboken (2004)

    Google Scholar 

  • Jeong, I.C., Ko, J.I., Hwang, S.O., Yoon, H.R.: A new method to estimate arterial blood pressure using photoplethysmographic signal. In: 2006 international conference of the IEEE engineering in medicine and biology society, IEEE, pp. 4667–4670 (2006)

  • Ji, X., Cheng, J., Tao, D., Wu, X., Feng, W.: The spatial Laplacian and temporal energy pyramid representation for human action recognition using depth sequences. Knowl. Based Syst. 122, 64–74 (2017)

    Article  Google Scholar 

  • Jin, F., Ioannis, P.: Thermistor at a distance: unobtrusive measurement of breathing. IEEE Trans. Biomed. Eng. 57(4), 988–998 (2010)

    Article  Google Scholar 

  • Kwon, S., Kim, H., Park, K.S.: Validation of heart rate extraction using video imaging on a built-in camera system of a smartphone. In: 2012 annual international conference of the IEEE engineering in medicine and biology society, IEEE, pp. 2174–2177 (2012)

  • Lewandowska, M., Rumiński, J., Kocejko, T., Nowak, J.: Measuring pulse rate with a webcam—a non-contact method for evaluating cardiac activity. In: 2011 federated conference on computer science and information systems (FedCSIS), IEEE, pp. 405–410 (2011)

  • Li, C., Lin, J.: Recent advances in Doppler radar sensors for pervasive healthcare monitoring. In: 2010 Asia-Pacific microwave conference, IEEE, pp. 283–290 (2010)

  • Li, L., Ng, C.S.L.: A, physically-based human skin reflection model. In: WSEAS International Conference. Proceedings. Mathematics and Computers in Science and Engineering, 10. World Scientific and Engineering Academy and Society (2009)

  • Lin, J.C.: Noninvasive microwave measurement of respiration. Proc. IEEE 63(10), 1530–1530 (1975)

    Article  Google Scholar 

  • Magdalena Nowara, E., Marks, T.K., Mansour, H., Veeraraghavan, A.: SparsePPG: Towards driver monitoring using camera-based vital signs estimation in near-infrared. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 1272–1281 (2018)

  • Monkaresi, H., Calvo, R.A., Yan, H.: A machine learning approach to improve contactless heart rate monitoring using a webcam. IEEE J. Biomed. Health Inf. 18(4), 1153–1160 (2013)

    Article  Google Scholar 

  • Mousavi, S.S., Firouzmand, M., Charmi, M., Hemmati, M., Moghadam, M., Ghorbani, Y.: Blood pressure estimation from appropriate and inappropriate ppg signals using a whole-based method. Biomed. Signal Process. Control 47, 196–206 (2019)

    Article  Google Scholar 

  • Muehlsteff, J., Aubert, X., Schuett, M.: Cuffless estimation of systolic blood pressure for short effort bicycle tests: the prominent role of the pre-ejection period. In: 2006 international conference of the IEEE engineering in medicine and biology society, IEEE, pp. 5088–5092 (2006)

  • Mukkamala, R., Hahn, J.O.: Toward ubiquitous blood pressure monitoring via pulse transit time: predictions on maximum calibration period and acceptable error limits. IEEE Trans. Biomed. Eng. 65(6), 1410–1420 (2017)

    Article  Google Scholar 

  • Patil, O.R., Gao, Y., Li, B., Jin, Z.: CamBP: a camera-based, non-contact blood pressure monitor. In: Proceedings of the 2017 ACM international joint conference on pervasive and ubiquitous computing and proceedings of the 2017 ACM international symposium on wearable computers, ACM pp. 524–529 (2017)

  • Poh, M.Z., McDuff, D.J., Picard, R.W.: Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt. Express 18(10), 10762–10774 (2010)

    Article  Google Scholar 

  • Shao, L., Zhen, X., Tao, D., Li, X.: Spatio-temporal Laplacian pyramid coding for action recognition. IEEE Trans. Cybern. 44(6), 817–827 (2013)

    Article  Google Scholar 

  • Shi, J., Tomasi, C.: Good features to track. In: Cornell University, Tech. rep. (1993)

  • Sun, N., Garbey, M., Merla, A., Pavlidis, I.: Imaging the cardiovascular pulse. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), IEEE, vol. 2, pp. 416–421 (2005)

  • Sun, G., Matsui, T., Watai, Y., Kim, S., Kirimoto, T., Suzuki, S., Hakozaki, Y.: Vital-SCOPE: design and evaluation of a smart vital sign monitor for simultaneous measurement of pulse rate, respiratory rate, and body temperature for patient monitoring. J Sensors 2018, 5 (2018)

    Google Scholar 

  • Suzuki, S., Matsui, T., Kagawa, M., Asao, T., Kotani, K.: An approach to a non-contact vital sign monitoring using dual-frequency microwave radars for elderly care. J. Biomed. Sci. Eng. 6(07), 704 (2013)

    Article  Google Scholar 

  • Talreja, P.S., Kasting, G.B., Kleene, N.K., Pickens, W.L., Wang, T.F.: Visualization of the lipid barrier and measurement of lipid pathlength in human stratum corneum. Aaps Pharmsci. 3(2), 48–56 (2001)

    Article  Google Scholar 

  • Tomasi, C., Detection, T.K.: Tracking of point features. In: Tech. rep., Tech. Rep. CMU-CS-91-132, Carnegie Mellon University (1991)

  • Tuchin, V.: Tissue Optics: Light Scattering Methods and Instruments for Medical Diagnostics. SPIE Press, Washington, DC (2015)

    Book  Google Scholar 

  • Villarroel, M., Jorge, J., Pugh, C., Tarassenko, L.: Non-contact vital sign monitoring in the clinic. In: 2017 12th IEEE international conference on automatic face & gesture recognition (FG 2017), IEEE, pp. 278–285 (2017)

  • Vinci, G., Lenhard, T., Will, C., Koelpin, A.: Microwave interferometer radar-based vital sign detection for driver monitoring syst. In: 2015 IEEE MTT-S international conference on microwaves for intelligent mobility (ICMIM), IEEE, pp. 1–4. (2015)

  • Viola, P., Jones, M., et al.: Rapid object detection using a boosted cascade of simple features. CVPR 1(1), 511–518 (2001)

    Google Scholar 

  • Wieringa, F.P., Mastik, F., van der Steen, A.F.: Contactless multiple wavelength photoplethysmographic imaging: a first step toward “SpO2 camera” technology. Ann. Biomed. Eng. 33(8), 1034–1041 (2005)

    Article  Google Scholar 

  • Wu, H.Y., Rubinstein, M., Shih, E., Guttag, J., Durand, F., Freeman, W.: Eulerian video magnification for revealing subtle changes in the world. ACM Trans. Graph. 31, 4 (2012)

    Article  Google Scholar 

  • Xing, X., Sun, M.: Optical blood pressure estimation with photoplethysmography and FFT-based neural networks. Biomed. Opt. Express 7(8), 3007–3020 (2016)

    Article  Google Scholar 

  • Zhang, Q., Zeng, X., Hu, W., Zhou, D.: A machine learning-empowered system for long-term motion-tolerant wearable monitoring of blood pressure and heart rate with ear-ECG/PPG. IEEE Access 5, 10547–10561 (2017)

    Article  Google Scholar 

  • Zhao, F., Li, M., Jiang, Z., Tsien, J.Z., Lu, Z.: Camera-based, non-contact, vital-signs monitoring technology may provide a way for the early prevention of SIDS in infants. Front. Neurol. 7, 236 (2016)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, Binghamton University, State University of New York, Binghamton, USA

    Omkar Patil

  2. Department of Computer Science and Engineering, University at Buffalo, State University of New York, Buffalo, USA

    Wei Wang, Yang Gao & Zhanpeng Jin

Authors
  1. Omkar Patil
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yang Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhanpeng Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhanpeng Jin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Patil, O., Wang, W., Gao, Y. et al. MobiEye: turning your smartphones into a ubiquitous unobtrusive vital sign monitoring system. CCF Trans. Pervasive Comp. Interact. 2, 97–112 (2020). https://doi.org/10.1007/s42486-020-00033-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42486-020-00033-3

Keywords

Search

Navigation