Skip to main content
SpringerLink
Log in

Remote Liveness and Heart Rate Detection from Video

  • Conference paper
  • First Online:
Pattern Recognition. ICPR International Workshops and Challenges (ICPR 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12668))

Included in the following conference series:

  • 1761 Accesses

Abstract

The remote detection of liveness is critical for senior and baby care, disaster response, the military, and law enforcement. Existing solutions are mostly based on special sensor hardware or the spectral signature of living skin. This paper uses commercial electro-optical and infrared (EO/IR) sensors to capture a very short video for low cost and fast liveness detection. The key components of our system include: tiny human body and face detection from long range and low-resolution video, and remote liveness detection based on micro-motion from a short human body and face video. These micro-motions are caused by breathing and heartbeat. A deep learning architecture is designed for remote body and face detection. A novel algorithm is proposed for adaptive sensor and background noise cancellation. An air platform motion compensation algorithm is tested on video data collected on a drone. The key advantages are: low cost, requires very short video, works with many parts of a human body even when skin is not visible, works on any motion caused by eyes, mouth, heartbeat, breathing, or body parts, and works in all lighting conditions. To the author’s best knowledge, this is the first work on video micro-motion based liveness detection on a moving platform and from a long standoff range of 100 m. Once a subject is deemed alive, video-based remote heart rate detection is applied to assess the physiological and psychological state of the subject. This is also the first work on outdoor remote heart rate detection from a long standoff range of 100 m. On a public available indoor COHFACE data evaluation, our heart rate estimation algorithm outperforms all published work on the same dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Balakrishnan, G., Durand, F., Guttag, J.: Detecting pulse from head motions in video. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3430–3437 (2013)

    Google Scholar 

  2. https://www.procpr.org/blog/training/cpr-length

  3. Hu, P., Ramanan, D.: Finding tiny faces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 951–959 (2017)

    Google Scholar 

  4. Neumann, L., Vedaldi, A.: Tiny people pose. In: Jawahar, C. V., Li, Hongdong, Mori, Greg, Schindler, Konrad (eds.) ACCV 2018. LNCS, vol. 11363, pp. 558–574. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20893-6_35

    Chapter  Google Scholar 

  5. Polonskiy, L., et al.: U.S. Patent No. 7,417,727. U.S. Patent and Trademark Office, Washington, DC (2008)

    Google Scholar 

  6. Rice, R.R., Zediker, M.S.: U.S. Patent No. 5,867,257. U.S. Patent and Trademark Office, Washington, DC (1999)

    Google Scholar 

  7. Yazdi, M., Bouwmans, T.: New trends on moving object detection in video images captured by a moving camera: a survey. Comput. Sci. Rev. 28, 157–177 (2018)

    Article  MathSciNet  Google Scholar 

  8. Yuen, A., Droitcour, A., Madsen, A.H., Park, B.K., El Hourani, C., Shing, T.: U.S. Patent No. 8,454,528. U.S. Patent and Trademark Office, Washington, DC (2013)

    Google Scholar 

  9. Deng, Y., Kumar, A.: Standoff Heart Reate Estimation from Video –A Review, SPIE Defense + Commercial Sensing Expo, April 2020, Online, CA

    Google Scholar 

  10. Verkruysse, W., Svaasand, L.O., Nelson, J.S.: Remote plethysmographic imaging using ambient light. Opt. Express 16(26), 21434–21445 (2008). https://doi.org/10.1364/OE.16.021434

    Article  Google Scholar 

  11. Procházka, A., Schätz, M., Vyšata, O., Vališ, M.: Microsoft kinect visual and depth sensors for breathing and heart rate analysis. Sensors 16(7), 996 (2016)

    Article  Google Scholar 

  12. Rouast, P.V., Adam, M.T.P., Chiong, R., Cornforth, D., Lux, E.: Remote heart rate measurement using low-cost RGB face video: a technical literature review. Front. Comput. Sci. 12(5), 858–872 (2018). https://doi.org/10.1007/s11704-016-6243-6

    Article  Google Scholar 

  13. Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vision 57(2), 137–154 (2004). https://doi.org/10.1023/B:VISI.0000013087.49260.fb

    Article  Google Scholar 

  14. Lin, Y.C., Lin, Y.H.: A study of color illumination effect on the SNR of rPPG signals. In: 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 4301–4304. IEEE, July 2017

    Google Scholar 

  15. Martinez, N., Bertran, M., Sapiro, G., Wu, H.: Non-contact photoplethysmogram and instantaneous heart rate estimation from infrared face video. In: 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, pp. 2020–2024 (2019)

    Google Scholar 

  16. The COHFACE Dataset made available by the Idiap Research Institute, Martigny, Switzerland. https://www.idiap.ch/dataset/cohface

  17. Heusch, G., Anjos, A., Marcel, S.: A reproducible study on remote heart rate measurement. arXiv preprint arXiv:1709.00962 (2017)

  18. Špetlík, R., Franc, V., Matas, J.: Visual heart rate estimation with convolutional neural network. In: Proceedings of the British Machine Vision Conference, Newcastle, UK, pp. 3–6, September 2018

    Google Scholar 

  19. Alahi, A., Ortiz, R., Vandergheynst, P.: Freak: fast retina keypoint. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 510–517. IEEE, June 2012

    Google Scholar 

  20. Torr, P.H., Murray, D.W.: The development and comparison of robust methods for estimating the fundamental matrix. Int. J. Comput. Vision 24(3), 271–300 (1997). https://doi.org/10.1023/A:1007927408552

    Article  Google Scholar 

  21. Seguin, C., Blaquière, G., Loundou, A., Michelet, P., Markarian, T.: Unmanned aerial vehicles (drones) to prevent drowning. Resuscitation 127, 63–67 (2018)

    Article  Google Scholar 

  22. Pensieri, M.G., Garau, M., Barone, P.M.: Drones as an integral part of remote sensing technologies to help missing people. Drones 4(2), 15 (2020)

    Article  Google Scholar 

  23. FLIR ONE PRO User Guide, Third Gneration for Android and iOS

    Google Scholar 

  24. Carreiras, C., Alves, A.P., Lourenço, A., Canento, F., Silva, H., Fred, A., et al.: BioSPPy - Biosignal Processing in Python (2015)

    Google Scholar 

  25. De Haan, G., Jeanne, V.: Robust pulse rate from chrominance-based rPPG. IEEE Trans. Biomed. Eng. 60(10), 2878–2886 (2013)

    Article  Google Scholar 

  26. Li, X., Chen, J., Zhao, G., Pietikainen, M.: Remote heart rate measurement from face videos under realistic situations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2014)

    Google Scholar 

  27. Wang, W., Stuijk, S., De Haan, G.: A novel algorithm for remote photoplethysmography: spatial subspace rotation. IEEE Trans. Biomed. Eng. 63(9), 1974–1984 (2015)

    Article  Google Scholar 

  28. Zhong, Y., Deng, Y., Meltzner, G.: Pace independent mobile gait biometrics. In: 2015 IEEE 7th International Conference on Biometrics Theory, Applications and Systems (BTAS), September 2015

    Google Scholar 

Download references

Acknowledgment

The author of this paper would like to thank the Empower team of BAE Systems, Inc. for funding this project. Volunteers from the Empower team helped with data collection for this research. The author would also like to thank Dr. Stephen DelMarco, Mr. Derek Baker, ICPR and WAAMI reviewers for reviewing this paper.

Author information

Authors and Affiliations

  1. BAE Systems’ FAST Labs, Burlington, MA, USA

    Yunbin Deng

Authors
  1. Yunbin Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yunbin Deng .

Editor information

Editors and Affiliations

  1. Dipartimento di Ingegneria dell’Informazione, University of Firenze, Firenze, Italy

    Alberto Del Bimbo

  2. Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Modena, Italy

    Rita Cucchiara

  3. Department of Computer Science, Boston University, Boston, MA, USA

    Stan Sclaroff

  4. Dipartimento di Matematica e Informatica, University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Cloud & AI, JD.COM, Beijing, China

    Tao Mei

  6. Dipartimento di Ingegneria dell’Informazione, University of Firenze, Firenze, Italy

    Marco Bertini

  7. Computational Sciences Department, National Institute of Astrophysics, Optics and Electronics (INAOE), Tonantzintla,, Puebla, Mexico

    Hugo Jair Escalante

  8. Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Modena, Italy

    Roberto Vezzani

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Deng, Y. (2021). Remote Liveness and Heart Rate Detection from Video. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12668. Springer, Cham. https://doi.org/10.1007/978-3-030-68793-9_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-68793-9_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-68792-2

  • Online ISBN: 978-3-030-68793-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation