Skip to main content

HR-TRACK: An rPPG Method for Heartrate Monitoring Using Temporal Convolution Networks

  • Conference paper
  • First Online:
Pattern Recognition (ICPR 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15313))

Included in the following conference series:

  • 114 Accesses

Abstract

The COVID-19 pandemic necessitates avoiding skin contact to minimize the spread of virus infection. It paves the way for an active surge in telehealthcare research. In this direction, Remote Photoplethysmography (rPPG) plays a crucial role in analyzing heart rate (HR) from non-contact face videos. Existing rPPG-based HR monitoring methods fail when face video duration is small and the video contains facial deformations. These issues are mitigated by our proposed method HR-TRACK, that is, rPPG method for Heart Rate moniToring using tempoRAl Convolution networK. It improves HR monitoring by introducing a novel architecture formed by sequentially stacking two novel networks. The networks are inspired by the temporal convolution network (TCN) to model long temporal sequences effectively. Our first network automatically mitigates the noise induced by facial deformations and performs blind source separation to predict pulse signals. The instantaneous HR obtained from the pulse signal can be erroneous. Thus, our second network analyzes all the computed HR values and rectifies the erroneous HR, if any. The experimental results conducted on the publicly available datasets reveal that our proposed method outperforms the state-of-the-art methods. Furthermore, the results justify the utilization of both networks to improve HR monitoring.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/lokendra7/rPPG-Publically.

References

  1. Balakrishnan, G., Durand, F., Guttag, J.: Detecting pulse from head motions in video. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3430–3437 (2013)

    Google Scholar 

  2. Birla, L., Gupta, P.: PATRON: exploring respiratory signal derived from non-contact face videos for face anti-spoofing. Expert Syst. Appl. 187, 115883 (2021)

    Article  Google Scholar 

  3. Birla, L., Shukla, S., Gupta, A.K., Gupta, P.: ALPINE: improving remote heart rate estimation using contrastive learning. In: IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5029–5038 (2023)

    Google Scholar 

  4. Bobbia, S., Macwan, R., Benezeth, Y., Mansouri, A., Dubois, J.: Unsupervised skin tissue segmentation for remote photoplethysmography. Pattern Recogn. Lett. 124, 82–90 (2019)

    Article  Google Scholar 

  5. Chen, W., McDuff, D.: DeepPhys: video-based physiological measurement using convolutional attention networks. In: European Conference on Computer Vision, pp. 349–365 (2018)

    Google Scholar 

  6. Ciftci, U.A., Demir, I., Yin, L.: FakeCatcher: detection of synthetic portrait videos using biological signals. IEEE Trans. Pattern Anal. Mach. Intell. (2020). https://doi.org/10.1109/TPAMI.2020.3009287

  7. De Haan, G., Jeanne, V.: Robust pulse rate from chrominance-based rPPG. IEEE Trans. Biomed. Eng. 60(10), 2878–2886 (2013)

    Article  Google Scholar 

  8. Gupta, A.K., Gupta, P., Rahtu, E.: FATALRead-fooling visual speech recognition models. Appl. Intell. 52, 9001–9016 (2021)

    Article  Google Scholar 

  9. Gupta, A.K., Kumar, R., Birla, L., Gupta, P.: RADIANT: better rPPG estimation using signal embeddings and transformer. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 4976–4986 (2023)

    Google Scholar 

  10. Gupta, P.: MERASTC: micro-expression recognition using effective feature encodings and 2D convolutional neural network. IEEE Trans. Affect. Comput. 14, 1431–1441 (2021)

    Article  Google Scholar 

  11. Gupta, P., Bhowmick, B., Pal, A.: Accurate heart-rate estimation from face videos using quality-based fusion. In: IEEE International Conference on Image Processing, pp. 4132–4136 (2017)

    Google Scholar 

  12. Gupta, P., Bhowmick, B., Pal, A.: Serial fusion of eulerian and lagrangian approaches for accurate heart-rate estimation using face videos. In: International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 2834–2837 (2017)

    Google Scholar 

  13. Gupta, P., Bhowmick, B., Pal, A.: Exploring the feasibility of face video based instantaneous heart-rate for micro-expression spotting. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1316–1323 (2018)

    Google Scholar 

  14. Gupta, P., Bhowmick, B., Pal, A.: MOMBAT: heart rate monitoring from face video using pulse modeling and bayesian tracking. Comput. Biol. Med. 121, 103813 (2020)

    Article  Google Scholar 

  15. Gupta, P., Bhowmik, B., Pal, A.: Robust adaptive heart-rate monitoring using face videos. In: IEEE Winter Conference on Applications of Computer Vision, pp. 530–538 (2018)

    Google Scholar 

  16. Heusch, G., Anjos, A., Marcel, S.: A reproducible study on remote heart rate measurement. arXiv preprint arXiv:1709.00962 (2017)

  17. Kuang, H., Ao, C., Ma, X., Liu, X.: Shuffle-rPPGNet: efficient network with global context for remote heart rate variability measurement. IEEE Sensors J. 23, 15199–15209 (2023)

    Article  Google Scholar 

  18. Lee, D., Kim, J., Kwon, S., Park, K.: Heart rate estimation from facial photoplethysmography during dynamic illuminance changes. In: International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 2758–2761 (2015)

    Google Scholar 

  19. Lee, E., Chen, E., Lee, C.Y.: Meta-rPPG: remote heart rate estimation using a transductive meta-learner. In: European Conference on Computer Vision, pp. 392–409. Springer (2020)

    Google Scholar 

  20. Lokendra, B., Puneet, G.: AND-rPPG: a novel denoising-rPPG network for improving remote heart rate estimation. Comput. Biol. Med. 141, 105146 (2021)

    Article  Google Scholar 

  21. Macwan, R., Benezeth, Y., Mansouri, A.: Heart rate estimation using remote photoplethysmography with multi-objective optimization. Biomed. Signal Process. Control 49, 24–33 (2019)

    Article  Google Scholar 

  22. Mirabet-Herranz, N., Mallat, K., Dugelay, J.L.: Deep learning for remote heart rate estimation: a reproducible and optimal state-of-the-art framework. In: International Conference on Pattern Recognition, pp. 558–573 (2022)

    Google Scholar 

  23. Moghadam, M.C., Masoumi, E., Kendale, S., Bagherzadeh, N.: Predicting hypotension in the ICU using noninvasive physiological signals. Comput. Biol. Med. 129, 104120 (2021)

    Article  Google Scholar 

  24. Nooralishahi, P., Loo, C.K., Shiung, L.W.: Robust remote heart rate estimation from multiple asynchronous noisy channels using autoregressive model with kalman filter. Biomed. Signal Process. Control 47, 366–379 (2019)

    Article  Google Scholar 

  25. Nowara, E., McDuff, D., Veeraraghavan, A.: The benefit of distraction: denoising remote vitals measurements using inverse attention. arXiv preprint arXiv:2010.07770 (2020)

  26. Odinaev, I., Wong, K.L., Chin, J.W., Goyal, R., Chan, T.T., So, R.H.: Robust heart rate variability measurement from facial videos. Bioengineering 10(7), 851 (2023)

    Article  Google Scholar 

  27. Parsi, A., Glavin, M., Jones, E., Byrne, D.: Prediction of paroxysmal atrial fibrillation using new heart rate variability features. Comput. Biol. Med. 133, 104367 (2021)

    Article  Google Scholar 

  28. Poh, M.Z., McDuff, D.J., Picard, R.W.: Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt. Express 18, 10762–10774 (2010)

    Article  Google Scholar 

  29. Qiu, Y., Liu, Y., Arteaga-Falconi, J., Dong, H., El Saddik, A.: EVM-CNN: real-time contactless heart rate estimation from facial video. IEEE Trans. Multimedia 21, 1778–1787 (2018)

    Article  Google Scholar 

  30. Rodriguez, A.M., Ramos-Castro, J.: Video pulse rate variability analysis in stationary and motion conditions. Biomed. Eng. Online 17, 1–26 (2018)

    Google Scholar 

  31. Saikia, T., Birla, L., Gupta, A.K., Gupta, P.: HREADAI: heart rate estimation from face mask videos by consolidating eulerian and lagrangian approaches. IEEE Trans. Instrum. Meas. 73, 1–11 (2023)

    Article  Google Scholar 

  32. Spetlik, R., Franc, V., Matas, J.: Visual heart rate estimation with convolutional neural network. In: British Machine Vision Conference, pp. 3–6 (2018)

    Google Scholar 

  33. Tarvainen, M.P., Ranta-Aho, P.O., Karjalainen, P.A.: An advanced detrending method with application to HRV analysis. IEEE Trans. Biomed. Eng. 49, 172–175 (2002)

    Article  Google Scholar 

  34. Tasli, H.E., Gudi, A., den Uyl, M.: Remote PPG based vital sign measurement using adaptive facial regions. In: IEEE International Conference on Image Processing, pp. 1410–1414 (2014)

    Google Scholar 

  35. Verkruysse, W., Svaasand, L.O., Nelson, J.S.: Remote plethysmographic imaging using ambient light. Opt. Express 16, 21434–21445 (2008)

    Article  Google Scholar 

  36. Yang, M., Liu, J., Xiao, Y., Liao, H.: 14.4 nW fourth-order bandpass filter for biomedical applications. Electron. Lett. 46, 973–974 (2010)

    Article  Google Scholar 

  37. Yu, Z., Li, X., Zhao, G.: Remote photoplethysmograph signal measurement from facial videos using spatio-temporal networks. In: British Machine Vision Conference, p. 277 (2019)

    Google Scholar 

  38. Zadeh, A., Chong Lim, Y., Baltrusaitis, T., Morency, L.P.: Convolutional experts constrained local model for 3D facial landmark detection. In: IEEE International Conference on Computer Vision Workshops, pp. 2519–2528 (2017)

    Google Scholar 

  39. Zhang, Z., Sabuncu, M.R.: Generalized cross entropy loss for training deep neural networks with noisy labels. In: Neural Information Processing Systems (2018)

    Google Scholar 

Download references

Acknowledgment

The authors are thankful to all those researchers who have provided us the access to COHFACE and UBFC-rPPG datasets. This work of Trishna Saikia is partially supported by the Prime Minister’s Research Fellowship (PMRF), the Ministry of Education, and the Government of India (2102743).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Trishna Saikia .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Birla, L., Shukla, S., Saikia, T., Gupta, P. (2025). HR-TRACK: An rPPG Method for Heartrate Monitoring Using Temporal Convolution Networks. In: Antonacopoulos, A., Chaudhuri, S., Chellappa, R., Liu, CL., Bhattacharya, S., Pal, U. (eds) Pattern Recognition. ICPR 2024. Lecture Notes in Computer Science, vol 15313. Springer, Cham. https://doi.org/10.1007/978-3-031-78201-5_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-78201-5_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-78200-8

  • Online ISBN: 978-3-031-78201-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics