Abstract
The COVID-19 pandemic necessitates avoiding skin contact to minimize the spread of virus infection. It paves the way for an active surge in telehealthcare research. In this direction, Remote Photoplethysmography (rPPG) plays a crucial role in analyzing heart rate (HR) from non-contact face videos. Existing rPPG-based HR monitoring methods fail when face video duration is small and the video contains facial deformations. These issues are mitigated by our proposed method HR-TRACK, that is, rPPG method for Heart Rate moniToring using tempoRAl Convolution networK. It improves HR monitoring by introducing a novel architecture formed by sequentially stacking two novel networks. The networks are inspired by the temporal convolution network (TCN) to model long temporal sequences effectively. Our first network automatically mitigates the noise induced by facial deformations and performs blind source separation to predict pulse signals. The instantaneous HR obtained from the pulse signal can be erroneous. Thus, our second network analyzes all the computed HR values and rectifies the erroneous HR, if any. The experimental results conducted on the publicly available datasets reveal that our proposed method outperforms the state-of-the-art methods. Furthermore, the results justify the utilization of both networks to improve HR monitoring.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Balakrishnan, G., Durand, F., Guttag, J.: Detecting pulse from head motions in video. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3430–3437 (2013)
Birla, L., Gupta, P.: PATRON: exploring respiratory signal derived from non-contact face videos for face anti-spoofing. Expert Syst. Appl. 187, 115883 (2021)
Birla, L., Shukla, S., Gupta, A.K., Gupta, P.: ALPINE: improving remote heart rate estimation using contrastive learning. In: IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5029–5038 (2023)
Bobbia, S., Macwan, R., Benezeth, Y., Mansouri, A., Dubois, J.: Unsupervised skin tissue segmentation for remote photoplethysmography. Pattern Recogn. Lett. 124, 82–90 (2019)
Chen, W., McDuff, D.: DeepPhys: video-based physiological measurement using convolutional attention networks. In: European Conference on Computer Vision, pp. 349–365 (2018)
Ciftci, U.A., Demir, I., Yin, L.: FakeCatcher: detection of synthetic portrait videos using biological signals. IEEE Trans. Pattern Anal. Mach. Intell. (2020). https://doi.org/10.1109/TPAMI.2020.3009287
De Haan, G., Jeanne, V.: Robust pulse rate from chrominance-based rPPG. IEEE Trans. Biomed. Eng. 60(10), 2878–2886 (2013)
Gupta, A.K., Gupta, P., Rahtu, E.: FATALRead-fooling visual speech recognition models. Appl. Intell. 52, 9001–9016 (2021)
Gupta, A.K., Kumar, R., Birla, L., Gupta, P.: RADIANT: better rPPG estimation using signal embeddings and transformer. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 4976–4986 (2023)
Gupta, P.: MERASTC: micro-expression recognition using effective feature encodings and 2D convolutional neural network. IEEE Trans. Affect. Comput. 14, 1431–1441 (2021)
Gupta, P., Bhowmick, B., Pal, A.: Accurate heart-rate estimation from face videos using quality-based fusion. In: IEEE International Conference on Image Processing, pp. 4132–4136 (2017)
Gupta, P., Bhowmick, B., Pal, A.: Serial fusion of eulerian and lagrangian approaches for accurate heart-rate estimation using face videos. In: International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 2834–2837 (2017)
Gupta, P., Bhowmick, B., Pal, A.: Exploring the feasibility of face video based instantaneous heart-rate for micro-expression spotting. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1316–1323 (2018)
Gupta, P., Bhowmick, B., Pal, A.: MOMBAT: heart rate monitoring from face video using pulse modeling and bayesian tracking. Comput. Biol. Med. 121, 103813 (2020)
Gupta, P., Bhowmik, B., Pal, A.: Robust adaptive heart-rate monitoring using face videos. In: IEEE Winter Conference on Applications of Computer Vision, pp. 530–538 (2018)
Heusch, G., Anjos, A., Marcel, S.: A reproducible study on remote heart rate measurement. arXiv preprint arXiv:1709.00962 (2017)
Kuang, H., Ao, C., Ma, X., Liu, X.: Shuffle-rPPGNet: efficient network with global context for remote heart rate variability measurement. IEEE Sensors J. 23, 15199–15209 (2023)
Lee, D., Kim, J., Kwon, S., Park, K.: Heart rate estimation from facial photoplethysmography during dynamic illuminance changes. In: International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 2758–2761 (2015)
Lee, E., Chen, E., Lee, C.Y.: Meta-rPPG: remote heart rate estimation using a transductive meta-learner. In: European Conference on Computer Vision, pp. 392–409. Springer (2020)
Lokendra, B., Puneet, G.: AND-rPPG: a novel denoising-rPPG network for improving remote heart rate estimation. Comput. Biol. Med. 141, 105146 (2021)
Macwan, R., Benezeth, Y., Mansouri, A.: Heart rate estimation using remote photoplethysmography with multi-objective optimization. Biomed. Signal Process. Control 49, 24–33 (2019)
Mirabet-Herranz, N., Mallat, K., Dugelay, J.L.: Deep learning for remote heart rate estimation: a reproducible and optimal state-of-the-art framework. In: International Conference on Pattern Recognition, pp. 558–573 (2022)
Moghadam, M.C., Masoumi, E., Kendale, S., Bagherzadeh, N.: Predicting hypotension in the ICU using noninvasive physiological signals. Comput. Biol. Med. 129, 104120 (2021)
Nooralishahi, P., Loo, C.K., Shiung, L.W.: Robust remote heart rate estimation from multiple asynchronous noisy channels using autoregressive model with kalman filter. Biomed. Signal Process. Control 47, 366–379 (2019)
Nowara, E., McDuff, D., Veeraraghavan, A.: The benefit of distraction: denoising remote vitals measurements using inverse attention. arXiv preprint arXiv:2010.07770 (2020)
Odinaev, I., Wong, K.L., Chin, J.W., Goyal, R., Chan, T.T., So, R.H.: Robust heart rate variability measurement from facial videos. Bioengineering 10(7), 851 (2023)
Parsi, A., Glavin, M., Jones, E., Byrne, D.: Prediction of paroxysmal atrial fibrillation using new heart rate variability features. Comput. Biol. Med. 133, 104367 (2021)
Poh, M.Z., McDuff, D.J., Picard, R.W.: Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt. Express 18, 10762–10774 (2010)
Qiu, Y., Liu, Y., Arteaga-Falconi, J., Dong, H., El Saddik, A.: EVM-CNN: real-time contactless heart rate estimation from facial video. IEEE Trans. Multimedia 21, 1778–1787 (2018)
Rodriguez, A.M., Ramos-Castro, J.: Video pulse rate variability analysis in stationary and motion conditions. Biomed. Eng. Online 17, 1–26 (2018)
Saikia, T., Birla, L., Gupta, A.K., Gupta, P.: HREADAI: heart rate estimation from face mask videos by consolidating eulerian and lagrangian approaches. IEEE Trans. Instrum. Meas. 73, 1–11 (2023)
Spetlik, R., Franc, V., Matas, J.: Visual heart rate estimation with convolutional neural network. In: British Machine Vision Conference, pp. 3–6 (2018)
Tarvainen, M.P., Ranta-Aho, P.O., Karjalainen, P.A.: An advanced detrending method with application to HRV analysis. IEEE Trans. Biomed. Eng. 49, 172–175 (2002)
Tasli, H.E., Gudi, A., den Uyl, M.: Remote PPG based vital sign measurement using adaptive facial regions. In: IEEE International Conference on Image Processing, pp. 1410–1414 (2014)
Verkruysse, W., Svaasand, L.O., Nelson, J.S.: Remote plethysmographic imaging using ambient light. Opt. Express 16, 21434–21445 (2008)
Yang, M., Liu, J., Xiao, Y., Liao, H.: 14.4 nW fourth-order bandpass filter for biomedical applications. Electron. Lett. 46, 973–974 (2010)
Yu, Z., Li, X., Zhao, G.: Remote photoplethysmograph signal measurement from facial videos using spatio-temporal networks. In: British Machine Vision Conference, p. 277 (2019)
Zadeh, A., Chong Lim, Y., Baltrusaitis, T., Morency, L.P.: Convolutional experts constrained local model for 3D facial landmark detection. In: IEEE International Conference on Computer Vision Workshops, pp. 2519–2528 (2017)
Zhang, Z., Sabuncu, M.R.: Generalized cross entropy loss for training deep neural networks with noisy labels. In: Neural Information Processing Systems (2018)
Acknowledgment
The authors are thankful to all those researchers who have provided us the access to COHFACE and UBFC-rPPG datasets. This work of Trishna Saikia is partially supported by the Prime Minister’s Research Fellowship (PMRF), the Ministry of Education, and the Government of India (2102743).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Birla, L., Shukla, S., Saikia, T., Gupta, P. (2025). HR-TRACK: An rPPG Method for Heartrate Monitoring Using Temporal Convolution Networks. In: Antonacopoulos, A., Chaudhuri, S., Chellappa, R., Liu, CL., Bhattacharya, S., Pal, U. (eds) Pattern Recognition. ICPR 2024. Lecture Notes in Computer Science, vol 15313. Springer, Cham. https://doi.org/10.1007/978-3-031-78201-5_24
Download citation
DOI: https://doi.org/10.1007/978-3-031-78201-5_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-78200-8
Online ISBN: 978-3-031-78201-5
eBook Packages: Computer ScienceComputer Science (R0)