Abstract
Heart Rate (HR) is one of the most important indicators reflecting the physiological state of the human body, and more researches have begun to focus on remote HR measurement in order to meet the challenging but practical non-contact requirements. Existing remote HR estimation methods rely on the high-resolution input signals constructed from low-resolution Spatial-Temporal Map (STMap) of facial sequences, but most of them use simple linear projection, which are difficult to capture the complex temporal and spatial relationships in between weak raw signals. To address this problem, we propose a Hierarchical Attentive Upsampling Module (HAUM) to obtain rich and discriminating input signals from STMap for accurate HR estimation. Our approach includes two parts: (1) a Hierarchical Upsampling Strategy (HUS) for progressively enriching the spatial-temporal information, and (2) an Attentive Space Module (ASM) to focus the model on more discriminating HR signal regions with clearer periodicity. The experiments performed on two public datasets VIPL-HR and MAHNOB-HCI show that the proposed approach achieves the state-of-the-art performance.
This work was supported by the National Science Fund of China under Grant Nos. 61876083.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
The position of ECG sensor is upper left corner of chest and under clavicle bone.
- 3.
- 4.
References
Brüser, C., Antink, C.H., Wartzek, T., Walter, M., Leonhardt, S.: Ambient and unobtrusive cardiorespiratory monitoring techniques. IEEE Rev. Biomed. Eng. 8, 30–43 (2015)
Chen, W., McDuff, D.: DeepPhys: video-based physiological measurement using convolutional attention networks. In: Proceedings of IEEE ECCV, pp. 349–365 (2018)
De Haan, G., Jeanne, V.: Robust pulse rate from chrominance-based rPPG. IEEE Trans. Biomed. Eng. 60(10), 2878–2886 (2013)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of IEEE CVPR, pp. 770–778 (2016)
Heusch, G., Anjos, A., Marcel, S.: A reproducible study on remote heart rate measurement. arXiv preprint arXiv:1709.00962 (2017)
Hsu, G.S., Ambikapathi, A., Chen, M.S.: Deep learning with time-frequency representation for pulse estimation from facial videos. In: 2017 IEEE International Joint Conference on Biometrics, pp. 383–389. IEEE (2017)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of IEEE CVPR, pp. 7132–7141 (2018)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)
Jain, P.K., Tiwari, A.K.: Heart monitoring systems-a review. Comput. Biol. Med. 54, 1–13 (2014)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Kwon, S., Kim, J., Lee, D., Park, K.: ROI analysis for remote photoplethysmography on facial video. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 4938–4941. IEEE (2015)
Lam, A., Kuno, Y.: Robust heart rate measurement from video using select random patches. In: ICCV, pp. 3640–3648 (2015)
Lee, E., Chen, E., Lee, C.Y.: Meta-RPPG: remote heart rate estimation using a transductive meta-learner. In: Proceedings of IEEE ECCV (2020)
Lempe, G., Zaunseder, S., Wirthgen, T., Zipser, S., Malberg, H.: ROI selection for remote photoplethysmography. In: Meinzer, H.P., Deserno, T., Handels, H., Tolxdorff, T. (eds.) Bildverarbeitung für die Medizin 2013, pp. 99–103. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36480-8_19
Li, X., Wang, W., Hu, X., Yang, J.: Selective kernel networks. In: Proceedings of IEEE CVPR, pp. 510–519 (2019)
Li, X., Chen, J., Zhao, G., Pietikainen, M.: Remote heart rate measurement from face videos under realistic situations. In: Proceedings of IEEE CVPR, pp. 4264–4271 (2014)
McDuff, D.: Deep super resolution for recovering physiological information from videos. In: CVPR Workshops, pp. 1367–1374 (2018)
Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: ICML (2010)
Niu, X., Han, H., Shan, S., Chen, X.: VIPL-HR: a multi-modal database for pulse estimation from less-constrained face video. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) ACCV 2018. LNCS, vol. 11365, pp. 562–576. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20873-8_36
Niu, X., Shan, S., Han, H., Chen, X.: RhythmNet: end-to-end heart rate estimation from face via spatial-temporal representation. TIP 29, 2409–2423 (2019)
Niu, X., Yu, Z., Han, H., Li, X., Shan, S., Zhao, G.: Video-based remote physiological measurement via cross-verified feature disentangling. In: Proceedings of IEEE ECCV (2020)
Niu, X., Zhao, X., Han, H., Das, A., Dantcheva, A., Shan, S., Chen, X.: Robust remote heart rate estimation from face utilizing spatial-temporal attention. In: 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition, pp. 1–8. IEEE (2019)
Poh, M.Z., McDuff, D.J., Picard, R.W.: Advancements in noncontact, multiparameter physiological measurements using a webcam. IEEE Trans. Biomed. Eng. 58(1), 7–11 (2010)
Poh, M.Z., McDuff, D.J., Picard, R.W.: Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt. Exp. 18(10), 10762–10774 (2010)
Qiu, Y., Liu, Y., Arteaga-Falconi, J., Dong, H., El Saddik, A.: EVM-CNN: real-time contactless heart rate estimation from facial video. IEEE Trans. Multimed. 21(7), 1778–1787 (2018)
Sabokrou, M., Pourreza, M., Li, X., Fathy, M., Zhao, G.: Deep-HR: fast heart rate estimation from face video under realistic conditions. arXiv preprint arXiv:2002.04821 (2020)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Soleymani, M., Lichtenauer, J., Pun, T., Pantic, M.: A multimodal database for affect recognition and implicit tagging. IEEE Trans. Affect. Comput. 3(1), 42–55 (2011)
Tsou, Y.Y., Lee, Y.A., Hsu, C.T., Chang, S.H.: Siamese-rPPG network: remote photoplethysmography signal estimation from face videos. In: Proceedings of the 35th Annual ACM Symposium on Applied Computing, pp. 2066–2073 (2020)
Verkruysse, W., Svaasand, L.O., Nelson, J.S.: Remote plethysmographic imaging using ambient light. Opt. Express 16(26), 21434–21445 (2008)
Wang, W., den Brinker, A.C., Stuijk, S., de Haan, G.: Algorithmic principles of remote PPG. IEEE Trans. Biomed. Eng. 64(7), 1479–1491 (2016)
Woo, S., Park, J., Lee, J.Y., So Kweon, I.: CBAM: convolutional block attention module. In: Proceedings of IEEE ECCV, pp. 3–19 (2018)
Yan, Y., Ma, X., Yao, L., Ouyang, J.: Noncontact measurement of heart rate using facial video illuminated under natural light and signal weighted analysis. Bio-Med. Mater. Eng. 26(s1), S903–S909 (2015)
Yu, Z., Peng, W., Li, X., Hong, X., Zhao, G.: Remote heart rate measurement from highly compressed facial videos: an end-to-end deep learning solution with video enhancement. In: ICCV, pp. 151–160 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, P., Li, X., Qian, J., Jin, Z., Yang, J. (2022). Hierarchical Attentive Upsampling on Input Signals for Remote Heart Rate Estimation. In: Wallraven, C., Liu, Q., Nagahara, H. (eds) Pattern Recognition. ACPR 2021. Lecture Notes in Computer Science, vol 13189. Springer, Cham. https://doi.org/10.1007/978-3-031-02444-3_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-02444-3_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-02443-6
Online ISBN: 978-3-031-02444-3
eBook Packages: Computer ScienceComputer Science (R0)