Skip to main content

Advertisement

Springer Nature Link
Log in

Generalized Eigenvalue Decomposition Applied to Estimation of Spatial rPPG Distribution of Skin

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Remote photoplethysmography (rPPG) has been at the forefront recently, thanks to its capacity in estimating non-contact physiological parameters such as heart rate and heart rate variability (Wang et al. in FBB 6:33, 2018). rPPG signals are typically extracted from facial videos by performing spatial averaging to obtain temporal RGB traces. Although this spatial averaging simplifies computation, it is accompanied by loss of essential spatial information which might reveal interesting relationships between signals from different spatial regions. In this article, we present a novel algorithm adapted from generalized eigenvalue decomposition (GEVD) to estimate this spatial rPPG distribution. GEVD is an extremely versatile algorithm that finds uses in signal and image processing and analytical problems such as principal component analysis and Fisher discriminant analysis (Ghojogh et al. in Tutorial 2: 1–8, 2019)(Han and Clemmensen in PR 49:43-54, 2016). It is performed using the QZ algorithm (Moler and Stewart in JNA 10(2):241–256, 2010), which in turn uses Householder transformations (Householder in JACM 5(4):339–342, 1958) to extract generalized eigenvectors of a pair of matrices. We adapt the QZ algorithm for the domain of spatio-temporal biomedical signals such as remote photoplethysmography (rPPG), electrocardiography and electroencephalography signals. We call this algorithm Temporal-QZ, which employs vectorization techniques to extract generalized eigenvectors over spatial data points simultaneously. We validate this extension in the domain of remote photoplethysmography (rPPG) measurement, for the estimation of spatial rPPG distribution of skin.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. Code and links to videos available at https://gitlab.com/rich4rd.macwan/temporal-qz.

References

  1. Aarts, L.A.M., Jeanne, V., Cleary, J.P., Lieber, C., Nelson, J.S., Bambang Oetomo, S., Verkruysse, W.: Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit: a pilot study. Early Hum. Dev. 89(12), 943–948 (2013). https://doi.org/10.1016/j.earlhumdev.2013.09.016. NIHMS150003

    Article  Google Scholar 

  2. Bobbia, S., Macwan, R., Benezeth, Y., Mansouri, A., Dubois, J.: Unsupervised skin tissue segmentation for remote photoplethysmography. Pattern Recognit. Lett. (2017). https://doi.org/10.1016/j.patrec.2017.10.017

    Article  Google Scholar 

  3. ClinicalTrialsgov-[Internet]: Can remote photoplethysmography be used for contactless vital sign acquisition in a healthcare setting? a prospective comparative study. identifier: Nct04489407 (2020). https://clinicaltrials.gov/ct2/show/study/NCT04489407

  4. De Haan, G., Jeanne, V.: Robust pulse rate from chrominance-based rPPG. IEEE Trans. Biomed. Eng. 60(10), 2878–2886 (2013). https://doi.org/10.1109/TBME.2013.2266196

    Article  Google Scholar 

  5. de Haan, G., van Leest, A.: Improved motion robustness of remote-PPG by using the blood volume pulse signature. Physiol. Meas. 35(9), 1913–1926 (2014). https://doi.org/10.1088/0967-3334/35/9/1913

    Article  Google Scholar 

  6. Douglas, R., Elliott, J.: Vectorization, Part 2: Why and What? https://www.quantifisolutions.com/vectorization-part-2-why-and-what (2017). https://www.quantifisolutions.com/vectorization-part-2-why-and-what

  7. Francis, J.G.F.: The QR transformation, a unitary analogue to LR transformation: part I. Comput. J. 4(3), 265–271 (1961)

    Article  MathSciNet  Google Scholar 

  8. Ghojogh, B., Karray, F., Crowley, M.: Eigenvalue and Generalized Eigenvalue Problems: Tutorial. arXiv (2), 1–8 (2019). arXiv:1903.11240

  9. Han, X., Clemmensen, L.: Regularized generalized eigen-decomposition with applications to sparse supervised feature extraction and sparse discriminant analysis. Pattern Recognit 49, 43–54 (2016). https://doi.org/10.1016/j.patcog.2015.07.008

    Article  MATH  Google Scholar 

  10. Hertzman, A.B.: Photoelectric plethysmography of the fingers and toes in man. Exp. Biol. Med. 37(3), 529–534 (1937). https://doi.org/10.3181/00379727-37-9630

    Article  Google Scholar 

  11. Householder, A.S.: Unitary triangularization of a nonsymmetric matrix. J. ACM 5(4), 339–342 (1958). https://doi.org/10.1145/320941.320947

    Article  MathSciNet  MATH  Google Scholar 

  12. Huang, P.W., Wu, B.J., Wu, B.F.: A Heart Rate Monitoring Framework for Real-World Drivers Using Remote Photoplethysmography. IEEE J Biomed Health Inform PP (2020)

  13. Kamshilin, A.A., Nippolainen, E., Sidorov, I.S., Vasilev, P.V., Erofeev, N.P., Podolian, N.P., Romashko, R.V.: A new look at the essence of the imaging photoplethysmography. Sci. Rep. 5, 10494 (2015). https://doi.org/10.1038/srep10494

    Article  Google Scholar 

  14. Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. CVPR (2014). https://doi.org/10.13140/2.1.1212.2243

    Article  Google Scholar 

  15. Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51(3), 455–500 (2009). https://doi.org/10.1137/07070111X. arXiv:1404.3905

    Article  MathSciNet  MATH  Google Scholar 

  16. Kranjec, J., Beguš, S., Geršak, G., Drnovšek, J.: Non-contact heart rate and heart rate variability measurements: a review. Biomed. Signal Process Control 13(1), 102–112 (2014). https://doi.org/10.1016/j.bspc.2014.03.004

    Article  Google Scholar 

  17. Kumar, M., Veeraraghavan, Ashok, Sabharwal, Ashutosh: DistancePPG: robust non-contact vital signs monitoring using a camera. Biomed. Opt. Express 6(5), 1565 (2015). https://doi.org/10.1364/BOE.6.001565. arXiv:1502.08040

    Article  Google Scholar 

  18. Latifis, I., Parashar, K., Dimitroulakos, G., Cappelle, H., Lezos, C., Masselos, K., Catthoor, F.: A MATLAB vectorizing compiler targeting application-specific instruction set processors. ACM Trans. Des. Autom. Electron. Syst. 22(2), 1–28 (2017). https://doi.org/10.1145/2996182

    Article  Google Scholar 

  19. Li, X., Chen, J., Zhao, G., Pietikainen, M., Pietik, M., Pietikäinen, M.: Remote heart rate measurement from face videos under realistic situations. In: Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit, pp. 4264–4271 (2014). https://doi.org/10.1109/CVPR.2014.543, http://openaccess.thecvf.com/content_cvpr_2014/papers/Li_Remote_Heart_Rate_2014_CVPR_paper.pdf

  20. Liang, X., Humos, A.A., Pei, T.: Vectorization and Parallelization of Loops in C / C++. In: Proc Int Conf Front Educ Comput Sci Comput Eng, pp. 203–206 (2018)

  21. Lu, H., Plataniotis, K.N., Venetsanopoulos, A.N.: Mpca: Multilinear principal component analysis of tensor objects. IEEE Trans. Neural Netw. 19(1), 18–39 (2008). https://doi.org/10.1109/TNN.2007.901277

    Article  Google Scholar 

  22. Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision (ijcai). vol. 81 (1981)

  23. Macwan, R., Benezeth, Y., Mansouri, A.: Heart rate estimation using Remote Photoplethysmography with Multi-objective Optimization. Biomed Signal Process Control (2018a)

  24. Macwan, R., Benezeth, Y., Nakamura, K., Gomez, R., Wu, Y., Mansouri, A.: Parameter-free adaptive step-size multiobjective optimization applied to remote photoplethysmography. In: Biomed. Heal. Informatics Conf., vol. 2018-Janua, pp. 267–270 (2018c). https://doi.org/10.1109/BHI.2018.8333420

  25. Macwan, R., Bobbia, S., Benezeth, Y., Dubois, J., Mansouri, A.: Periodic Variance Maximization using Generalized Eigenvalue Decomposition applied to Remote Photoplethysmography estimation. In: IEEE Conf. Comput. Vis. Pattern Recognit. Work., pp. 1445–1453 (2018d). https://doi.org/10.1109/CVPRW.2018.00181, https://hal-univ-bourgogne.archives-ouvertes.fr/hal-01830541

  26. Macwan, R., Benezeth, Y., Mansouri, A.: Remote photoplethysmography with constrained ICA using periodicity and chrominance constraints. Biomed. Eng. Online 17(1), 22 (2018b). https://doi.org/10.1186/s12938-018-0450-3

    Article  Google Scholar 

  27. MATLAB and Statistics Toolbox Release 2016b, The MathWorks, Inc, Natick, Massachusetts US (2021) Vectorization. https://fr.mathworks.com/help/matlab/matlab_prog/vectorization.html

  28. Mcduff, D.J., Estepp, J.R., Piasecki, A.M., Blackford, E.B.: A Survey of Remote Optical Photoplethysmographic Imaging Methods. In: 37th Annu. Int. Conf. IEEE EMBC, pp. 6398–6404 (2015)

  29. McDuff, D., Gontarek, S., Picard, R.: Improvements in remote cardiopulmonary measurement using a five band digital camera. IEEE Trans. Biomed. Eng. 61(10), 2593–2601 (2014). https://doi.org/10.1109/TBME.2014.2323695

    Article  Google Scholar 

  30. Moler, C.B., Stewart, G.W.: An algorithm for generalized matrix eigenvalue problems. Soc. Ind. Appl. Math. J. Numer. Anal. 10(2), 241–256 (2010)

    MathSciNet  MATH  Google Scholar 

  31. Nowara, E.M., Marks, T.K., Mansour, H., Veeraraghavan, A.: SparsePPG: Towards Driver Monitoring Using Camera-Based Vital Signs Estimation in Near-Infrared. In: IEEE Conf. CVPR (2018). http://openaccess.thecvf.com/CVPR2018_workshops/content_CVPR_2018/papers/w27/Nowara_SparsePPG_Towards_Driver_CVPR_2018_paper.pdf

  32. PD, W., CY, L.: Python for Data Analysis, vol 344. O’Reilly, Beijing (2012). https://doi.org/10.1097/MAJ.0b013e318228aef8. http://www.embase.com/search/results?subaction=viewrecord&from=export&id=L51559769%0A, arXiv:1011.1669v3

  33. Peters, G., Wilkinson, J.H.: Eigenvectors of real and complex matrices by LR and QR triangularizations. Numer. Math. 16(3), 181–204 (1970). https://doi.org/10.1007/BF02219772

    Article  MathSciNet  MATH  Google Scholar 

  34. Poh, M.Z., McDuff, D.J., Picard, R.W.: Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt. Express 18(10), 10762–10774 (2010). https://doi.org/10.1364/OE.18.010762

    Article  Google Scholar 

  35. Poh, M.Z., McDuff, D.J., Picard, R.W.: Advancements in noncontact, multiparameter physiological measurements using a webcam. IEEE Trans. Biomed. Eng. 58(1), 7–11 (2011). https://doi.org/10.1109/TBME.2010.2086456

    Article  Google Scholar 

  36. Qu, F., Wang, S., Yan, W., Li, H., Wu, S., Fu, X.: Cas(me)\(^2\) : a database for spontaneous macro-expression and micro-expression spotting and recognition. IEEE Trans. Affect. Comput. 9(4), 424–436 (2018). https://doi.org/10.1109/TAFFC.2017.2654440

    Article  Google Scholar 

  37. Sameni, R., Jutten, C., Shamsollahi, M., Electrocardiogram, M.: Multichannel electrocardiogram decomposition using periodic component analysis. IEEE Trans. Biomed. Eng. 1935–1940 (2008)

  38. Srivastav, M.: Vectorization: Writing C/C++ code in VECTOR Format (2012). https://software.intel.com/en-us/articles/vectorization-writing-cc-code-in-vector-format, https://software.intel.com/en-us/articles/vectorization-writing-cc-code-in-vector-format

  39. Tulyakov, S., Alameda-Pineda, X., Ricci, E., Yin, L., Cohn, J.F., Sebe, N., Sommarive, V., Kessler, F.B., Sommarive, V.: Self-Adaptive Matrix Completion for Heart Rate Estimation from Face Videos under Realistic Conditions. In: 2016 IEEE Conf Comput Vis Pattern Recognit, pp. 2396–2404 (2016). https://doi.org/10.1109/CVPR.2016.263, http://ieeexplore.ieee.org/document/7780632/

  40. Tursa, J.: MTIMESX: Fast Matrix Multiply with Multi-Dimensional Support (2011). https://mathworks.com/matlabcentral/fileexchange/25977-mtimesx-fast-matrix-multiply-with-multi-dimensional-support

  41. von Arx, T., Tamura, K., Yukiya, O., Lozanoff, S.: The face: a vascular perspective—a literature review. Swiss Dent. J. 128, 382–392 (2018)

    Google Scholar 

  42. Wang, W., Stuijk, S., De Haan, G.: Exploiting spatial redundancy of image sensor for motion robust rPPG. IEEE Trans. Biomed. Eng. 62(2), 415–425 (2015). https://doi.org/10.1109/TBME.2014.2356291. arXiv:15334406

    Article  Google Scholar 

  43. Wang, W., Den Brinker, A., Stuijk, S., De Haan, G.: Algorithmic principles of remote-PPG. IEEE Trans. Biomed. Eng. 64(7), 1479–1491 (2016a). https://doi.org/10.1109/TBME.2016.2609282

    Article  Google Scholar 

  44. Wang, W., Stuijk, S., De Haan, G.: A novel algorithm for remote photoplethysmography: spatial subspace rotation. IEEE Trans. Biomed. Eng. 63(9), 1974–1984 (2016b). https://doi.org/10.1109/TBME.2015.2508602

    Article  Google Scholar 

  45. Wang, W., den Brinker, A.C., Stuijk, S., de Haan, G.: Amplitude-selective filtering for remote-PPG. Biomed. Opt. Express 8(3), 1965–1980 (2017). https://doi.org/10.1364/BOE.8.001965

    Article  Google Scholar 

  46. Wang, C., Pun, T., Chanel, G.: A comparative survey of methods for remote heart rate detection from frontal face videos. Front. Bioeng. Biotechnol. 6, 33 (2018)

    Article  Google Scholar 

  47. Wilkinson, J.H.: The Algebraic Eigenvalue Problem. Oxford University Press, Oxford (1965)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ImViA, Univ. Bourgogne Franche-Comté, EA3575, Dijon, France

    Richard Macwan, Yannick Benezeth & Alamin Mansouri

Authors
  1. Richard Macwan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yannick Benezeth
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alamin Mansouri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Richard Macwan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Macwan, R., Benezeth, Y. & Mansouri, A. Generalized Eigenvalue Decomposition Applied to Estimation of Spatial rPPG Distribution of Skin. J Math Imaging Vis 63, 807–820 (2021). https://doi.org/10.1007/s10851-021-01025-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-021-01025-3