Skip to main content
SpringerLink
Log in

Assessment of physiological states from contactless face video: a sparse representation approach

  • Special Issue Article
  • Published:
Computing Aims and scope Submit manuscript

Abstract

The vital signs are estimated from remote photoplethysmography (rPPG) using the sparse representation signal reconstruction approach. The rPPG signal is used to estimate the physical parameters with the help of a non-invasive smartphone camera. This paper presents a health monitoring method by estimating vital signs using an RGB video camera and uses a pre-specified dictionary based on a hybrid discrete ridgelet transform with a Ricker wavelet basis function, to reconstruct a sparse signal prone to less error. The physical parameters such as heart rate (HR), breathing rate (BR), heart rate variability (HRV), and SpO2 are estimated using a smartphone video camera with the proposed sparse signal reconstruction technique. The inter-beat intervals (IBIs) are used to extract the power ratio in the frequency domain. Changes in HRV are more discriminative indicators of cognitive stress than those in HR and BR. The physiological states such as stress and fatigue could be measured using IBIs ratio in the frequency domain. The morning and evening dataset sessions are recruited for this experiment to check the stress and fatigue factors based on the power ratio extracted from the IBI signal. In the results, a lower mean absolute probability error value shows that the proposed method produces better results than state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. McDuff D, Gontarek S, Picard R (2014) Remote measurement of cognitive stress via heart rate variability. In: 2014 36th annual international conference of the IEEE engineering in medicine and biology society, pp 2957–2960

  2. Tarassenko L, Villarroel M, Guazzi A, Jorge J, Clifton DA, Pugh C (2014) Non-contact video-based vital sign monitoring using ambient light and auto-regressive models. Physiol Meas 35(5):807–831

    Article  Google Scholar 

  3. Kumar M, Veeraraghavan A, Sabharwal A (2015) DistancePPG: Robust non-contact vital signs monitoring using a camera. Biomed Opt Express 6(5):1565

    Article  Google Scholar 

  4. Jeong IC, Finkelstein J (2016) Introducing contactless blood pressure assessment using a high speed video camera. J Med Syst 40(4):77

    Article  Google Scholar 

  5. Couderc J-P et al (2015) Detection of atrial fibrillation using contactless facial video monitoring. Hear Rhythm 12(1):195–201

    Article  Google Scholar 

  6. Poh M-Z, McDuff DJ, Picard RW (2011) Advancements in noncontact, multiparameter physiological measurements using a webcam. IEEE Trans Biomed Eng 58(1):7–11

    Article  Google Scholar 

  7. Tasli HE, Gudi A, den Uyl M (2014) Remote PPG based vital sign measurement using adaptive facial regions. In: IEEE international conference on image processing (ICIP) vol 2014, pp 1410–1414

  8. Bousefsaf F, Maaoui C, Pruski A (2013) Remote assessment of the heart rate variability to detect mental stress. In: Proceedings of the ICTs for improving patients rehabilitation research techniques

  9. Liu S, Yuen PC, Zhang S, Zhao G (2016) 3D Mask Face Anti-spoofing with Remote Photoplethysmography, pp 85–100

  10. de Haan G, Jeanne V (2013) Robust pulse rate from chrominance-based rPPG. IEEE Trans Biomed Eng 60(10):2878–2886

    Article  Google Scholar 

  11. de Haan G, van Leest A (2014) Improved motion robustness of remote-PPG by using the blood volume pulse signature. Physiol Meas 35(9):1913–1926

    Article  Google Scholar 

  12. Wang W, den Brinker A, Stuijk S, de Haan G (2016) Algorithmic principles of remote-PPG. IEEE Trans Biomed Eng, pp 1–1

  13. Wang W, Stuijk S, de Haan G (2016) A novel algorithm for remote photoplethysmography: spatial subspace rotation. IEEE Trans Biomed Eng 63(9):1974–1984

    Article  Google Scholar 

  14. Wang W, den Brinker AC, Stuijk S, de Haan G (2017) Color-distortion filtering for remote Photoplethysmography. In: 2017 12th IEEE international conference on automatic face & gesture recognition (FG), 2017, pp 71–78

  15. Liu S, Liu D, Zhang J, Zeng Y (2011) Extraction of fetal electrocardiogram using recursive least squares and normalized least mean squares algorithms. In: 2011 3rd international conference on advanced computer control, pp 333–336

  16. Han H, Kim J (2012) Artifacts in wearable photoplethysmographs during daily life motions and their reduction with least mean square based active noise cancellation method. Comput Biol Med 42(4):387–393

    Article  Google Scholar 

  17. Torres J MM, Ghosh A, Stepanov EA, Riccardi G (2016) Heal-T: an efficient PPG-based heart-rate and IBI estimation method during physical exercise. In: 2016 24th European signal processing conference (EUSIPCO), pp 1438–1442

  18. Temko A (2015) Estimation of heart rate from photoplethysmography during physical exercise using Wiener filtering and the phase vocoder. In: 2015 37th annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 1500–1503

  19. Zhang X, Ding Q (2017) Respiratory rate estimation from the photoplethysmogram via joint sparse signal reconstruction and spectra fusion. Biomed Signal Process Control 35:1–7

    Article  Google Scholar 

  20. Gorodnitsky IF, Rao BD (1997) Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm. IEEE Trans Signal Process 45(3):600–616

    Article  Google Scholar 

  21. Zhang X, Ding Q (2016) Respiratory rate monitoring from the photoplethysmogram via sparse signal reconstruction. Physiol Meas 37(7)

  22. Zhang Z, Pi Z, Liu B (2015) TROIKA: a general framework for heart rate monitoring using wrist-type photoplethysmographic signals during intensive physical exercise. IEEE Trans Biomed Eng 62(2):522–531

    Article  Google Scholar 

  23. Qayyum A, Malik AS, Nuafal M, Mazher M, Ahmad RF, Abdullah MF (2015) Evaluation of optimization algorithms for sparse and redundant dictionaries. IEEE Stud Symp Biomed Eng Sci 2015:128–133

    Google Scholar 

  24. Zhang Z (2015) Photoplethysmography-based heart rate monitoring in physical activities via joint sparse spectrum reconstruction. IEEE Trans Biomed Eng 62(8):1902–1910

    Article  MathSciNet  Google Scholar 

  25. Rao BD, Engan K, Cotter SF, Palmer J, Kreutz-Delgado K (2003) Subset selection in noise based on diversity measure minimization. IEEE Trans Signal Process 51(3):760–770

    Article  Google Scholar 

  26. Pati Y C, Rezaiifar R, Krishnaprasad PS Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In: Proceedings of 27th Asilomar conference on signals, systems and computers, pp 40–44

  27. Davenport MA, Wakin MB (2010) Analysis of orthogonal matching pursuit using the restricted isometry property. IEEE Trans Inf Theory 56(9):4395–4401

    Article  MathSciNet  MATH  Google Scholar 

  28. Cai TT, Wang L (2011) Orthogonal matching pursuit for sparse signal recovery with noise. IEEE Trans Inf Theory 57(7):4680–4688

    Article  MathSciNet  MATH  Google Scholar 

  29. Sungjun K, Hyunseok K, Kwang SP (2012) Validation of heart rate extraction using video imaging on a built-in camera system of a smartphone. In: Annual international conference of the IEEE engineering in medicine and biology society 2012, pp 2174–2177

  30. Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154

    Article  Google Scholar 

  31. Phung SL, Bouzerdoum A, Chai D (2002) A novel skin color model in YCbCr color space and its application to human face detection. In: Proceedings of international conference on image processing, vol 1, pp I-289-I-292

  32. McDuff D, Blackford E, Estepp J (2017) Fusing partial camera signals for non-contact pulse rate variability measurement. IEEE Trans Biomed Eng, pp 1–1

  33. Aznaveh M M, Mirzaei H, Roshan E, Saraee M.H (2009) A new and improved skin detection method using mixed color space, pp 471–480

  34. Wang W, den Brinker AC, Stuijk S, de Haan G (2017) Robust heart rate from fitness videos. Physiol Meas 38(6):1023–1044

    Article  Google Scholar 

  35. Wiede C, Richter J, Hirtz G (2016) Signal fusion based on intensity and motion variations for remote heart rate determination. In: IEEE international conference on imaging systems and techniques (IST) 2016, pp 526–531

  36. Siddiqui SA, Zhang Y, Feng Z, Kos A (2016) A pulse rate estimation algorithm using PPG and smartphone camera. J Med Syst 40(5):126

    Article  Google Scholar 

  37. Scully CG et al (2012) Physiological parameter monitoring from optical recordings with a mobile phone. IEEE Trans Biomed Eng 59(2):303–306

    Article  Google Scholar 

  38. Baheti PK, Garudadri H (2009) An ultra low power pulse oximeter sensor based on compressed sensing. In: Sixth international workshop on wearable and implantable body sensor networks 2009, pp 144–148

  39. Castells F, Laguna P, Sörnmo L, Bollmann A, Roig JM (2007) Principal component analysis in ECG signal processing. EURASIP J Adv Signal Process 1:74580

    Article  MATH  Google Scholar 

  40. Tobaldini E, Nobili L, Strada S, Casali KR, Braghiroli A, Montano N (2013) Heart rate variability in normal and pathological sleep. Physiol Front

  41. Vicente J, Laguna P, Bartra A, Bailón R (2016) Drowsiness detection using heart rate variability. Med Biol Eng Comput 54(6):927–937

    Article  Google Scholar 

  42. Patel M, Lal SKL, Kavanagh D, Rossiter P (2011) Applying neural network analysis on heart rate variability data to assess driver fatigue. Expert Syst Appl 38(6):7235–7242

    Article  Google Scholar 

  43. Qayyum A, Razzak I (2020) Deep residual neural network for child’s spontaneous facial expressions recognition. In S+ SSPR, pp 282-291

  44. Qayyum A, Razzak I, Mumtaz W (2020) Hybrid deep shallow network for assessment of depression using electroencephalogram signals. In: International conference on neural information processing, pp 245–257. Springer, Cham

  45. Rehman A, Naz S, Razzak I (2021) Leveraging big data analytics in healthcare enhancement: trends, challenges and opportunities. Multimed Syst, pp 1–33

  46. Razzak MI, Imran M, Xu G (2020) Big data analytics for preventive medicine. Neural Comput Appl 32(9):4417–4451

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Universiti de Bourgogne, Dijon, France

    Abdul Qayyum

  2. ENIB, UMR CNRS 6285 LabSTICC, 29238, Brest, France

    Moona Mazher

  3. DEIM, Universitat Rovira i Virgili, Tarragona, Spain

    Aliyu Nuhu

  4. Department of Electrical and Electronics Engineering, University of Jos, Jos, Nigeria

    Abdesslam Benzinou

  5. Department of Electrical and Electronics Engineering, Centre for Intelligent Signal and Imaging Research (CISIR), Universiti Teknologi Petronas, Seri Iskandar, Perak, Malaysia

    Aamir Saeed Malik

  6. School of Information Technology, Deakin University, Geelong, Australia

    Imran Razzak

Authors
  1. Abdul Qayyum
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Moona Mazher
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aliyu Nuhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Abdesslam Benzinou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Aamir Saeed Malik
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Imran Razzak
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Imran Razzak.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qayyum, A., Mazher, M., Nuhu, A. et al. Assessment of physiological states from contactless face video: a sparse representation approach. Computing 105, 761–781 (2023). https://doi.org/10.1007/s00607-021-01028-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00607-021-01028-3

Keywords

Mathematics Subject Classification

Search

Navigation