Skip to main content

Advertisement

SpringerLink
Log in

AutoDep: automatic depression detection using facial expressions based on linear binary pattern descriptor

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

The psychological health of a person plays an important role in their daily life activities. The paper addresses depression issues with the machine learning model using facial expressions of the patient. Some research has already been done on visual based on depression detection methods, but those are illumination variant. The paper uses feature extraction using LBP (Local Binary Pattern) descriptor, which is illumination invariant. The Viola-Jones algorithm is used for face detection and SVM (support vector machine) is considered for classification along with the LBP descriptor to make a complete model for depression level detection. The proposed method captures frontal face from the videos of subjects and their facial features are extracted from each frame. Subsequently, the facial features are analyzed to detect depression levels with the post-processing model. The performance of the proposed system is evaluated using machine learning algorithms in MATLAB. For the real-time system design, it is necessary to test it on the hardware platform. The LBP descriptor has been implemented on FPGA using Xilinx VIVADO 16.4. The results of the proposed method show satisfactory performance and accuracy for depression detection comparison with similar previous work.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. WH Organization et al (2017) Depression and other common mental disorders: global health estimates. Technical Report, World Health Organization

  2. Singh OP (2018) Closing treatment gap of mental disorders in India: Opportunity in new competency-based medical council of India curriculum. Indian J psychiatry 60:375

    PubMed  PubMed Central  Google Scholar 

  3. Cao Z, Lin C-T, Ding W, Chen M-H, Li C-T, Su T-P (2018) Identifying ketamine responses in treatment-resistant depression using a wearable forehead eeg. IEEE Trans Biomed Eng 66:1668–1679

    Article  Google Scholar 

  4. Zhu Y, Shang Y, Shao Z, Guo G (2017) Automated depression diagnosis based on deep networks to encode facial appearance and dynamics. IEEE Trans Affect Comput 9:578–584

    Article  Google Scholar 

  5. Uguz F, Yakut E, Aydogan S, Bayman MG, Gezginc K (2019) The impact of maternal major depression, anxiety disorders and their comorbidities on gestational age, birth weight, preterm birth and low birth weight in newborns. J Affect disord 259:382– 385

    Article  Google Scholar 

  6. Zhao K, So H-C (2018) Drug repositioning for schizophrenia and depression/anxiety disorders: A machine learning approach leveraging expression data. IEEE J Biomed Health Inform 23:1304–1315

    Article  Google Scholar 

  7. Pawluski JL, Lonstein JS, Fleming AS (2017) The neurobiology of postpartum anxiety and depression. Trends Neurosci 40:106– 120

    Article  CAS  Google Scholar 

  8. Peng H, Xia C, Wang Z, Zhu J, Zhang X, Sun S, Li J, Huo X, Li X (2019) Multivariate pattern analysis of eeg-based functional connectivity: A study on the identification of depression. IEEE Access 7:92630–92641

    Article  Google Scholar 

  9. Pancholi S, Joshi AM (2018) Portable emg data acquisition module for upper limb prosthesis application. IEEE Sensors J 18:3436–3443

    Article  Google Scholar 

  10. Bhadoria RS, Bajpai D (2019) Stabilizing sensor data collection for control of environment-friendly clean technologies using internet of things. Wirel Pers Commun 108:493–510

    Article  Google Scholar 

  11. Bhurane AA, Bhadoria RS (2019) Behavioral biometrics: A prognostic measure for activity recognition. In: The biometric computing: recognition and registration, p 71

  12. Pathak Y, Arya K, Tiwari S (2019) Feature selection for image steganalysis using levy flight-based grey wolf optimization. Multimed Tools Appl 78:1473–1494

    Article  Google Scholar 

  13. Pancholi S, Jain P, Varghese A, et al. (2019) A novel time-domain based feature for emg-pr prosthetic and rehabilitation application. In: 2019 41st annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, pp 5084–5087

  14. Likforman-Sulem L, Esposito A, Faundez-Zanuy M, Clémençon S, Cordasco G (2017) Emothaw: A novel database for emotional state recognition from handwriting and drawing. IEEE Trans Human-Mach Syst 47:273–284

    Article  Google Scholar 

  15. Nasir M, Jati A, Shivakumar PG, Nallan Chakravarthula S, Georgiou P (2016) Multimodal and multiresolution depression detection from speech and facial landmark features. In: Proceedings of the 6th international workshop on audio/visual emotion challenge, pp 43–50

  16. Ackermann P, Kohlschein C, Bitsch JÁ, Wehrle K, Jeschke S Eeg-based automatic emotion recognition: Feature extraction, selection and classification methods, IEEE

  17. Sharma G, Parashar A, Joshi AM (2021) Dephnn: A novel hybrid neural network for electroencephalogram (eeg)-based screening of depression. Biomed Signal Process Control 66:102393

    Article  Google Scholar 

  18. Subhani AR, Mumtaz W, Saad MNBM, Kamel N, Malik AS (2017) Machine learning framework for the detection of mental stress at multiple levels. IEEE Access 5:13545–13556

    Article  Google Scholar 

  19. Girard JM, Cohn JF, Mahoor MH, Mavadati S, Rosenwald DP (2013) Social risk and depression: Evidence from manual and automatic facial expression analysis. In: 2013 10th IEEE international conference and workshops on automatic face and gesture recognition (FG). IEEE, pp 1–8

  20. He L, Jiang D, Sahli H (2015) Multimodal depression recognition with dynamic visual and audio cues, IEEE

  21. Deshpande M, Rao V (2017) Depression detection using emotion artificial intelligence. In: 2017 international conference on intelligent sustainable systems (ICISS). IEEE, pp 858–862

  22. Chao L, Tao J, Yang M, Li Y (2015) Multi task sequence learning for depression scale prediction from video. In: 2015 international conference on affective computing and intelligent interaction. IEEE, ACII, pp 526–531

  23. Zhou X, Jin K, Shang Y, Guo G (2018) Visually interpretable representation learning for depression recognition from facial images. IEEE Trans Affect Comput

  24. De Melo WC, Granger E, Hadid A (2019) Depression detection based on deep distribution learning. In: 2019 IEEE international conference on image processing (ICIP). IEEE, pp 4544–4548

  25. de Melo WC, Granger E, Hadid A (2020) A deep multiscale spatiotemporal network for assessing depression from facial dynamics. IEEE Trans Affect Comput

  26. Liu J-Q, Huang Y, Huang X-Y, Xia X-T, Niu X-X, Lin L, Chen Y-W (2020) Dynamic facial features in positive-emotional speech for identification of depressive tendencies. In: Innovation in Medicine and Healthcare. Springer, pp 127–134

  27. Low C-Y, Teoh AB-J, Ng C-J (2017) Multi-fold gabor, pca, and ica filter convolution descriptor for face recognition. IEEE Trans Circuits Syst Video Technol 29:115–129

    Article  Google Scholar 

  28. Kotsia I, Pitas I (2006) Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Trans Image Process 16:172–187

    Article  Google Scholar 

  29. Ghimire D, Jeong S, Lee J, Park SH (2017) Facial expression recognition based on local region specific features and support vector machines. Multimed Tools Appl 76:7803–7821

    Article  Google Scholar 

  30. Zhang Y-D, Yang Z-J, Lu H-M, Zhou X-X, Phillips P, Liu Q-M, Wang S-H (2016) Facial emotion recognition based on biorthogonal wavelet entropy, fuzzy support vector machine, and stratified cross validation. IEEE Access 4:8375–8385

    Article  Google Scholar 

  31. Tikoo S, Malik N (2017) Detection of face using Viola Jones and recognition using back propagation neural network. arXiv:1701.08257

  32. Mujahid O, Ullah Z, Mahmood H, Hafeez A (2018) Fast pattern recognition through an lbp driven cam on fpga. IEEE Access 6:39525–39531

    Article  Google Scholar 

  33. Jain P, Joshi AM, Agrawal N, Mohanty S (2020) Iglu 2.0: A new non-invasive, accurate serum glucometer for smart healthcare. arXiv:2001.09182

  34. Pancholi S, Joshi AM (2019) Time derivative moments based feature extraction approach for recognition of upper limb motions using emg. IEEE Sensors Lett 3:1–4

    Google Scholar 

  35. Joshi AM, Bramha A (2020) Vlsi architecture of block matching algorithms for motion estimation in high efficiency video coding. Wirel Pers Commun :1–16

  36. Kyrkou C, Bouganis C-S, Theocharides T, Polycarpou MM (2015) Embedded hardware-efficient real-time classification with cascade support vector machines. IEEE Trans Neural Netw Learn Syst 27:99–112

    Article  Google Scholar 

  37. Lekdioui K, Messoussi R, Ruichek Y, Chaabi Y, Touahni R (2017) Facial decomposition for expression recognition using texture/shape descriptors and svm classifier. Signal Process Image Commun 58:300–312

    Article  Google Scholar 

  38. Tong Z, Chen X, He Z, Tong K, Fang Z, Wang X (2018) Emotion recognition based on photoplethysmogram and electroencephalogram. In: 2018 IEEE 42nd annual computer software and applications conference (COMPSAC), vol 02, pp 402–407

  39. Murtaza M, Sharif M, AbdullahYasmin M, Ahmad T (2019) Facial expression detection using six facial expressions hexagon (sfeh) model. In: 2019 IEEE 9th annual computing and communication workshop and conference (CCWC), pp 0190–0195

  40. Dham S, Sharma A, Dhall A (2017) Depression scale recognition from audio, visual and text analysis. arXiv:1709.05865

  41. Peng Z, Hu Q, Dang-Zanuy J (2019) Multi-kernel svm based depression recognition using social media data. Int J Mach Learn Cybern 10:43–57

    Article  CAS  Google Scholar 

  42. Pampouchidou A, Marias K, Tsiknakis M, Simos P, Yang F, Meriaudeau F (2015) Designing a framework for assisting depression severity assessment from facial image analysis. In: 2015 IEEE international conference on signal and image processing applications (ICSIPA), pp 578–583

  43. Xiang J, Zhu G (2017) Joint face detection and facial expression recognition with mtcnn. In: 2017 4th international conference on information science and control engineering (ICISCE), pp 424–427

  44. Tong Z, Chen X, He Z, Tong K, Fang Z, Wang X (2018) Emotion recognition based on photoplethysmogram and electroencephalogram. In: 2018 IEEE 42nd annual computer software and applications conference (COMPSAC), vol 2, pp 402–407

  45. Mantri S, Patil D, Agrawal P, Wadhai V (2015) Non invasive eeg signal processing framework for real time depression analysis. In: 2015 SAI intelligent systems conference (IntelliSys). IEEE, pp 518–521

Download references

Acknowledgements

The author be thankful to SMDP-C2SD Lab, MNIT-Jaipur and Ministry of Human Resource Development (MHRD), Govt. of India to provide the support of CAD Tools to carry out the experiments. The results of the paper are carried out in SMDP-C2SD Lab, MNIT-Jaipur.

Author information

Authors and Affiliations

  1. Malaviya National Institute of Technology, Jaipur, India

    Manjunath Tadalagi & Amit M. Joshi

Authors
  1. Manjunath Tadalagi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amit M. Joshi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amit M. Joshi.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tadalagi, M., Joshi, A.M. AutoDep: automatic depression detection using facial expressions based on linear binary pattern descriptor. Med Biol Eng Comput 59, 1339–1354 (2021). https://doi.org/10.1007/s11517-021-02358-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-021-02358-2

Keywords

Search

Navigation