Skip to main content
SpringerLink
Log in

Dep-Emotion: Suppressing Uncertainty to Recognize Real Emotions in Depressed Patients

  • Conference paper
  • First Online:
Pattern Recognition, Computer Vision, and Image Processing. ICPR 2022 International Workshops and Challenges (ICPR 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13643))

Included in the following conference series:

  • 420 Accesses

Abstract

In depression, affective and emotional dysfunction are important components of the clinical syndrome. At present, doctors mainly judge the real emotions of depressed patients through the naked eye, with a strong subjective consciousness. We collected images of seven expressions voluntarily imitated by 168 subjects, and then recruited 9 raters to recognize these images. The study found that depressed patients have deficits in Facial Emotion Expression, resulting in great uncertainty in their facial expressions. Therefore, we propose the Dep-Emotion to solve this problem. For the depression expression dataset with uncertainty, we use Self-Cure Network to correct the sample label to suppress the uncertainty. At the same time, the input part and downsampling block of ResNet18 are adjusted to better extract facial features. The input image is regularized by Cutout, which enhances the generalization ability of the model. The results show that Dep-Emotion achieves the best accuracy of 40.0%. The study has important implications for automatic emotion analysis and adjunctive treatment of depression.

Supported by Qilu University of Technology (Shandong Academy of Sciences).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Arnold, A.J., Winkielman, P.: Smile (but only deliberately) though your heart is aching: loneliness is associated with impaired spontaneous smile mimicry. Soc. Neurosci. 1, 1–13 (2020)

    Google Scholar 

  2. Association, A., American, P.A.: Diagnostic and statistical manual of mental disorders. Essentials Pain Med. 51(1), 4–8 (1980)

    Google Scholar 

  3. Azadi, S., Feng, J., Jegelka, S., Darrell, T.: Auxiliary image regularization for deep CNNs with noisy labels. arXiv preprint arXiv:1511.07069 (2015)

  4. Barsoum, E., Zhang, C., Ferrer, C.C., Zhang, Z.: Training deep networks for facial expression recognition with crowd-sourced label distribution, pp. 279–283 (2016)

    Google Scholar 

  5. Darwin, C.: The expression of emotions in man and animals. Tredition Classics 123(1), 146 (2009)

    Google Scholar 

  6. Dehghani, M., Severyn, A., Rothe, S., Kamps, J.: Avoiding your teacher’s mistakes: training neural networks with controlled weak supervision. arXiv preprint arXiv:1711.00313 (2017)

  7. DeVries, T., Taylor, G.W.: Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552 (2017)

  8. Dhall, A., Goecke, R., Ghosh, S., Joshi, J., Hoey, J., Gedeon, T.: From individual to group-level emotion recognition: Emotiw 5, 524–528 (2017)

    Google Scholar 

  9. Dhall, A., Goecke, R., Joshi, J., Hoey, J., Gedeon, T.: Emotiw 2016: video and group-level emotion recognition challenges, pp. 427–432 (2016)

    Google Scholar 

  10. Dhall, A., Ramana Murthy, O., Goecke, R., Joshi, J., Gedeon, T.: Video and image based emotion recognition challenges in the wild: Emotiw 2015, 423–426 (2015)

    Google Scholar 

  11. Dimberg, U.: Facial reactions to facial expressions. Psychophysiology 19(6), 643–647 (1982)

    Article  Google Scholar 

  12. Donahue, J., et al.: Decaf: a deep convolutional activation feature for generic visual recognition, 647–655 (2014)

    Google Scholar 

  13. Dosovitskiy, A., et al.: An image is worth 16\(\times \)16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

  14. Ekman, P.: Strong evidence for universals in facial expressions: a reply to Russell’s mistaken critique. Psychol. Bull. 115(2), 268–287 (1994)

    Article  Google Scholar 

  15. Ekman, P.: Constants across culture in the face and emotion. J. Pers. Soc. Psychol. 17 (1971)

    Google Scholar 

  16. Fasel, B.: Head-pose invariant facial expression recognition using convolutional neural networks. In: Proceedings of the Fourth IEEE International Conference on Multimodal Interfaces, pp. 529–534. IEEE (2002)

    Google Scholar 

  17. Fasel, B.: Robust face analysis using convolutional neural networks 2, 40–43 (2002)

    Google Scholar 

  18. Gaebel, W., WöLwer, W.: Facial expression and emotional face recognition in schizophrenia and depression 242(1), 46–52 (1992)

    Google Scholar 

  19. Gessler, S.: Schizophrenic inability to judge facial emotion: a controlled study. British J. Clin. Psychol. 28 (2011)

    Google Scholar 

  20. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation, pp. 580–587 (2014)

    Google Scholar 

  21. Goodfellow, I.J., et al.: Challenges in representation learning: a report on three machine learning contests. In: Lee, M., Hirose, A., Hou, Z.-G., Kil, R.M. (eds.) ICONIP 2013. LNCS, vol. 8228, pp. 117–124. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-42051-1_16

    Chapter  Google Scholar 

  22. Hamilton, M.: The Hamilton rating scale for depression. Springer (1986). https://doi.org/10.1007/978-3-030-22009-9_826

  23. He, T., Zhang, Z., Zhang, H., Zhang, Z., Xie, J., Li, M.: Bag of tricks for image classification with convolutional neural networks, pp. 558–567 (2019)

    Google Scholar 

  24. Li, J., et al.: Facial expression recognition with faster R-CNN. Procedia Comput. Sci. 107, 135–140 (2017)

    Article  Google Scholar 

  25. Li, S., Deng, W., Du, J.: Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild, pp. 2852–2861 (2017)

    Google Scholar 

  26. Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression, pp. 94–101 (2010)

    Google Scholar 

  27. Matsugu, M., Mori, K., Mitari, Y., Kaneda, Y.: Subject independent facial expression recognition with robust face detection using a convolutional neural network. Neural Netw. 16(5–6), 555–559 (2003)

    Article  Google Scholar 

  28. Minaee, S., Minaei, M., Abdolrashidi, A.: Deep-emotion: Facial expression recognition using attentional convolutional network. Sensors 21(9), 3046 (2021)

    Article  Google Scholar 

  29. Mnih, V., Hinton, G.E.: Learning to label aerial images from noisy data, pp. 567–574 (2012)

    Google Scholar 

  30. Natarajan, N., Dhillon, I.S., Ravikumar, P., Tewari, A.: Learning with noisy labels. In: Advances in Neural Information Processing Systems, vol. 26, pp. 1196–1204 (2013)

    Google Scholar 

  31. Shan, C., Gong, S., Mcowan, P.W.: Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis. Comput. 27(6), 803–816 (2009)

    Article  Google Scholar 

  32. Shan, L., Deng, W.: Deep facial expression recognition: a survey. IEEE Trans. Affect. Comput. (99) (2018)

    Google Scholar 

  33. Sharif Razavian, A., Azizpour, H., Sullivan, J., Carlsson, S.: CNN features off-the-shelf: an astounding baseline for recognition, pp. 806–813 (2014)

    Google Scholar 

  34. Sukhbaatar, S., Fergus, R.: Learning from noisy labels with deep neural networks. arXiv preprint arXiv:1406.20802(3), 4 (2014)

  35. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. IEEE, pp. 2818–2826 (2016)

    Google Scholar 

  36. Tian, Y.I., Kanade, T., Cohn, J.F.: Recognizing action units for facial expression analysis. IEEE Trans. Pattern Anal. Mach. Intell. 23(2), 97–115 (2001)

    Article  Google Scholar 

  37. Tracy, A., Weightman, M.J., Baune, B.T.: Symptom severity of depressive symptoms impacts on social cognition performance in current but not remitted major depressive disorder. Front. Psychol. 6, 1118 (2015)

    Google Scholar 

  38. Veit, A., Alldrin, N., Chechik, G., Krasin, I., Gupta, A., Belongie, S.: Learning from noisy large-scale datasets with minimal supervision, pp. 839–847 (2017)

    Google Scholar 

  39. Vick, S.J., Waller, B.M., Parr, L.A., Smith Pasqualini, M.C., Bard, K.A.: A cross-species comparison of facial morphology and movement in humans and chimpanzees using the facial action coding system (facs). J. Nonverbal Behav. 31(1), 1–20 (2007)

    Article  Google Scholar 

  40. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features 1, I-I (2001)

    Google Scholar 

  41. Wang, K., Peng, X., Yang, J., Lu, S., Qiao, Y.: Suppressing uncertainties for large-scale facial expression recognition, pp. 6897–6906 (2020)

    Google Scholar 

  42. Wexler, B.E., Levenson, L., Warrenburg, S., Price, L.H.: Decreased perceptual sensitivity to emotion-evoking stimuli in depression. Psych. Res. 51(2), 127 (1994)

    Article  Google Scholar 

  43. Wood, A., Rychlowska, M., Korb, S., Niedenthal, P.: Fashioning the face: sensorimotor simulation contributes to facial expression recognition. Trends Cogn. Sci. 20(3), 227–240 (2016)

    Article  Google Scholar 

  44. Zeng, J., Shan, S., Chen, X.: Facial expression recognition with inconsistently annotated datasets, pp. 222–237 (2018)

    Google Scholar 

  45. Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29, 915–928 (2007)

    Article  Google Scholar 

  46. Zhi, R., Flierl, M., Ruan, Q., Kleijn, W.B.: Graph-preserving sparse nonnegative matrix factorization with application to facial expression recognition. IEEE Trans. Syst. Man Cybern. B Cybern. 41(1), 38–52 (2011)

    Article  Google Scholar 

  47. Zhong, L., Liu, Q., Yang, P., Liu, B., Huang, J., Metaxas, D.N.: Learning active facial patches for expression analysis, pp. 2562–2569 (2012)

    Google Scholar 

Download references

Acknowledgements

This work was supported by the Shandong Provincial Natural Science Foundation, China (Grant No: ZR2021MF079, ZR2020MF039). The National Natural Science Foundation of China (Grant No: 81573829). The 20 Planned Projects in Jinan (No.2021GXRC046). The Key Research and Development Program of Shandong Province (Grant No.2020CXGC010901).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology Qilu University of Technology (Shandong Academy of Sciences), Jinan, 250353, China

    Gang Fu, Jiayu Ye & Qingxiang Wang

Authors
  1. Gang Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiayu Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qingxiang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qingxiang Wang .

Editor information

Editors and Affiliations

  1. York University, Toronto, ON, Canada

    Jean-Jacques Rousseau

  2. Ontario Tech University, Oshawa, ON, Canada

    Bill Kapralos

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Fu, G., Ye, J., Wang, Q. (2023). Dep-Emotion: Suppressing Uncertainty to Recognize Real Emotions in Depressed Patients. In: Rousseau, JJ., Kapralos, B. (eds) Pattern Recognition, Computer Vision, and Image Processing. ICPR 2022 International Workshops and Challenges. ICPR 2022. Lecture Notes in Computer Science, vol 13643. Springer, Cham. https://doi.org/10.1007/978-3-031-37660-3_46

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-37660-3_46

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-37659-7

  • Online ISBN: 978-3-031-37660-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation