Skip to main content

Advertisement

SpringerLink
Log in

Fully automated age-weighted expression classification using real and apparent age

  • Industrial and Commercial Application
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

After decades of research, automatic facial expression recognition (AFER) has been shown to work well when restricted to subjects with a limited range of ages. Expression recognition in subjects having a large range of ages is harder as it has been shown that ageing, health, and lifestyle affect facial expression. In this paper, we present a discriminative system that explicitly predicts expression across a large range of ages, which we show to perform better than an equivalent system which ignores age. In our system, we first build a fully automatic facial feature point detector (FFPD) using random forest regression voting in a constrained local mode (RFRV-CLM) framework (Cootes et al., in: European conference on computer vision, Springer, Berlin, 2012) which we use to automatically detect the location of key facial points, study the effect of ageing on the accuracy of point localization task. Second, a set of age group estimator and age-specific expression recognizers are trained from the extracted features that include shape, texture, appearance and a fusion of shape with texture, to analyse the effect of ageing on the face features and subsequently on the performance of AFER. We then propose a simple and effective method to recognize the expression across a large range of ages through using a weighted combination rule of a set of age group estimator and age specific expression recognizers (one for each age group), where the age information is used as prior knowledge to the expression classification. The advantage of using the weighted combination of all the classifiers is that more information about the classification can be obtained and subjects whose apparent age puts them in the wrong chronological age group will be dealt with more effectively. The performance of the proposed system was evaluated using three age-expression databases of static and dynamic images for deliberate and spontaneous expressions: FACES (Ebner et al., in Behav Res Methods 42:351–362, 2010) (2052 images), Lifespan (Minear and Park in Behav Res Methods Instrum Comput 36:630–633, 2004) (844 images) and NEMO (Dibeklioğlu et al., in: European conference on computer vision, Springer, Berlin, 2012) (1,243 videos). The results show the system to be accurate and robust against a wide variety of expressions and the age of the subject. Evaluation of point localization, age group estimation and expression recognition against ground truth data was obtained and compared with the existing results of alternative approaches tested on the same data. The quantitative results with 2.1% error rates (using manual points) and 3.0% error rates (fully automatic) of expression classification demonstrated that the results of our novel system were encouraging in comparison with the state-of-the-art systems which ignore age and alternative models recently applied to the problem.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Adolphs R, Tranel D (2004) Impaired judgments of sadness but not happiness following bilateral amygdala damage. J Cogn Neurosci 16(3):453–462

    Article  Google Scholar 

  2. Ahonen T, Hadid A, Pietikäinen M (2004) Face recognition with local binary patterns. Comput Vision-ECCV 2004:469–481

    MATH  Google Scholar 

  3. Alnajar F, Lou Z, Álvarez JM, Gevers T, et al (2014) Expression-invariant age estimation. In: BMVC

  4. Antipov G, Baccouche M, Berrani SA, Dugelay JL (2016) Apparent age estimation from face images combining general and children-specialized deep learning models. In: proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 96–104

  5. Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  Google Scholar 

  6. Bromiley P, Adams J, Cootes T (2015) Localisation of vertebrae on dxa images using constrained local models with random forest regression voting. In: Yao J, Glocker B, Klinder T, Li S (eds) Recent advances in computational methods and clinical applications for spine imaging. Springer, Cham, pp 159–171

    Chapter  Google Scholar 

  7. Bromiley PA, Adams JE, Cootes TF (2015) Automatic localisation of vertebrae in dxa images using random forest regression voting. In: international workshop on computational methods and clinical applications for spine imaging, pp. 38–51. Springer

  8. Bromiley PA, Kariki EP, Adams JE, Cootes TF (2016) Fully automatic localisation of vertebrae in ct images using random forest regression voting. In: international workshop on computational methods and clinical applications for spine imaging, pp. 51–63. Springer

  9. Clapés A, Bilici O, Temirova D, Avots E, Anbarjafari G, Escalera S (2018) From apparent to real age: gender, age, ethnic, makeup, and expression bias analysis in real age estimation. In: proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 2373–2382

  10. Cohn JF, Kruez TS, Matthews I, Yang Y, Nguyen MH, Padilla MT, Zhou F, De la Torre F (2009) Detecting depression from facial actions and vocal prosody. In: 3rd international conference on affective computing and intelligent interaction and workshops, ACII 2009. pp. 1–7. IEEE

  11. Cootes TF, Edwards GJ, Taylor CJ (2001) Active appearance models. IEEE Trans Pattern Anal Mach Intell 23(6):681–685

    Article  Google Scholar 

  12. Cootes TF, Ionita MC, Lindner C, Sauer P (2012) Robust and accurate shape model fitting using random forest regression voting. In: European conference on computer vision, pp. 278–291. Springer

  13. Dibeklioğlu H, Salah AA, Gevers T (2012) Are you really smiling at me? spontaneous versus posed enjoyment smiles. In: European conference on computer vision, pp. 525–538. Springer

  14. Dibeklioğlu H, Salah AA, Gevers T (2015) Recognition of genuine smiles. IEEE Trans Multimed 17(3):279–294

    Article  Google Scholar 

  15. Dodgson NA (2004) Variation and extrema of human interpupillary distance. In: stereoscopic displays and virtual reality systems XI, vol. 5291, pp. 36–47. International society for optics and photonics

  16. Ebner NC, Johnson MK (2010) Age-group differences in interference from young and older emotional faces. Cognit Emotion 24(7):1095–1116

    Article  Google Scholar 

  17. Ebner NC, Riediger M, Lindenberger U (2010) Faces-a database of facial expressions in young, middle-aged, and older women and men: development and validation. Beh Res Methods 42(1):351–362

    Article  Google Scholar 

  18. Fölster M, Hess U, Werheid K (2014) Facial age affects emotional expression decoding. Front Psychol 5:30

    Article  Google Scholar 

  19. Goldman AI, Sripada CS (2005) Simulationist models of face-based emotion recognition. Cognition 94(3):193–213

    Article  Google Scholar 

  20. Guo G, Guo R, Li X (2013) Facial expression recognition influenced by human aging. IEEE Trans Affect Comput 4(3):291–298

    Article  Google Scholar 

  21. Hess U, Adams RB, Simard A, Stevenson MT, Kleck RE (2012) Smiling and sad wrinkles: age-related changes in the face and the perception of emotions and intentions. J Exp Soc Psychol 48(6):1377–1380

    Article  Google Scholar 

  22. Houstis O, Kiliaridis S (2009) Gender and age differences in facial expressions. Eur J Orthodont 31(5):459–466

    Article  Google Scholar 

  23. Huo Z, Yang X, Xing C, Zhou Y, Hou P, Lv J, Geng X (2016) Deep age distribution learning for apparent age estimation. In: proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 17–24

  24. Kanade T, Cohn JF, Tian Y (2000) Comprehensive database for facial expression analysis. In: proceedings fourth IEEE international conference on automatic face and gesture recognition (Cat. No. PR00580), pp. 46–53. IEEE

  25. Lawrence AD, Calder AJ (2004) Homologizing human emotions

  26. Lindner C, Bromiley PA, Ionita MC, Cootes TF (2015) Robust and accurate shape model matching using random forest regression-voting. IEEE Trans Pattern Anal Mach Intell 37(9):1862–1874

    Article  Google Scholar 

  27. Lindner C, Cootes T (2015) Fully automatic cephalometric evaluation using random forest regression-voting. In: proceedings of the IEEE international symposium on biomedical imaging (ISBI) 2015–grand challenges in dental X-ray image analysis–automated detection and analysis for diagnosis in cephalometric X-ray image. Citeseer

  28. Lindner C, Thiagarajah S, Wilkinson J, Consortium T (2013) Fully automatic segmentation of the proximal femur using random forest regression voting. IEEE Trans Med Imaging 32(8):1462–1472

    Article  Google Scholar 

  29. Lou Z, Alnajar F, Alvarez JM, Hu N, Gevers T (2018) Expression-invariant age estimation using structured learning. IEEE Trans Pattern Anal Mach Intell 40(2):365–375

    Article  Google Scholar 

  30. Lyons M, Akamatsu S, Kamachi M, Gyoba J (1998) Coding facial expressions with gabor wavelets. In: proceedings third IEEE international conference on automatic face and gesture recognition, 1998, pp. 200–205. IEEE

  31. Martinez B, Valstar MF (2016) Advances, challenges, and opportunities in automatic facial expression recognition. In: Kawulok M, Celebi E, Smolka B (eds) Advances in face detection and facial image analysis. Springer, Berlin, pp 63–100

    Google Scholar 

  32. Mary R, Jayakumar T (2016) A review on how human aging influences facial expression recognition (fer). In: Abraham A, Haqiq A, Muda AK, Gandhi N (eds) Innovations in bio-inspired computing and applications. Springer, Berlin, pp 313–322

    Chapter  Google Scholar 

  33. Minear M, Park DC (2004) A lifespan database of adult facial stimuli. Beh Res Methods Instruments Comput 36(4):630–633

    Article  Google Scholar 

  34. Nguyen DT, Cho SR, Shin KY, Bang JW, Park KR (2014) Comparative study of human age estimation with or without preclassification of gender and facial expression. Sci World J. https://doi.org/10.1155/2014/905269

    Article  Google Scholar 

  35. Phillips LH, Allen R (2004) Adult aging and the perceived intensity of emotions in faces and stories. Aging Clin Exp Res 16(3):190–199

    Article  Google Scholar 

  36. Prkachin KM, Solomon PE (2008) The structure, reliability and validity of pain expression: evidence from patients with shoulder pain. Pain 139(2):267–274

    Article  Google Scholar 

  37. Rothe R, Timofte R, Van Gool L (2015) Dex: deep expectation of apparent age from a single image. In: proceedings of the IEEE international conference on computer vision workshops, pp. 10–15

  38. Sariyanidi E, Gunes H, Cavallaro A (2015) Automatic analysis of facial affect: a survey of registration, representation, and recognition. IEEE Trans Pattern Anal Mach Intell 37(6):1113–1133

    Article  Google Scholar 

  39. Sawant MM, Bhurchandi KM (2019) Age invariant face recognition: a survey on facial aging databases, techniques and effect of aging. Artif Intell Rev 52(2):981–1008

    Article  Google Scholar 

  40. Uřičař M, Timofte R, Rothe R, Matas J, et al (2016) Structured output svm prediction of apparent age, gender and smile from deep features. In: proceedings of the 29th IEEE conference on computer vision and pattern recognision workshop (CVPRW 2016), pp. 730–738. IEEE

  41. Vural E, Cetin M, Ercil A, Littlewort G, Bartlett M, Movellan J (2007) Drowsy driver detection through facial movement analysis. In: international workshop on human-computer interaction, pp. 6–18. Springer

  42. Wang S, Wu S, Gao Z, Ji Q (2016) Facial expression recognition through modeling age-related spatial patterns. Multimed Tools Appl 75(7):3937–3954

    Article  Google Scholar 

  43. Whitehill J, Bartlett M, Movellan J (2008) Automatic facial expression recognition for intelligent tutoring systems. In: IEEE computer society conference on computer vision and pattern recognition workshops, 2008. CVPRW’08, pp. 1–6. IEEE

  44. Yang HF, Lin BY, Chang KY, Chen CS (2018) Joint estimation of age and expression by combining scattering and convolutional networks. ACM Trans Multimed Comput Commun Appl (TOMM) 14(1):1–18

    Article  Google Scholar 

  45. Zhu Y, Li Y, Mu G, Guo G (2015) A study on apparent age estimation. In: proceedings of the IEEE international conference on computer vision workshops, pp. 25–31

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Faculty of Education for Girls, University of Kufa, Najaf, Iraq

    Nora Al-Garaawi

  2. Department of Computer Science, The University of Manchester, Manchester, United Kingdom

    Tim Morris

  3. Division of Informatics, Imaging and Data Science, The University of Manchester, Manchester, United Kingdom

    Timothy F. Cootes

Authors
  1. Nora Al-Garaawi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tim Morris
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Timothy F. Cootes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nora Al-Garaawi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Al-Garaawi, N., Morris, T. & Cootes, T.F. Fully automated age-weighted expression classification using real and apparent age. Pattern Anal Applic 25, 451–466 (2022). https://doi.org/10.1007/s10044-021-01044-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-021-01044-1

Keywords

Search

Navigation