Skip to main content
SpringerLink
Log in

Jointly learning distribution and expectation in a unified framework for facial age and attractiveness estimation

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Label distribution learning achieved promising results on ordinal regression tasks such as facial age and attractiveness estimation, especially using deep label distribution learning (DLDL) methods, introducing the label distribution learning into deep convolutional neural networks. However, existing DLDL methods have an inconsistency between the training objectives and the evaluation metric, so they may be suboptimal. In addition, these methods always adopt image classification or face recognition models with a large amount of parameters, which carry expensive computation cost and storage overhead. In this paper, we firstly analyze the essential relationship between two state-of-the-art methods (ranking CNN and DLDL) and show that the ranking method is in fact learning label distribution implicitly. This result thus firstly unifies two existing popular state-of-the-art methods into the DLDL framework. Second, in order to alleviate the inconsistency and reduce resource consumption, we design a lightweight network architecture and propose a unified framework which can jointly learn label distribution and regress expectation value. The effectiveness of our approach has been demonstrated on typical ordinal regression tasks including facial age and attractiveness estimation. Our method achieves new state-of-the-art results using the single model with 36× fewer parameters and 3× faster inference speed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

All datasets, ChaLearn15 [1], ChaLearn16 [23], Morph [25], UTKFace [24], SCUT-FBP [26] and CFD [27] in this study are included in these published articles ([1, 23,24,25,26] and [27]).

Notes

  1. 0.5 compression rate means every Conv layer has only 50% channels as that in VGG-16.

  2. https://www.csie.ntu.edu.tw/~cjlin/liblinear/.

  3. http://ldl.herokuapp.com/download.

References

  1. Escalera S, Fabian J, Pardo P, Baró" X, Gonzalez J, Escalante HJ, Misevic D, Steiner U, Guyon I (2015) ChaLearn looking at people 2015: apparent age and cultural event recognition datasets and results. In: Proceeding of the IEEE international conference on computer vision workshop

  2. Deng Y, Loy CC, Tang X (2017) Image aesthetic assessment: an experimental survey. IEEE Signal Proces Mag 34(4):80–106

    Article  Google Scholar 

  3. Lebedeva I, Guo Y, Ying F (2022) Mebeauty: a multi-ethnic facial beauty dataset in-the-wild. Neural Comput Appl 34(17):14169–14183

    Article  Google Scholar 

  4. Schwarz A, Haurilet M, Martinez M, Stiefelhagen R (2017) DriveAHead: a Large-scale driver head pose dataset. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops

  5. Li X, Zhao H, Wang Z, Yu Z (2020) Research on movie rating prediction algorithms. In: 2020 5th IEEE international conference on big data analytics (ICBDA)

  6. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) ImageNet: a large-scale hierarchical image database. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  7. Parkhi OM, Vedaldi A, Zisserman A et al (2015) Deep face recognition. In: Proceedings of the British machine vision conference

  8. Guo Y, Zhang L, Hu Y, He X, Gao J (2016) MS-Celeb-1M: a dataset and benchmark for large-scale face recognition. In: Proceedings of the European conference on computer vision

  9. Shakeel MS, Lam K-M (2019) Deep-feature encoding-based discriminative model for age-invariant face recognition. Pattern Recognit 93:442–457

    Article  Google Scholar 

  10. Ranjan R, Sankaranarayanan S, Castillo CD, Chellappa R (2017) An all-in-one convolutional neural network for face analysis. In: Proceedings of the IEEE conference on automatic face and gesture recognition

  11. Rothe R, Timofte R, Van Gool L (2016) Deep expectation of real and apparent age from a single image without facial landmarks. Int J Comput Vis 126(2):144–157

    MathSciNet  Google Scholar 

  12. Gao B-B, Xing C, Xie C-W, Wu J, Geng X (2017) Deep label distribution learning with label ambiguity. IEEE Trans Image Process 26(6):2825–2838

    Article  MathSciNet  MATH  Google Scholar 

  13. Niu Z, Zhou M, Wang L, Gao X, Hua G (2016) Ordinal regression with multiple output CNN for age estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  14. Chen S, Zhang C, Dong M, Le J, Rao M (2017) Using ranking-CNN for age estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  15. Chen S, Zhang C, Dong M (2017) Deep age estimation: from classification to ranking. IEEE Trans Multimed 20(8):2209–2222

    Article  Google Scholar 

  16. Fan Y-Y, Liu S, Li B, Guo Z, Samal A, Wan J, Li SZ (2017) Label distribution-based facial attractiveness computation by deep residual learning. IEEE Trans Multimed 20(8):2196–2208

    Article  Google Scholar 

  17. Rodríguez P, Cucurull G, Gonfaus JM, Roca FX, Gonzalez J (2017) Age and gender recognition in the wild with deep attention. Pattern Recognit 72:563–571

    Article  Google Scholar 

  18. Antipov G, Baccouche M, Berrani S-A, Dugelay J-L (2016) Apparent age estimation from face images combining general and children-specialized deep learning models. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshop

  19. Tan Z, Yang Y, Wan J, Guo G, Li SZ (2019) Deeply-learned hybrid representations for facial age estimation. In: Proceedings of the international joint conference on artificial intelligence, p 3548–3554

  20. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: Proceedings of the international conference on learning representations

  21. Luo J-H, Wu J, Lin W (2017) ThiNet: a filter level pruning method for deep neural network compression. In: Proceedings of the IEEE international conference on computer vision

  22. Greco A, Saggese A, Vento M, Vigilante V (2021) Effective training of convolutional neural networks for age estimation based on knowledge distillation. Neural Computing and Applications, 1–16

  23. Escalera S, Torres Torres M, Martinez B, Baró" X, Jair Escalante H, Guyon I, Tzimiropoulos G, Corneou C, Oliu M, Ali Bagheri M et al (2016) ChaLearn looking at people and faces of the world: face analysis workshop and challenge 2016. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshop

  24. Zhang Z, Song Y, Qi H (2017) Age progression/regression by conditional adversarial autoencoder. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  25. Ricanek K, Tesafaye T (2006) Morph: A longitudinal image database of normal adult age-progression. In: Proceedings of the IEEE conference on automatic face and gesture recognition

  26. Xie D, Liang L, Jin L, Xu J, Li M (2015) SCUT-FBP: a benchmark dataset for facial beauty perception. In: IEEE international conference on systems, man, and cybernetics

  27. Ma DS, Correll J, Wittenbrink B (2015) The Chicago face database: a free stimulus set of faces and norming data. Behav Res Methods 47(4):1122–1135

    Article  Google Scholar 

  28. Geng X (2016) Label distribution learning. IEEE Trans Knowl Data Eng 28(7):1734–1748

    Article  Google Scholar 

  29. Gao B-B, Zhou H-Y, Wu J, Geng X (2018) Age estimation using expectation of label distribution learning. In: Proceedings of the international joint conference on artificial intelligence

  30. Cootes TF, Edwards GJ, Taylor CJ (2001) Active appearance models. IEEE Trans Pattern Anal Mach Intell 23(6):681–685

    Article  Google Scholar 

  31. Guo G, Mu G, Fu Y, Huang TS (2009) Human age estimation using bio-inspired features. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  32. Zhang D, Zhao Q, Chen F (2011) Quantitative analysis of human facial beauty using geometric features. Pattern Recognit 44(4):940–950

    Article  Google Scholar 

  33. Kagian A, Dror G, Leyvand T, Cohen-Or D, Ruppin E (2007) A humanlike predictor of facial attractiveness. Proc. Advances in Neural Information Processing Systems

  34. Huang D, Han L, De la Torre F (2017) Soft-margin mixture of regressions. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  35. Chang K-Y, Chen C-S, Hung Y-P (2011) Ordinal hyperplanes ranker with cost sensitivities for age estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  36. Chen K, Gong S, Xiang T, Change Loy C (2013) Cumulative attribute space for age and crowd density estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  37. Li C, Liu Q, Dong W, Zhu X, Liu J, Lu H (2015) Human age estimation based on locality and ordinal information. IEEE Trans Cybern 45(11):2522–2534

    Article  Google Scholar 

  38. Geng X, Yin C, Zhou Z-H (2013) Facial age estimation by learning from label distributions. IEEE Trans Pattern Anal Mach Intell 35(10):2401–2412

    Article  Google Scholar 

  39. Ren Y, Geng X (2017) Sense beauty by label distribution learning. In: Proceedings of the international joint conference on artificial intelligence

  40. Xing C, Geng X, Xue H (2016) Logistic boosting regression for label distribution learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  41. He Z, Li X, Zhang Z, Wu F, Geng X, Zhang Y, Yang M-H, Zhuang Y (2017) Data-dependent label distribution learning for age estimation. IEEE Trans Image Process 26(8):3846–3858

    Article  MathSciNet  Google Scholar 

  42. Shen W, Zhao K, Guo Y, Yuille A (2017) Label distribution learning forests. In: Proceedings of the 31st conference on neural information processing systems

  43. Hu Z, Wen Y, Wang J, Wang M, Hong R, Yan S (2017) Facial age estimation with age difference. IEEE Trans Image Process 26(7):3087–3097

    Article  MathSciNet  MATH  Google Scholar 

  44. Yang J, She D, Sun M (2017) Joint image emotion classification and distribution learning via deep convolutional neural network. In: Proceedings of the international joint conference on artificial intelligence

  45. Shen W, Guo Y, Wang Y, Zhao K, Wang B, Yuille AL (2018) Deep regression forests for age estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  46. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the international conference on machine learning

  47. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  48. Zhang K, Zhang Z, Li Z, Qiao Y (2016) Joint face detection and alignment using multi-task cascaded convolutional networks. IEEE Signal Process Lett 23(10):1499–1503

    Article  Google Scholar 

  49. Berg A, Oskarsson M, O’Connor M (2021) Deep ordinal regression with label diversity. In: Proceedings of the 25th international conference on pattern recognition

  50. Savchenko AV (2019) Efficient facial representations for age, gender and identity recognition in organizing photo albums using multi-output convnet. PeerJ Comput Sci 5:e197

    Article  Google Scholar 

  51. Han H, Otto C, Liu X, Jain AK (2015) Demographic estimation from face images: human versus machine performance. IEEE Trans Pattern Anal Mach Intell 37(6):1148–1161

    Article  Google Scholar 

  52. Antipov G, Baccouche M, Berrani S-A, Dugelay J-L (2017) Effective training of convolutional neural networks for face-based gender and age prediction. Pattern Recognit 72:15–26

    Article  Google Scholar 

  53. Pan H, Han H, Shan S, Chen X (2018) Mean-variance loss for deep age estimation from a face. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  54. Tan Z, Wan J, Lei Z, Zhi R, Guo G, Li SZ (2018) Efficient group-n encoding and decoding for facial age estimation. IEEE Trans Pattern Anal Mach Intell 40(11):2610–2623

    Article  Google Scholar 

  55. Zhang C, Liu S, Xu X, Zhu C (2019) C3AE: Exploring the limits of compact model for age estimation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

  56. Li W, Lu J, Feng J, Xu C, Zhou J, Tian Q (2019) Bridgenet: A continuity-aware probabilistic network for age estimation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

  57. Liu H, Sun P, Zhang J, Wu S, Yu Z, Sun X (2020) Similarity-aware and variational deep adversarial learning for robust facial age estimation. IEEE Trans Multimed 22(7):1808–1822

    Article  Google Scholar 

  58. Lim K, Shin N-H, Lee Y-Y, Kim C-S (2020) Order learning and its application to age estimation. In: Proceedings of the international conference on learning representations

  59. Li W, Huang X, Lu J, Feng J, Zhou J (2021) Learning probabilistic ordinal embeddings for uncertainty-aware regression. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

  60. Deng Z, Liu H, Wang Y, Wang C, Yu Z, Sun X (2021) PML: progressive margin loss for long-tailed age classification. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

  61. Liu H, Lu J, Feng J, Zhou J (2017) Ordinal deep learning for facial age estimation. IEEE Trans Circuits Syst Video Technol 29(2):486–501

    Article  Google Scholar 

  62. Huo Z, Yang X, Xing C, Zhou Y, Hou P, Lv J, Geng X (2016) Deep age distribution learning for apparent age estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops

  63. Uricár M, Timofte R, Rothe R, Matas J, Van Gool L (2016) Structured output svm prediction of apparent age, gender and smile from deep features. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops

  64. Cao W, Mirjalili V, Raschka S (2020) Rank consistent ordinal regression for neural networks with application to age estimation. Pattern Recognit Lett 140:325–331

    Article  Google Scholar 

  65. Gustafsson FK, Danelljan M, Bhat G, Schön TB (2020) Energy-based models for deep probabilistic regression. In: Proceedings of the European conference on computer vision

  66. Lin L, Liang L, Jin L (2018) R2-ResNeXt: A ResNeXt-based regression model with relative ranking for facial beauty prediction. In: Proceedings of the 24th international conference on pattern recognition

  67. Chen L, Deng W (2019) Facial attractiveness prediction by deep adaptive label distribution learning. In: Proceedings of the Chinese conference on biometric recognition

  68. Xu J (2021) Mt-resnet: a multi-task deep network for facial attractiveness prediction. In: Proceedings of the 2nd international conference on computing and data science

  69. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  70. Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: Proceedings of the European conference on computer vision

  71. Larson DB, Chen MC, Lungren MP, Halabi SS, Stence NV, Langlotz CP (2018) Performance of a deep-learning neural network model in assessing skeletal maturity on pediatric hand radiographs. Radiology f278(1):313–322

    Article  Google Scholar 

  72. Kalayeh MM, Seifu M, LaLanne W, Shah M (2015) How to Take a Good Selfie?. In: Proceedings of the the 23rd annual ACM conference on multimedia conference

Download references

Author information

Authors and Affiliations

  1. Tencent YouTu Lab, Shenzhen, 518057, China

    Bin-Bin Gao

Authors
  1. Bin-Bin Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bin-Bin Gao.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, BB. Jointly learning distribution and expectation in a unified framework for facial age and attractiveness estimation. Neural Comput & Applic 35, 15583–15599 (2023). https://doi.org/10.1007/s00521-023-08563-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08563-4

Keywords

Search

Navigation