Elsevier

Pattern Recognition

Volume 66, June 2017, Pages 82-94
Pattern Recognition

Group-aware deep feature learning for facial age estimation

https://doi.org/10.1016/j.patcog.2016.10.026Get rights and content

Highlights

  • We propose a group-aware deep feature learning (GA-DFL) method under the deep convolutional neural networks framework. With the learned nonlinear filters, the chronological age information can be well exploited.

  • We propose an overlapped coupled learning method to achieve the smoothness for the neighboring age groups. With this learning strategy, the age difference information on the age-group specific overlaps can be well measured.

  • We employ a multi-path deep CNN architecture to integrate multi-scale facial information into the learned face presentation to further improve the estimation performance.

  • Compared with most state-of-the-arts, experimental results show that our proposed methods have obtain significant performance on three released face aging datasets.

Abstract

In this paper, we propose a group-aware deep feature learning (GA-DFL) approach for facial age estimation. Unlike most existing methods which utilize hand-crafted descriptors for face representation, our GA-DFL method learns a discriminative feature descriptor per image directly from raw pixels for face representation under the deep convolutional neural networks framework. Motivated by the fact that age labels are chronologically correlated and the facial aging datasets are usually lack of labeled data for each person in a long range of ages, we split ordinal ages into a set of discrete groups and learn deep feature transformations across age groups to project each face pair into the new feature space, where the intra-group variances of positive face pairs from the training set are minimized and the inter-group variances of negative face pairs are maximized, simultaneously. Moreover, we employ an overlapped coupled learning method to exploit the smoothness for adjacent age groups. To further enhance the discriminative capacity of face representation, we design a multi-path CNN approach to integrate the complementary information from multi-scale perspectives. Experimental results show that our approach achieves very competitive performance compared with most state-of-the-arts on three public face aging datasets that were captured under both controlled and uncontrolled environments.

Introduction

Facial age estimation attempts to predict the real age value or age group based on facial images, which has widely potential applications such as facial bio-metrics, human-computer interaction, social media analysis and entertainments [1], [2], [3], [4]. While extensive efforts have been devoted, facial age estimation still remains a challenging problem due to two aspects: 1) lack of sufficient training data where each person should contain multiple images in a wide range of ages, 2) large variations such as lighting, occlusion and cluttered background of face images which were usually captured in wild conditions.

Most existing facial age estimation systems usually consist of two key modules: face representation and age estimation. Representative face representation approaches include holistic subspace features [5], [6], active appearance model (AAM) [7], Gabor wavelets [7], local binary pattern (LBP) [8] and bio-inspired feature (BIF) [9]. Having obtained face representations, age estimation can be addressed as a classification or regression problem [9], [10], [11]. However, the face representations employed most existing methods are hand-crafted, which requires strong prior knowledge to engineer it by hand. To address this problem, learning-based feature representation methods [5], [12], [13], [3] have been made to learn discriminative feature representation directly from raw pixels. However, these methods aim to learn linear feature filters to project face images into another feature space such that they may not be powerful enough to exploit the nonlinear relationship of data. To address this nonlinear issue, deep learning-based methods have been adopted to learn a series of nonlinear mapping functions between face image and age label [14], [15], [16], [16], [17], [18]. Unfortunately, these deep models cannot explicitly achieve the ordinal relationship among the chronological ages, which are still far from the practical satisfactory in most cases because they usually encounter unbalanced and insufficient training data for each age label.

Notice that age labels are chronologically correlated, so that it is desirable to employ nonlinear discriminative methods to exploit the correlated order information from facing images. Unlike existing deep learning-based facial age estimation methods that ignored the ordinal information of face aging data, we proposed a group-aware deep feature learning method (GA-DFL) under deep convolutional neural networks (CNN), by learning discriminative face representations directly from image pixels and exploiting the aging order information. Since facial aging datasets usually lack of face images from the same person covering a wide range of ages, our proposed GA-DFL first separates the chronological aging progress into several overlapped groups and then learns a series of hierarchical nonlinear mapping functions to project raw pixel values into another common feature space, so that face pairs in the same age groups are projected as close as possible while those in different age groups are projected as far as possible. Moreover, we link every discrete groups by overlapping structures and develop an overlapped coupled learning method, which aims to smooth the age differences lying on the overlaps of the adjacent age groups. We also propose a multi-path CNN architecture to enhance the capacity of feature representation to integrate complementary information from multiple scales to improve the performance. Fig. 1 illustrates the main procedure of our proposed approach. To evaluate the effectiveness of our proposed GA-DFL, we conducted experiments on three widely used facial age estimation datasets that were captured in both constrained and unconstrained environments. Experimental results show that our proposed GA-DFL obtains superior performance compared with most state-of-the-art facial age estimation methods.

The contributions of this work are summarized as follows:

  • (1)

    We develop a deep feature learning method to discriminatively learn a face representation directly from raw pixels. With the learned nonlinear filters, the chronological age information can be well exploited with a perspective of age groups in the obtained face descriptor.

  • (2)

    We propose an overlapped coupled learning method to achieve the smoothness on the neighboring age groups. With this learning strategy, the age difference information on the age-group specific overlaps can be well measured.

  • (3)

    We employ a multi-path deep CNN architecture to integrate multiple scale information into the learned face presentation.

The rest of this paper is organized as follows: Section 2 reviews some backgrounds. Section 3 details the proposed GA-DFL method. Section 4 provides the experimental results and Section 5 concludes this paper.

Section snippets

Related work

In this section, we briefly review two related topics: facial age estimation and deep learning.

Proposed approach

In this section, we present the proposed model GA-DFL and multi-path network architecture.

Experiments

In this section, we conducted facial age estimation experiments on the widely used FG-NET [25], MORPH (Album2) [50] and Chalearn Challenge Dataset [51] to show the effectiveness of the proposed GA-DFL. The followings describe the details of experimental settings and results.

Conclusions and future work

In this paper, we have proposed a group-aware deep feature learning (GA-DFL) for facial age estimation. Since the real-world age labels are correlated and hand-crafted face descriptors are not powerful to model the relationship between face images and age values, we have defined a set of age groups to describe the aging order relationship of face data and implicitly achieved the ordinal age-group relationship. Moreover, we developed an overlapped coupled learning to smooth the adjacent age

Acknowledgement

This work is supported by the National Key Research and Development Program of China under Grant 2016YFB1001001, the National Natural Science Foundation of China under Grants 61225008, 61672306, 61572271, 61527808, 61373074 and 61373090, the National 1000 Young Talents Plan Program, the National Basic Research Program of China under Grant 2014CB349304, the Ministry of Education of China under Grant 20120002110033, and the Tsinghua University Initiative Scientific Research Program.

Hao Liu received the B.S. degree in software engineering from Sichuan University, China, in 2011 and the Engineering Master degree in computer technology from University of Chinese Academy of Sciences, China, in 2014. He is currently pursuing the Ph.D. degree at the department of automation, Tsinghua University. His research interests include face recognition, facial age estimation and deep learning.

References (70)

  • Y.H. Kwon et al.

    Age classification from facial images

    Comput. Vis. Image Underst.

    (1999)
  • G. Guo et al.

    A framework for joint estimation of age, gender and ethnicity on a large database

    Image Vis. Comput.

    (2014)
  • Y. Dong et al.

    Automatic age estimation based on deep learning algorithm

    Neurocomputing

    (2016)
  • X. Geng et al.

    Automatic age estimation based on facial aging patterns

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2007)
  • R. He et al.

    Maximum correntropy criterion for robust face recognition

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2011)
  • J. Lu et al.

    Cost-sensitive local binary feature learning for facial age estimation

    IEEE Trans. Image Process.

    (2015)
  • X. Shu, J. Tang, H. Lai, L. Liu, S. Yan, Personalized age progression with aging dictionary, in: Proceedings of the...
  • Y. Fu et al.

    Human age estimation with regression on discriminative aging manifold

    IEEE Trans. Multimed.

    (2008)
  • G. Guo et al.

    Image-based human age estimation by manifold learning and locally adjusted robust regression

    IEEE Trans. Image Process.

    (2008)
  • T.F. Cootes et al.

    Active appearance models

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2001)
  • T. Ahonen et al.

    Face description with local binary patterns: application to face recognition

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2006)
  • G. Guo, G. Mu, Y. Fu, T.S. Huang, Human age estimation using bio-inspired features, in: IEEE Conference on Computer...
  • Y. Fu et al.

    Human age estimation with regression on discriminative aging manifold

    IEEE Trans. Multimed.

    (2008)
  • Y. Fu et al.

    Age synthesis and estimation via faces: a survey

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2010)
  • R. He et al.

    Half-quadratic based iterative minimization for robust sparse representation

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2014)
  • X. Liu, S. Li, M. Kan, J. Zhang, S. Wu, W. Liu, H. Han, S. Shan, X. Chen, Agenet: Deeply learned regressor and...
  • Z. Kuang, C. Huang, W. Zhang, Deeply learned rich coding for cross-dataset facial age estimation, in: Proceedings of...
  • X. Yang, B. Gao, C. Xing, Z. Huo, X. Wei, Y. Zhou, J. Wu, X. Geng, Deep label distribution learning for apparent age...
  • R. Ranjan, S. Zhou, J. Chen, A. Kumar, A. Alavi, V. M. Patel, R. Chellappa, Unconstrained age estimation with deep...
  • X. Wang, R. Guo, C. Kambhamettu, Deeply-learned feature for age estimation, in: Proceedings of the IEEE Winter...
  • T. Wu et al.

    Age estimation and face verification across aging using landmarks

    IEEE Trans. Inf. Forensics Secur.

    (2012)
  • K. Chang, C. Chen, Y. Hung, Ordinal hyperplanes ranker with cost sensitivities for age estimation, in: Proceedings of...
  • Y. Chen et al.

    Subspace learning for facial age estimation via pairwise age ranking

    IEEE Trans. Inf. Forensics Secur.

    (2013)
  • Z. Li et al.

    A discriminative model for age invariant face recognition

    IEEE Trans. Inf. Forensics Secur.

    (2011)
  • X. Geng et al.

    Facial age estimation by learning from label distributions

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2013)
  • G. Guo, G. Mu, Simultaneous dimensionality reduction and human age estimation via kernel partial least squares...
  • A. Lanitis et al.

    Toward automatic simulation of aging effects on face images

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2002)
  • C. Li, Q. Liu, J. Liu, H. Lu, Learning ordinal discriminative features for age estimation, in: Proceedings of the IEEE...
  • Y. Zhang, D. Yeung, Multi-task warped gaussian process for personalized age estimation, in: Proceedings of the IEEE...
  • K. Chang, C. Chen, Y. Hung, A ranking approach for human ages estimation based on face images, in: Proceedings of the...
  • G. Guo, G. Mu, Y. Fu, T.S. Huang, Human age estimation using bio-inspired features, in: Proceedings of the IEEE...
  • Y. Bengio, P. Lamblin, D. Popovici, H. Larochelle, Greedy layer-wise training of deep networks, in: Neural Information...
  • G.E. Hinton et al.

    A fast learning algorithm for deep belief nets

    Neural Comput.

    (2006)
  • A. Krizhevsky, I. Sutskever, G.E. Hinton, Imagenet classification with deep convolutional neural networks, in: Neural...
  • D.C. Ciresan et al.

    Deep, big, simple neural nets for handwritten digit recognition

    Neural Comput.

    (2010)
  • Cited by (87)

    • AttM-CNN: Attention and metric learning based CNN for pornography, age and Child Sexual Abuse (CSA) Detection in images

      2021, Neurocomputing
      Citation Excerpt :

      Different than biological age estimation, another research direction is apparent age estimation [62,7,1] which involves age estimation on the images labeled by volunteers based on their subjective age estimate from the visual appearance of a human [22,23]. Age estimation solutions proposed so far have been formulated mainly as classification [21,7,37,19,47], regression [78,67,20,43], ranking [17,46,44], or their combinations [62,16,48]. Earlier methods were mostly based on handcrafted features For instance, Eidinger et al. [21] presented dropout-SVM to classify features extracted with Local binary patterns (LBP) and Four Patch LBP (FPLBP) descriptors, and Iqbal et al. [37] introduced a face descriptor named Directional Age-Primitive Pattern (DAPP) to model discernible aging cues.

    • Vec2UAge: Enhancing underage age estimation performance through facial embeddings

      2021, Forensic Science International: Digital Investigation
      Citation Excerpt :

      The MAE in different age ranges were also evaluated with the aforementioned database; the best performance lies within the age range 0–9 (1.911 years) followed by the age range 10–19 (3.52 years) with the C-lsLPP algorithm approach. In 2017, Liu et al. presented a Group-aware deep feature learning approach that consists in learning a discriminative feature descriptor per image of the raw pixels for face representation (Liu et al., 2017). The main motivation is that age labels are chronologically correlated and face ageing datasets lack labelled data in certain groups.

    View all citing articles on Scopus

    Hao Liu received the B.S. degree in software engineering from Sichuan University, China, in 2011 and the Engineering Master degree in computer technology from University of Chinese Academy of Sciences, China, in 2014. He is currently pursuing the Ph.D. degree at the department of automation, Tsinghua University. His research interests include face recognition, facial age estimation and deep learning.

    Jiwen Lu received the B.Eng. degree in mechanical engineering and the M.Eng. degree in electrical engineering from the Xi'an University of Technology, Xi'an, China, and the Ph.D. degree in electrical engineering from the Nanyang Technological University, Singapore, in 2003, 2006, and 2012, respectively. He is currently an Associate Professor with the Department of Automation, Tsinghua University, Beijing, China. From March 2011 to November 2015, he was a Research Scientist with the Advanced Digital Sciences Center, Singapore. His current research interests include computer vision, pattern recognition, and machine learning. He has authored/co-authored over 130 scientific papers in these areas, where more than 50 papers are published in the IEEE Transactions journals and top-tier computer vision conferences. He serves/has served as an Associate Editor of Pattern Recognition Letters, Neurocomputing, and the IEEE Access, a Guest Editor of Pattern Recognition, Computer Vision and Image Understanding, Image and Vision Computing and Neurocomputing, and an elected member of the Information Forensics and Security Technical Committee of the IEEE Signal Processing Society. He is/was a Workshop Chair/Special Session Chair/Area Chair for more than 10 international conferences. He has given tutorials at several international conferences including ACCV’16, CVPR’15, FG’15, ACCV’14, ICME’14, and IJCB’14. He was a recipient of the First-Prize National Scholarship and the National Outstanding Student Award from the Ministry of Education of China in 2002 and 2003, the Best Student Paper Award from Pattern Recognition and Machine Intelligence Association of Singapore in 2012, the Top 10% Best Paper Award from IEEE International Workshop on Multimedia Signal Processing in 2014, and the National 1000 Young Talents Plan Program in 2015, respectively. He is a senior member of the IEEE.

    Jianjiang Feng is an associate professor in the Department of Automation at Tsinghua University, Beijing. He received the B.S. and Ph.D. degrees from the School of Telecommunication Engineering, Beijing University of Posts and Telecommunications, China, in 2000 and 2007. From 2008 to 2009, he was a Post Doctoral researcher in the PRIP lab at Michigan State University. He is an Associate Editor of Image and Vision Computing. His research interests include fingerprint recognition and computer vision.

    Jie Zhou received the BS and MS degrees both from the Department of Mathematics, Nankai University, Tianjin, China, in 1990 and 1992, respectively, and the Ph.D. degree from the Institute of Pattern Recognition and Artificial Intelligence, Huazhong University of Science and Technology (HUST), Wuhan, China, in 1995. From then to 1997, he served as a postdoctoral fellow in the Department of Automation, Tsinghua University, Beijing, China. Since 2003, he has been a full professor in the Department of Automation, Tsinghua University. His research interests include computer vision, pattern recognition, and image processing. In recent years, he has authored more than 100 papers in peer-reviewed journals and conferences. Among them, more than 30 papers have been published in top journals and conferences such as the IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE Transactions on Image Processing, and CVPR. He is an associate editor for the International Journal of Robotics and Automation and two other journals. He received the National Outstanding Youth Foundation of China Award. He is a senior member of the IEEE.

    View full text