Cumulative attribute relation regularization learning for human age estimation
Introduction
In machine learning, a large number of problems are related to human face due to that rich information is contained in it, such as facial expression, gender, race and age, in which the problem of human face-based age estimation has aroused increasing attention due to its wide applications such as web security control [12], [17], ancillary identity authentication [14], and advertisement recommendation [27], etc.
In order to conduct age estimation based on human face, a variety of approaches have been proposed to date. Generally, they fall into three categories: classification-based, e.g., [17], [9], [30], [1], [28], regression-based, e.g., [18], [7], [22], [33], [32], [10], [4], [19], [20], and their hybrid, e.g., [12], [13], [16].
When we consider each age as a separate class, the age estimation can be made under ordinary classification framework. For example, Lanitis et al. [17] extracted AAM features from facial images and respectively applied the nearest neighbor classifier and artificial neural networks for age estimation and achieved comparable performance. Geng et al. [9] specially designed a three-layer conditional probability neural network (CPNN) to capture the age contribution information for age classification. Moreover, Ueki et al. [30] conducted age group classification by building Gaussian mixture models after discriminative dimensionality-reduction and received promising results respectively for male and female on several famous age datasets. More recently, Alnajar et al. [1] employed the soft coding to extract codebooks for age group classification and received better estimation on an unconstrained real-life dataset than the hard coding approaches. And Sai et al. [28] even used the extreme learning machines to perform age group estimation and obtained competitive results.
Actually, the age estimation is more of a regression problem than a generic multi-class classification due to the continuity of aging. According to this characteristic, many attempts have been made. For instance, Lanitis et al. [18] established a quadratic function to fit the ages with facial images represented by AAM features. Fu et al. [7] borrowed the multiple linear regression to learn an aging prediction function in the manifold space. And Luu et al. [22] employed the off-the-shelf ξ-SVR [31] for aging function learning. Moreover, to handle the uncertainty of annotations of age labels, Yan et al. [33] constructed a semi-definite programming (SDP) regression model to train an aging regressor. Although the SDP regressor can relatively model the age labels׳ uncertainty better, the learning is very time-consuming. To reduce the time complexity, they [32] then proposed to speed up the SDP learning by using the Expectation–Maximization (EM). Furthermore, Geng et al. [10] proposed the aging pattern regressing (AGES) to generate age labels for missing patterns. Although the methods afore-mentioned can yield age estimation performance to different extents, they ignored the fact that there exists natural ordinality among ages [5], [4]. To this end, Chang et al. [4] specially designed an ordinal hyperplanes ranker (OHRank) for age estimation and on FG-NET dataset they obtained better performance than AGES. Later, Li et al. [19] presented a distance-based ordinal regressor for age estimation, in which the ordinal information of ages is incorporated into the metric and on FG-NET they obtained competitive performance. Moreover, they [20] took the ordinality and local manifold structure preserving ability as a criterion to perform feature selection and conducted age regression with much competitive results. More recently, they [21] presented an ordinal metric learning method for image ranking by preserving both the local geometry information and the ordinal relationship of the data.
Although the methods reviewed above can perform encouraging human age estimation with different performance, they have not exploited another essential characteristic of the ages that neighboring ages are generally more similar in facial appearance than those apart. For example, the facial appearance of 11-year-old is more similar to that of 13 compared to that of 30, as exhibited in Fig. 2 (in Section 2). This characteristic is of help in estimating the ages, especially when the age distribution is imbalanced [5], because similar ages can be used to partially depict their neighboring ages that are absent in the learning and thus alleviate the imbalance. Therefore, such neighbor-similarity of ages should also be incorporated into the estimation. To simultaneously consider both the ordinality and the neighbor-similarity of the ages,1 Chen et al. [5] proposed the cumulative attribute (CA) coding to represent the age. Concretely, they first used the multivariate ridge regression (mRR) [2] to transform the instance from its original input feature to a CA code; and then applied a second-layer scalar-output regressor to map the CA code to a scalar age label. The flowchart of the two-layer regression is shown in Fig. 1, and by this way they obtained competitive age estimations.
Although the characteristics of ordinality and neighbor-similarity of the ages are considered in the CA coding, the inherent mutual relations explicitly or implicitly existing between the CA codes have not been exploited for learning, thus leaving us a room of promoting its performance. To this end, in this work we first derive such relations by performing difference-like operations on the CA coding matrix2 to construct so-called 0-order and 1-order relation matrices,3 respectively. Then, we formulate the relation matrices as two corresponding regularization terms, coined as CA-oriented ordinal structure regularization (CAOSR) and CA-oriented adjacent difference orthogonal regularization (CAADOR). And, in order to take the mutual relations into the CA learning, we regularize the first-layer regressor (as shown in Fig. 1) by embedding the regularization terms, CAOSR and CAADOR, into its objective. Finally, through extensive experiments, we demonstrate the effectiveness of our strategies in improving CA learning on human age estimation.
The rest of this paper is organized as follows. In Section 2, we briefly review related work on CA coding. In Section 3, we derive two types of regularization terms, coined as CAOSR and CAADOR, to depict the mutual relations among the CA codes, and embed them into the objectives of the mRR and mLS-SVR, both of which act as the first-layer regressor in the CA learning, in Section 4. In Section 5, we conduct experiments to evaluate our strategies. Finally, we conclude the paper in Section 6.
Section snippets
Related work
Following the spirit of literature such as [23], Chen et al. [5] presented the cumulative attribute(CA) coding for learning in such scenarios as human age estimation. Concretely, given a set of N training samples {xi, li} , , i=1,2,…,N, where xi denotes the ith instance and li is its corresponding scalar label, D denotes the feature dimensionality of xi and K is the number of classes (e.g., the scale of the aging range). Here for the ith sample xi, its scalar label value li,
Two types of CA-oriented regularization
According to the definition of CA coding in Section 2, for the given K ordinal classes, we can demonstrate their CA codes together in a CA coding matrix as shown in Fig. 3(a).
Regularized mRR and mLS-SVR with the CAOSR and CAADOR
To validate the effectiveness of the two derived regularization terms, CAOSR and CAADOR, in incorporating the mutual relations between the CA codes into the learning and thus promoting the performance, we take the mRR as the first-layer regressor as in [5] by embedding them into the objective to regularize its learning. In addition, to evaluate the popularization ability of the terms, we also conduct evaluations in the framework of large-margin learning by taking the mLS-SVR as the first-layer
Experiments
In this section, we conduct experiments to validate the effectiveness of our regularization schemes in capturing the mutual relations among the CA codes to improve the CA-based human age estimation. Specifically, first we make a general comparison with related methods, by experimenting with data represented with high-level (i.e., Active Appearance Model) and low-level (i.e., raw-pixel) features, respectively; Then, to detailedly evaluate the ability of our strategies in improving CA-based age
Conclusions
In this work, in order to exploit and incorporate the mutual relations existing explicitly or implicitly between the CA codes into learning on human age estimation, we first derived the so-called 0-order and 1-order relation matrices by performing the difference-like operations on the CA codes, and formulated them correspondingly as two regularization terms, coined as the CAOSR and the CAADOR, respectively. Then, we embedded the two regularization terms into the objective of the mRR and mLS-SVR
Acknowledgments
This work was partially supported by the National Natural Science Foundation of China under Grant 61472186, the Specialized Research Fund for the Doctoral Program of Higher Education under Grant 20133218110032, the Funding of Jiangsu Innovation Program for Graduate Education under Grant CXLX13_159, the Fundamental Research Funds for the Central Universities and Jiangsu Qinglan Project.
Qing Tian received the B.S. degree in computer science from Southwest University for Nationalities, China, and the M.S. degree in computer science from Zhejiang University of Technology, China, respectively with the honor of Sichuan province-level outstanding graduate and Zhejiang province-level outstanding graduate in 2008 and 2011. From February 2011 to February 2012, as a researcher in the field of gender/age recognition, he worked in ArcSoft, Inc., USA. Since April 2012, he has been a Ph.D.
References (33)
- et al.
Learning-based encoding with soft assignment for age estimation under unconstrained imaging conditions
Image Vis. Comput.
(2012) - et al.
Ecoc-drfdiscriminative random fields based on error correcting output codes
Pattern Recognit.
(2014) - et al.
Hierarchical age estimation with dissimilarity-based classification
Neurocomputing
(2013) - et al.
Facial age range estimation with extreme learning machines
Neurocomputing
(2015) - S. An, W. Liu, S. Venkatesh, Face recognition using kernel ridge regression, in: IEEE Conference on Computer Vision and...
- et al.(2006)
- K.Y. Chang, C.S. Chen, Y.P. Hung, Ordinal hyperplanes ranker with cost sensitivities for age estimation, in: IEEE...
- K. Chen, S. Gong, T. Xiang, Q. Mary, C.C. Loy, Cumulative attribute space for age and crowd density estimation, in:...
- Y. Fu, Y. Xu, T.S. Huang, Estimating human age by manifold analysis of face pictures and regression on aging features,...
- et al.
Prototype selection for nearest neighbor classificationtaxonomy and empirical study
IEEE Trans. Pattern Anal. Mach. Intell.
(2012)
Facial age estimation by learning from label distributions
IEEE Trans. Pattern Anal. Mach. Intell.
Automatic age estimation based on facial aging patterns
IEEE Trans. Pattern Anal. Mach. Intell.
Image-based human age estimation by manifold learning and locally adjusted robust regression
IEEE Trans. Image Process.
Cited by (14)
Facial age estimation with bilateral relationships exploitation
2021, NeurocomputingCitation Excerpt :However, it is worth noting that such a joint modeling manner is not trivial, as the relationship-exploiting matrices are coupled together, which induces a non-convex objective that needs specialized optimization algorithm. In addition, unlike the hand-defined correlations utilized in CAADOR [43], LDL [38,52,53], and age grouping [49,57,58], our modeling manner can guide relationship-exploiting adaptively from the training data. Another benefit is that, the captured relationships can be directly visualized via the correlation matrices, regardless the correlations are positive, negative, or unrelated.
Cumulative attribute space regression for head pose estimation and color constancy
2019, Pattern RecognitionCitation Excerpt :In this work, we show that this hypothesis holds for a wider class of vision problems. Inspired by the success of CA for scalar-valued regression [13], we extend CA to the multivariate output setting. A straightforward extension is to apply CA regression to each output variable independently.
Joint gender classification and age estimation by nearly orthogonalizing their semantic spaces
2018, Image and Vision ComputingCitation Excerpt :Although most of the above methods, especially the CNNs based deep models can classify the ages with good results, the AE problem should be viewed as regression more than classification because the facial appearance is aging continuously. To this end, the support Vector Regression (SVR) [51–54], Ridge Regression (RR) [54,55] as well as Sparse Regression (SR) [56–58] methods have been adopted to regress human ages. Apart from the commonly-used regressors above, Hu et al. [59] adopted the Lie algebra Gaussian method to extract age-related feature representations from the face images and took the refined regressing for AE.
Unsupervised Multitarget Domain Adaptation With Dictionary-Bridged Knowledge Exploitation
2024, IEEE Transactions on Neural Networks and Learning SystemsResearch on the coupled-relationships self-learning human facial age estimation
2022, CAAI Transactions on Intelligent SystemsCDBN: Crow Deep Belief Network Based on Scattering and AAM Features for Age Estimation
2021, Journal of Signal Processing Systems
Qing Tian received the B.S. degree in computer science from Southwest University for Nationalities, China, and the M.S. degree in computer science from Zhejiang University of Technology, China, respectively with the honor of Sichuan province-level outstanding graduate and Zhejiang province-level outstanding graduate in 2008 and 2011. From February 2011 to February 2012, as a researcher in the field of gender/age recognition, he worked in ArcSoft, Inc., USA. Since April 2012, he has been a Ph.D. candidate in computer science at Nanjing University of Aeronautics and Astronautics, and his current research interests include machine learning and pattern recognition.
Songcan Chen received the B.S. degree from Hangzhou University (now merged into Zhejiang University), the M.S. degree from Shanghai Jiao Tong University and the Ph.D. degree from Nanjing University of Aeronautics and Astronautics (NUAA) in 1983, 1985, and 1997, respectively. He joined in NUAA in 1986, and since 1998, he has been a full-time Professor with the Department of Computer Science and Engineering. He has authored/co-authored over 170 scientific peer-reviewed papers and ever obtained Honorable Mentions of 2006, 2007 and 2010 Best Paper Awards of Pattern Recognition Journal respectively. His current research interests include pattern recognition, machine learning, and neural computing.