Abstract
In the last decade, demographic profiling from facial imagery has grown in its importance in the computer vision field. For demographic profiling, we usually mean gender, ethnicity, and age identification from face images. In this paper, we propose an efficient and effective profiling framework and we assess the quality of the proposed approach comparing the results obtained by our system with those achieved by other recently published methods on large datasets of facial images with different age, gender, and ethnicity. These results show how a carefully engineered pipeline of efficient image analysis and pattern recognition techniques leads to state-of-the-art results at 20 FPS using a single thread on a 1.6 GHZ i5-2467M processor.
Similar content being viewed by others
Notes
“Ethnicity: definition of ethnicity”. Oxford Dictionaries. Oxford University Press. Retrieved 28 December 2013.
We call “white” a dataset of points sampled from a probability distribution with \({\varvec{\mu }}={\varvec{0}}\), and \({\varvec{\varSigma }}={\varvec{I}}\) where \({\varvec{I}}\) is the identity matrix.
d is a parameter to be set. Usually a good value is \(d \simeq min(log_2^2 N,D)\)
A video with showing the output of our algorithm on every subject in this dataset is available as supplementary material.
References
Acharya, K.A., Babu, R.V., Vadhiyar, S.S.: A real-time implementation of sift using gpu. J. Real Time Image Process. 14(2), 267–277 (2018)
Akata, Z., Perronnin, F., Harchaoui, Z., Schmid, C.: Good practice in large-scale learning for image classification. IEEE Trans. Pattern Anal. Mach. Intell. 36(3), 507–520 (2014)
Alexandre, L.A.: Gender recognition: a multiscale decision fusion approach. Pattern Recognit. Lett. 31(11), 1422–1427 (2010)
Antipov, G., Baccouche, M., Berrani, S.-A., Dugelay, J.-L.: Apparent age estimation from face images combining general and children-specialized deep learning models. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 96–104 (2016)
Canziani, A., Paszke, A., Culurciello, E.: An analysis of deep neural network models for practical applications. arXiv preprint arXiv:1605.07678 (2016)
Chang, K.-Y., Chen, C.-S., Hung, Y.-P.: A ranking approach for human ages estimation based on face images. In: Proceedings of ICPR, Aug, pp. 3396–3399 (2010)
Chang, K.-Y., Chen, C.-S., Hung, Y.-P.: Ordinal hyperplanes ranker with cost sensitivities for age estimation. In: Proceedings of CVPR, June, pp. 585–592 (2011)
Cirne, M.V.M., Pedrini, H.: Gender recognition from face images using a geometric descriptor. In: 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 2006–2011. IEEE (2017)
Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001)
Demirkus, M., Precup, D., Clark, J., Arbel, T.: Hierarchical spatio-temporal probabilistic graphical model with multiple feature fusion for estimating binary facial attribute classes in real-world face videos. IEEE Trans. Pattern Anal. Mach. Intell. 6(1185), 1203 (2015)
Du, S., Ward, R.: Wavelet-based illumination normalization for face recognition. In: Proceedings of ICPR (2005)
Duan, M., Li, K., Li, K.: An ensemble cnn2elm for age estimation. IEEE Trans. Inf. Forensics Secur. 13(3), 758–772 (2018)
Duan, M., Li, K., Yang, C., Li, K.: A hybrid deep learning cnn-elm for age and gender classification. Neurocomputing 275, 448–461 (2018)
Eidinger, E., Enbar, R., Hassner, T.: Age and gender estimation of unfiltered faces. IEEE Trans. Inf. Forensics Secur. 9(12), 2170–2179 (2014)
Escalera, S., Torres Torres, M., Martinez, B., Baró, X., Jair Escalante, H., Guyon, I., Tzimiropoulos, G., Corneou, C., Oliu, M., Ali Bagheri, M., et al: Chalearn looking at people and faces of the world: face analysis workshop and challenge 2016. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1–8 (2016)
Fourney, A., Laganiere, R.: Constructing face image logs that are both complete and concise. In: Proceedings of CRV, pp. 488–494 (2007)
Geng, X., Yin, C., Zhou, Z.-H.: Facial age estimation by learning from label distributions. IEEE Trans. Pattern Anal. Mach. Intell. 35(10), 2401–2412 (2013)
Guo, G., Mu, G., Fu, Y., Huang, T.S.: Human age estimation using bio-inspired features. In: Proceedings of CVPR, June, pp. 112–119 (2009)
Guo, G., Mu, G.: A study of large-scale ethnicity estimation with gender and age variations. In: Proceedings of CVPRW, pp. 79–86. IEEE (2010)
Guo, G., Mu, G.: Human age estimation: what is the influence across race and gender? In: Proceedings of CVPRW, June, pp. 71–78 (2010)
Guo, G., Mu, G.: Simultaneous dimensionality reduction and human age estimation via kernel partial least squares regression. In: Proceedings of CVPR, June, pp. 657–664 (2011)
Guo, G., Mu, G.: Simultaneous dimensionality reduction and human age estimation via kernel partial least squares regression. In: Proceedings of CVPR, June, pp. 657–664 (2011)
Guo, G., Zhang, C.: A study on cross-population age estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4257–4263 (2014)
Gurpinar, F., Kaya, H., Dibeklioglu, H., Salah, A.: Kernel elm and cnn based facial age estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 80–86 (2016)
Han, H., Otto, C., Jain, A.K.: Age estimation from face images: human vs. machine performance. In: Proceedings of ICB (2013)
Hansen, P.C.: The truncated SVD as a method for regularization. BIT Numer. Math. 27(4), 534–553 (1987)
Hassner, T., Harel, S., Paz, E., Enbar, R.: Effective face frontalization in unconstrained images. In: Proceedings of CVPR, June 2015
Hou, L., Yu, C.-P., Samaras, D.: Squared earth movers distance loss for training deep neural networks on ordered-classes. In: Proceedings of NIPS (2017)
Huang, G.-B., Chen, L., Siew, C.K., et al.: Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans. Neural Netw. 17(4), 879–892 (2006)
Huerta, I., Fernández, C., Prati, A.: Facial age estimation through the fusion of texture and local appearance descriptors. In: European Conference on Computer Vision, pp. 667–681. Springer (2014)
Huo, Z., Yang, X., Xing, C., Zhou, Y., Hou, P., Lv, J., Geng, X.: Deep age distribution learning for apparent age estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 17–24 (2016)
Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: Proceedings of CVPR (2014)
King, D.E.: Max-Margin Object Detection. ArXiv e-prints, January (2015)
Lapuschkin, S., Binder, A., Müller, K.-R., Samek, W.: Understanding and comparing deep neural networks for age and gender classification. arXiv preprint arXiv:1708.07689 (2017)
Levi, G., Hassner, T.: Age and gender classification using convolutional neural networks. In: Proceedings of CVPRW, June (2015)
Liu, X., Li, S., Kan, M., Zhang, J., Wu, S., Liu, W., Han, H., Shan, S., Chen, X.: Agenet: Deeply learned regressor and classifier for robust apparent age estimation. In: The IEEE International Conference on Computer Vision (ICCV) Workshops, December 2015
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)
Malli, R.C., Aygün, M., Ekenel, H.K.: Apparent age estimation using ensemble of deep learning models. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 714–721. IEEE (2016)
Meyers, E., Wolf, L.: Using biologically inspired features for face processing. Int. J. Comput. Vis. 76(1), 93–104 (2008)
Nasrollahi, K., Moeslund, T.B: Face quality assessment system in video sequences. In: Biometrics and Identity Management, pp. 10–18. Springer (2008)
Ng, C.B., Tay, Y.H., Goi, B.-M.: Vision-based human gender recognition: a survey. CoRR, abs/1204.1611 (2012)
Ngan, M., Grother, P.: Face recognition vendor test (frvt) performance of automated age estimation algorithms. NIST Interagency Report, 7995 (2014)
Ngan, M., Grother, P.: Face recognition vendor test (frvt) performance of automated gender classification algorithms. In: Technical Report NIST IR 8052. National Institute of Standards and Technology (2015)
Ni, B., Song, Z., Yan, S.: Web image and video mining towards universal and robust age estimator. IEEE Trans. Multimed. 13(6), 1217–1229 (2011)
Nvidia cublas library for fast gpu-accelerated implementation of the standard basic linear algebra subroutines (blas). https://developer.nvidia.com/cublas. Accessed 18 Sep 2018
Nvidia embedded gpu boards. https://developer.nvidia.com/embedded/develop/hardware. Accessed 18 Sep 2018
Platt, J.C., Cristianini, N., Shawe-taylor, J.: Large margin DAGs for multiclass classification. In: Proceedings of NIPS, pp. 547–553. MIT Press (2000)
Riesenhuber, Maximilian, Poggio, Tomaso: Hierarchical models of object recognition in cortex. Nat. Neurosci. 2(11), 1019–1025 (1999)
Rothe, R., Timofte, R., Van Gool, L.: Dex Deep expectation of apparent age from a single image. In: The IEEE International Conference on Computer Vision (ICCV) Workshops, December 2015
Rozza, A., Lombardi, G., Casiraghi, E.: Novel ipca-based classifiers and their application to spam filtering. In: Proceedings of ISDA, pp. 797–802 (2009)
Rozza, A., Lombardi, G., Rosa, M., Casiraghi, E: O-ipcac and its application to eeg classification. In: Proceedings of WAPA, pp. 4–11 (2010)
Rozza, Alessandro, Lombardi, Gabriele, Casiraghi, Elena, Campadelli, Paola: Novel fisher discriminant classifiers. Pattern Recognit. 45(10), 3725–3737 (2012)
Seidenari, L., Rozza, A., Del Bimbo, A.: Real-time age estimation from face imagery using fisher vectors. In: Proceedings of ICIAP (2015)
Seidenari, L., Serra, G., Badanov, A.D., Bimbo, A.: Local pyramidal descriptors for image recognition. IEEE Trans. Pattern Anal. Mach. Intell. 36, 1033–1040 (2014)
Simonyan, K., Parkhi, O.M., Vedaldi, A., Zisserman, A.: Fisher Vector Faces in the Wild. In: Proceedings of British Machine Vision Conference (BMVC) (2013)
Snchez, Jorge, Perronnin, Florent, Mensink, Thomas, Verbeek, Jakob: Image classification with the fisher vector: theory and practice. Int. J. Comput. Vis. 105(3), 222–245 (2013)
Uricár, M., Timofte, R., Rothe, R., Matas, J., Van Gool, L.: Structured output svm prediction of apparent age, gender and smile from deep features. In: Proceedings CVPRW, pp. 25–33 (2016)
Wang, J.G., Li, J., Yau, W.Y., Sung, E.: Boosting dense SIFT descriptors and shape contexts of face images for gender recognition. In: IEEE Workshop on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 96–102 (2010)
Yang, X., Gao, B.-B., Xing, C., Huo, Z.W., Wei, X.S., Zhou, Y., Wu, J., Geng, X.: Deep label distribution learning for apparent age estimation. In: The IEEE International Conference on Computer Vision (ICCV) Workshops, December 2015
Yun, F., Huang, T.S.: Human age estimation with regression on discriminative aging manifold. IEEE Trans. Multimed. 10(4), 578–584 (2008)
Zhang, K., Gao, C., Guo, L., Sun, M., Yuan, X., Han, T.X., Zhao, Z., Li, B.: Age group and gender estimation in the wild with deep ror architecture. IEEE Access 5, 22492–22503 (2017)
Zhang, K., Sun, M., Han, X., Yuan, X., Guo, L., Liu, T.: Residual networks of residual networks: multilevel residual networks. IEEE Trans. Circuits Syst. Video Technol. 28, 1303–1314 (2017)
Zhu, Y., Li, Y., Mu, G., Guo, G.: A study on apparent age estimation. In: The IEEE International Conference on Computer Vision (ICCV) Workshops, December 2015
Acknowledgements
Lorenzo Seidenari is partially supported by “THE SOCIAL MUSEUM AND SMART TOURISM”, MIUR project no. CTN01_00034_23154_SMST.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This work was performed when Dr. Rozza was affiliated to another company.
Rights and permissions
About this article
Cite this article
Seidenari, L., Rozza, A. & Del Bimbo, A. Real-time demographic profiling from face imagery with Fisher vectors. Machine Vision and Applications 30, 359–374 (2019). https://doi.org/10.1007/s00138-018-0991-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00138-018-0991-2