research-article

Joint Estimation of Age and Expression by Combining Scattering and Convolutional Networks

Authors:

Huei-Fang Yang,

Kuang-Yu Chang,

Chu-Song ChenAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 14, Issue 1

Article No.: 9, Pages 1 - 18

https://doi.org/10.1145/3152118

Published: 04 January 2018 Publication History

Abstract

This article tackles the problem of joint estimation of human age and facial expression. This is an important yet challenging problem because expressions can alter face appearances in a similar manner to human aging. Different from previous approaches that deal with the two tasks independently, our approach trains a convolutional neural network (CNN) model that unifies ordinal regression and multi-class classification in a single framework. We demonstrate experimentally that our method performs more favorably against state-of-the-art approaches.

References

[1]

Timo Ahonen, Abdenour Hadid, and Matti Pietikäinen. 2006. Face description with local binary patterns: Application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28, 12 (2006), 2037--2041.

Digital Library

[2]

Fares Alnajar, Zhongyu Lou, José Manuel Álvarez, and Theo Gevers. 2014. Expression-invariant age estimation. In Proc. BMVC.

[3]

Grigory Antipov, Moez Baccouche, Sid-Ahmed Berrani, and Jean-Luc Dugelay. 2016. Apparent age estimation from face images combining general and children-specialized deep learning models. In Proc. CVPR Workshops. 801--809.

[4]

Joan Bruna and Stéphane Mallat. 2013. Invariant scattering convolution networks. IEEE Trans. Pattern Anal. Mach. Intell. 35, 8 (2013), 1872--1886.

Digital Library

[5]

Kuang-Yu Chang and Chu-Song Chen. 2015. A learning framework for age rank estimation based on face images with scattering transform. IEEE Trans. Image Processing 24, 3 (2015), 785--798.

[6]

Kuang-Yu Chang, Chu-Song Chen, and Yi-Ping Hung. 2010. A ranking approach for human ages estimation based on face images. In Proc. ICPR. 3396--3399.

Digital Library

[7]

Kuang-Yu Chang, Chu-Song Chen, and Yi-Ping Hung. 2011. Ordinal hyperplanes ranker with cost sensitivities for age estimation. In Proc. CVPR. 585--592.

Digital Library

[8]

Kuang-Yu Chang, Chu-Song Chen, and Yi-Ping Hung. 2013. Intensity rank estimation of facial expressions based on a single image. In Proc. SMC. 3157--3162.

Digital Library

[9]

Timothy F. Cootes, Gareth J. Edwards, and Christopher J. Taylor. 2001. Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23, 6 (2001), 681--685.

Digital Library

[10]

Hamdi Dibeklioglu, Fares Alnajar, Albert Ali Salah, and Theo Gevers. 2015. Combining facial dynamics with appearance for age estimation. IEEE Trans. Image Processing 24, 6 (2015), 1928--1943.

[11]

Natalie C. Ebner, Michaela Riediger, and Ulman Lindenberger. 2010. FACES--A database of facial expressions in young, middle-aged, and older women and men: Development and validation. Behavior Research Methods 42, 1 (2010), 351--362.

[12]

David Freedman. 2005. Statistical Models: Theory and Practice. Cambridge University Press.

[13]

Bin-Bin Gao, Chao Xing, Chen-Wei Xie, Jianxin Wu, and Xin Geng. 2017. Deep label distribution learning with label ambiguity. IEEE Trans. Image Processing 26, 6 (2017), 2825--2838.

Digital Library

[14]

Xin Geng, Zhi-Hua Zhou, and Kate Smith-Miles. 2007. Automatic age estimation based on facial aging patterns. IEEE Trans. Pattern Anal. Mach. Intell. 29, 12 (2007), 2234--2240.

Digital Library

[15]

Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. 2016. Region-based convolutional networks for accurate object detection and segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 38, 1 (2016), 142--158.

Digital Library

[16]

Guodong Guo, Rui Guo, and Xin Li. 2013. Facial expression recognition influenced by human aging. IEEE Trans. Affective Computing 4, 3 (2013), 291--298.

Digital Library

[17]

Guodong Guo and Guowang Mu. 2010a. Human age estimation: What is the influence across race and gender? In Proc. CVPRW.

[18]

Guodong Guo and Guowang Mu. 2010b. A study of large-scale ethnicity estimation with gender and age variations. In Proc. CVPRW.

[19]

Guodong Guo and Guowang Mu. 2013. Joint estimation of age, gender and ethnicity: CCA vs. PLS. In Proc. FG. 1--6.

[20]

Guodong Guo, Guowang Mu, Yun Fu, and Thomas S. Huang. 2009. Human age estimation using bio-inspired features. In Proc. CVPR. 112--119.

[21]

Guodong Guo and Xiaolong Wang. 2012. A study on human age estimation under facial expression changes. In Proc. CVPR. 2547--2553.

[22]

Guodong Guo and Chao Zhang. 2014. A study on cross-population age estimation. In Proc. CVPR. 4257--4263.

Digital Library

[23]

Abdenour Hadid. 2011. Analyzing facial behavioral features from videos. In Proc. Int’l Workshop Human Behavior Unterstanding. 52--61.

Digital Library

[24]

Hu Han, Charles Otto, Xiaoming Liu, and Anil K. Jain. 2015. Demographic estimation from face images: Human vs. machine performance. IEEE Trans. Pattern Anal. Mach. Intell. 37, 6 (2015), 1148--1161.

Digital Library

[25]

Ivan Huerta, Carles Fernández, Carlos Segura, Javier Hernando, and Andrea Prati. 2015. A deep analysis on age estimation. Pattern Recognition Letters 68 (2015), 239--249.

Digital Library

[26]

Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. CAFFE: Convolutional architecture for fast feature embedding. In Proc. ACM MM. 675--678.

Digital Library

[27]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Proc. NIPS.

Digital Library

[28]

Andreas Lanitis, Chrisina Draganova, and Chris Christodoulou. 2004. Comparing different classifiers for automatic age estimation. IEEE Trans. Syst., Man, Cybern. B, Cybern. 34, 1 (2004), 621--628.

Digital Library

[29]

Changsheng Li, Qingshan Liu, Weishan Dong, Xiaobin Zhu, Jing Liu, and Hanqing Lu. 2015. Human age estimation based on locality and ordinal information. IEEE Trans. Cybernetics 45, 11 (2015), 2522--2534.

[30]

Changsheng Li, Qingshan Liu, Jing Liu, and Hanqing Lu. 2012. Learning ordinal discriminative features for age estimation. In Proc. CVPR. 2570--2577.

[31]

Hsuan-Tien Lin and Ling Li. 2012. Reduction from cost-sensitive ordinal ranking to weighted binary classification. Neural Computation 24, 5 (2012), 1329--1367.

Digital Library

[32]

Refik Can Malli, Mehmet Aygun, and Hazim Kemal Ekenel. 2016. Apparent age estimation using ensemble of deep learning models. In Proc. CVPR Workshops. 714--721.

[33]

Meredith Minear and Denise C. Park. 2004. A lifespan database of adult facial stimuli. Behavior Research Methods, Instruments, 8 Computers 36, 4 (2004), 630--633.

[34]

Zhenxing Niu, Mo Zhou, Le Wang, Xinbo Gao, and Gang Hua. 2016. Ordinal regression with multiple output CNN for age estimation. In Proc. CVPR. 4920--4928.

[35]

Jhony K. Pontes, Alceu S. Britto Jr., Clinton Fookes, and Alessandro L. Koerich. 2016. A flexible hierarchical approach for facial age estimation based on multiple features. Pattern Recognition 54 (2016), 34--51.

Digital Library

[36]

Haoyu Ren and Ze-Nian Li. 2014. Age estimation based on complexity-aware features. In Proc. ACCV. 115--128.

[37]

Karl Ricanek and Tamirat Tesafaye. 2006. MORPH: A longitudinal image database of normal adult age-progression. In Proc. FGR. 341--345.

Digital Library

[38]

Rasmus Rothe, Radu Timofte, and Luc Van Gool. 2016. Deep expectation of real and apparent age from a single image without facial landmarks. International Journal of Computer Vision (2016), 1--14.

Digital Library

[39]

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. 2015. ImageNet large scale visual recognition challenge. Int’l J. Computer Visionl (2015), 1--42.

[40]

Yi Sun, Xiaogang Wang, and Xiaoou Tang. 2015. Deeply learned face representations are sparse, selective, and robust. In Proc. CVPR. 2892--2900.

[41]

Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proc. CVPR. 1--9.

[42]

Yaniv Taigman, Ming Yang, Marc’Aurelio Ranzato, and Lior Wolf. 2014. DeepFace: Closing the gap to human-level performance in face verification. In Proc. CVPR. 1701--1708.

Digital Library

[43]

Zichang Tan, Shuai Zhou, Jun Wan, Zhen Lei, and Stan Z. Li. 2016. Age estimation based on a single network with soft softmax of aging modeling. In Proc. ACCV. 203--216.

[44]

Shangfei Wang, Shan Wu, Zhen Gao, and Qiang Ji. 2016. Facial expression recognition through modeling age-related spatial patterns. Multimedia Tools Appl. 75, 7 (2016), 3937--3954.

Digital Library

[45]

Xiaolong Wang, Rui Guo, and Chandra Kambhamettu. 2015. Deeply-learned feature for age estimation. In Proc. WACV. 534--541.

Digital Library

[46]

Xiaolong Wang and Chandra Kambhamettu. 2015. Age estimation via unsupervised neural networks. In Proc. FG. 1--6.

[47]

Huei-Fang Yang, Bo-Yao Lin, Kuang-Yu Chang, and Chu-Song Chen. 2015. Automatic age estimation from face images via deep ranking. In Proc. BMVC. 55.1--55.11.

[48]

Peng Yang, Lin Zhong, and Dimitris N. Metaxas. 2010. Ranking model for facial age estimation. In Proc. ICPR. 3404--3407.

Digital Library

[49]

Dong Yi, Zhen Lei, and Stan Z. Li. 2014. Age estimation by multi-scale convolutional network. In Proc. ACCV. 144--158.

[50]

Chao Zhang and Guodong Guo. 2013. Age estimation with expression changes using multiple aging subspaces. In Proc. IEEE BTAS. 1--6.

[51]

Zhanpeng Zhang, Ping Luo, Chen Change Loy, and Xiaoou Tang. 2014. Facial landmark detection by deep multi-task learning. In Proc. ECCV. 94--108.

Cited By

Zhou YYang BLiu ZWang QXiong P(2024)Cross-Domain Facial Expression Recognition by Combining Transfer Learning and Face-Cycle Generative Adversarial NetworkMultimedia Tools and Applications10.1007/s11042-024-18713-yOnline publication date: 11-Mar-2024
https://doi.org/10.1007/s11042-024-18713-y
Ali SGanapathi IAlsarhan TWerghi N(2023)Enhanced authentication system with robust features for the secure user template2023 IEEE 12th International Conference on Communication Systems and Network Technologies (CSNT)10.1109/CSNT57126.2023.10134714(742-747)Online publication date: 8-Apr-2023
https://doi.org/10.1109/CSNT57126.2023.10134714
Favorskaya MPakhirka A(2023)Age-Group Estimation of Facial Images Using Multi-task Ranking CNNIntelligent Decision Technologies10.1007/978-981-99-2969-6_13(147-156)Online publication date: 30-May-2023
https://doi.org/10.1007/978-981-99-2969-6_13
Show More Cited By

Index Terms

Joint Estimation of Age and Expression by Combining Scattering and Convolutional Networks
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Biometrics
  2. Machine learning
    1. Learning paradigms
      1. Multi-task learning

Recommendations

Research Progress of Facial Expression Recognition Based on Deep Learning
CAIH2020: Proceedings of the 2020 Conference on Artificial Intelligence and Healthcare

Facial expression recognition is a practical application in the field of artificial intelligence. This article focuses on the introduction of the principles of each facial expression recognition method based on deep learning in chronological order, as ...
Person-independent facial expression recognition method based on improved Wasserstein generative adversarial networks in combination with identity aware
Abstract
Since the distinction between two expressions is fairly vague, usually a subtle change in one part of the human face is enough to change a facial expression. Most of the existing facial expression recognition algorithms are not robust enough ...
Integrated Expression-Invariant Face Recognition with Constrained Optical Flow
PSIVT '09: Proceedings of the 3rd Pacific Rim Symposium on Advances in Image and Video Technology

Face recognition is one of the most intensively studied topics in computer vision and pattern recognition. A constrained optical flow algorithm, which combines the advantages of the unambiguous correspondence of feature point labeling and the flexible ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 14, Issue 1

February 2018

287 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/3173554

Editor:
Alberto Del Bimbo
University of Firenze, Italy

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 January 2018

Accepted: 01 September 2017

Revised: 01 September 2017

Received: 01 December 2016

Published in TOMM Volume 14, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Ministry of Science and Technology of Taiwan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
198
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)2

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhou YYang BLiu ZWang QXiong P(2024)Cross-Domain Facial Expression Recognition by Combining Transfer Learning and Face-Cycle Generative Adversarial NetworkMultimedia Tools and Applications10.1007/s11042-024-18713-yOnline publication date: 11-Mar-2024
https://doi.org/10.1007/s11042-024-18713-y
Ali SGanapathi IAlsarhan TWerghi N(2023)Enhanced authentication system with robust features for the secure user template2023 IEEE 12th International Conference on Communication Systems and Network Technologies (CSNT)10.1109/CSNT57126.2023.10134714(742-747)Online publication date: 8-Apr-2023
https://doi.org/10.1109/CSNT57126.2023.10134714
Favorskaya MPakhirka A(2023)Age-Group Estimation of Facial Images Using Multi-task Ranking CNNIntelligent Decision Technologies10.1007/978-981-99-2969-6_13(147-156)Online publication date: 30-May-2023
https://doi.org/10.1007/978-981-99-2969-6_13
Badr MSarhan AElbasiony R(2022)ICRLJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-21126743:1(79-92)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.3233/JIFS-211267
Qayyum ARazzak ITanveer MMazher M(2022)Spontaneous Facial Behavior Analysis Using Deep Transformer-based Framework for Child–computer InteractionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/353957720:2(1-17)Online publication date: 26-May-2022
https://dl.acm.org/doi/10.1145/3539577
Al-Garaawi NMorris TCootes T(2022)Fully automated age-weighted expression classification using real and apparent agePattern Analysis & Applications10.1007/s10044-021-01044-125:2(451-466)Online publication date: 1-May-2022
https://dl.acm.org/doi/10.1007/s10044-021-01044-1
Badr MElbasiony RSarhan A(2022)LRTI: landmark ratios with task importance toward accurate age estimation using deep neural networksNeural Computing and Applications10.1007/s00521-022-06955-634:12(9647-9659)Online publication date: 30-Jan-2022
https://doi.org/10.1007/s00521-022-06955-6
Sharma UTomar PAli SSaxena NBhadoria R(2021)Optimized Authentication System with High Security and PrivacyElectronics10.3390/electronics1004045810:4(458)Online publication date: 13-Feb-2021
https://doi.org/10.3390/electronics10040458
Zhang LCai JPeng FLong M(2021)A Benchmark Database for the Comparison of Face Morphing Detection Methods2021 International Conference on Electronic Information Technology and Smart Agriculture (ICEITSA)10.1109/ICEITSA54226.2021.00082(393-401)Online publication date: Dec-2021
https://doi.org/10.1109/ICEITSA54226.2021.00082
Bekhouche SDornaika FOuafi ATaleb-Ahmed A(2021)Human Facial Age Estimation: Handcrafted Features Versus Deep FeaturesEmerging Technologies in Biomedical Engineering and Sustainable TeleMedicine10.1007/978-3-030-14647-4_3(31-37)Online publication date: 18-Aug-2021
https://doi.org/10.1007/978-3-030-14647-4_3
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents