skip to main content
10.1145/3323873.3325053acmconferencesArticle/Chapter ViewAbstractPublication PagesicmrConference Proceedingsconference-collections
short-paper

Increasingly Packing Multiple Facial-Informatics Modules in A Unified Deep-Learning Model via Lifelong Learning

Published: 05 June 2019 Publication History

Abstract

Simultaneously running multiple modules is a key requirement for a smart multimedia system for facial applications including face recognition, facial expression understanding, and gender identification. To effectively integrate them, a continual learning approach to learn new tasks without forgetting is introduced. Unlike previous methods growing monotonically in size, our approach maintains the compactness in continual learning. The proposed packing-and-expanding method is effective and easy to implement, which can iteratively shrink and enlarge the model to integrate new functions. Our integrated multitask model can achieve similar accuracy with only 39.9% of the original size.

References

[1]
Martin Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, et al. 2016. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv (2016).
[2]
Q. Cao, L. Shen, W. Xie, O. M. Parkhi, and A. Zisserman. 2018. VGGFace2: A dataset for recognising faces across pose and age. In Proceedings of IEEE FG.
[3]
Eran Eidinger, Roee Enbar, and Tal Hassner. 2014. Age and gender estimation of unfiltered faces. IEEE Trans. Inf. Forensics Security (2014).
[4]
Sergio Escalera, Mercedes Torres Torres, Brais Martinez, Xavier Baró, Hugo Jair Escalante, Isabelle Guyon, Georgios Tzimiropoulos, Ciprian Corneou, Marc Oliu, Mohammad Ali Bagheri, et al. 2016. Chalearn looking at people and faces of the world: Face analysis workshop and challenge 2016. In Proceedings of IEEE CVPRW.
[5]
Ariel Gordon, Elad Eban, Ofir Nachum, Bo Chen, Hao Wu, Tien-Ju Yang, and Edward Choi. 2018. Morphnet: Fast & simple resource-constrained structure learning of deep networks. In Proceedings of IEEE CVPR.
[6]
Song Han, Huizi Mao, and William J Dally. 2016. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. ICLR (2016).
[7]
Corentin Kervadec, Valentin Vielzeuf, Stéphane Pateux, Alexis Lechervy, Frédéric Jurie, and Cesson-Sévigné. 2018. CAKE: Compact and Accurate K-dimensional representation of Emotion. In Proceedings of BMVC.
[8]
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences (2017).
[9]
Erik Learned-Miller, Gary B Huang, Aruni RoyChowdhury, Haoxiang Li, and Gang Hua. 2016. Labeled faces in the wild: A survey. In Advances in face detection and facial image analysis. Springer.
[10]
Jia-Hong Lee, Yi-Ming Chan, Ting-Yen Chen, and Chu-Song Chen. 2018. Joint Estimation of Age and Gender from Unconstrained Face Images using Lightweight Multi-task CNN for Mobile Applications. In Proceedings of IEEE MIPR.
[11]
Gil Levi and Tal Hassner. 2015. Age and gender classification using convolutional neural networks. In Proceedings of IEEE CVPRW.
[12]
Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. 2017. SphereFace: Deep Hypersphere Embedding for Face Recognition. In Proceedings of IEEE CVPR.
[13]
Arun Mallya and Svetlana Lazebnik. 2018. Packnet: Adding multiple tasks to a single network by iterative pruning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7765--7773.
[14]
James L McClelland, Bruce L McNaughton, and Randall C O'reilly. 1995. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological review (1995).
[15]
Michael McCloskey and Neal J Cohen. 1989. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation. Elsevier.
[16]
Ali Mollahosseini, Behzad Hasani, and Mohammad H Mahoor. 2017. AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild. IEEE Trans. Affective Comput. (2017).
[17]
Amal Rannen Ep Triki, Rahaf Aljundi, Matthew Blaschko, and Tinne Tuytelaars. 2017. Encoder based lifelong learning. In Proceedings of ICCV.
[18]
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. 2017. icarl: Incremental classifier and representation learning. In Proceedings of IEEE CVPR.
[19]
Rasmus Rothe, Radu Timofte, and Luc Van Gool. 2016. Deep expectation of real and apparent age from a single image without facial landmarks. Int. J. Comput. Vis. (IJCV) (2016).
[20]
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. 2016. Progressive neural networks. arXiv (2016).
[21]
Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. Facenet: A unified embedding for face recognition and clustering. In Proceedings of IEEE CVPR.
[22]
Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. 2017. Continual learning with deep generative replay. In Proceedings of NeurIPS.
[23]
Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A Alemi. 2017. Inception-v4, inception-resnet and the impact of residual connections on learning. In Proceedings of AAAI.
[24]
Sebastian Thrun. 1995. -A Lifelong Learning Perspective for Mobile Robot Control. In Intelligent Robots and Systems. Elsevier.
[25]
Tianjun Xiao, Jiaxing Zhang, Kuiyuan Yang, Yuxin Peng, and Zheng Zhang. 2014. Error-driven incremental learning in deep convolutional neural network for large-scale image classification. In Proceedings of ACM-MM.
[26]
Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. 2018. Lifelong Learning with Dynamically Expandable Networks. In Proceedings of ICLR.
[27]
Jiabei Zeng, Shiguang Shan, and Xilin Chen. 2018. Facial Expression Recognition with Inconsistently Annotated Datasets. In Proceedings of ECCV. Springer.
[28]
Friedemann Zenke, Ben Poole, and Surya Ganguli. 2017. Continual Learning Through Synaptic Intelligence. In Proceedings of ICML.
[29]
Michael Zhu and Suyog Gupta. 2017. To prune, or not to prune: exploring the efficacy of pruning for model compression. In Proceedings of NeurIPS Workshop on Machine Learning of Phones and other Consumer Devices.

Cited By

View all
  • (2024)AnyFace++: Deep Multi-Task, Multi-Domain Learning for Efficient Face AISensors10.3390/s2418599324:18(5993)Online publication date: 15-Sep-2024
  • (2024)Expression Complementary Disentanglement Network for Facial Expression RecognitionChinese Journal of Electronics10.23919/cje.2022.00.35133:3(742-752)Online publication date: May-2024
  • (2024)CP-Prompt: Composition-Based Cross-modal Prompting for Domain-Incremental Continual LearningProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681481(2729-2738)Online publication date: 28-Oct-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICMR '19: Proceedings of the 2019 on International Conference on Multimedia Retrieval
June 2019
427 pages
ISBN:9781450367653
DOI:10.1145/3323873
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 June 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. compact model
  2. continual learning
  3. deep learning
  4. facial informatics
  5. lifelong learning
  6. neural networks

Qualifiers

  • Short-paper

Conference

ICMR '19
Sponsor:

Acceptance Rates

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)53
  • Downloads (Last 6 weeks)1
Reflects downloads up to 16 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)AnyFace++: Deep Multi-Task, Multi-Domain Learning for Efficient Face AISensors10.3390/s2418599324:18(5993)Online publication date: 15-Sep-2024
  • (2024)Expression Complementary Disentanglement Network for Facial Expression RecognitionChinese Journal of Electronics10.23919/cje.2022.00.35133:3(742-752)Online publication date: May-2024
  • (2024)CP-Prompt: Composition-Based Cross-modal Prompting for Domain-Incremental Continual LearningProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681481(2729-2738)Online publication date: 28-Oct-2024
  • (2024)Emerging Frontiers in Human–Robot InteractionJournal of Intelligent & Robotic Systems10.1007/s10846-024-02074-7110:2Online publication date: 18-Mar-2024
  • (2024)A novel deep learning approach for facial emotion recognition: application to detecting emotional responses in elderly individuals with Alzheimer’s diseaseNeural Computing and Applications10.1007/s00521-024-10938-0Online publication date: 30-Dec-2024
  • (2024)The role of lifelong machine learning in bridging the gap between human and machine learning: A scientometric analysisWIREs Data Mining and Knowledge Discovery10.1002/widm.152614:2Online publication date: 10-Jan-2024
  • (2023)Toward Label-Efficient Emotion and Sentiment AnalysisProceedings of the IEEE10.1109/JPROC.2023.3309299111:10(1159-1197)Online publication date: Oct-2023
  • (2023)Robust Object Detection Against Multi-Type Corruption Without Catastrophic Forgetting During Adversarial Training Under Harsh Autonomous-Driving EnvironmentsIEEE Access10.1109/ACCESS.2023.325862611(26862-26876)Online publication date: 2023
  • (2023)Large-scale comparison and demonstration of continual learning for adaptive data-driven building energy predictionApplied Energy10.1016/j.apenergy.2023.121481347(121481)Online publication date: Oct-2023
  • (2023)Fast Image Classification Algorithms Based on Sequential AnalysisJournal of Mathematical Sciences10.1007/s10958-023-06524-9273:4(628-638)Online publication date: 22-Jun-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media