research-article

Multi-task Deep Neural Network for Joint Face Recognition and Facial Attribute Prediction

Authors:

Zhanxiong Wang,

Xiangyang XueAuthors Info & Claims

ICMR '17: Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval

Pages 365 - 374

https://doi.org/10.1145/3078971.3078973

Published: 06 June 2017 Publication History

Abstract

Deep neural networks have significantly improved the performance of face recognition and facial attribute prediction, which however are still very challenging on the million scale dataset, i.e. MegaFace. In this paper, we for the first time, advocate a multi-task deep neural network for jointly learning face recognition and facial attribute prediction tasks. Extensive experimental evaluation clearly demonstrates the effectiveness of our architecture. Remarkably, on the largest face recognition benchmark -- MegaFace dataset, our networks can achieve the Rank-1 identication accuracy of 77.74% and face verication accuracy 79.24% TAR at 10^-6 FAR, which are the best performance on the small protocol among all the publicly released methods.

References

[1]

Abrar H Abdulnabi, Gang Wang, Jiwen Lu, and Kui Jia. 2015. Multi-task cnn model for attribute prediction. IEEE TMM (2015).

Digital Library

[2]

Timo Ahonen, Abdenour Hadid, and Matti Pietikainen. 2006. Face description with local binary patterns: Application to face recognition. IEEE TPAMI (2006).

Digital Library

[3]

Lacey Best-Rowden, Hu Han, Charles Otto, Brendan F Klare, and Anil K Jain. 2014. Unconstrained face recognition: Identifying a person of interest from a media collection. IEEE Transactions on Information Forensics and Security (2014).

Digital Library

[4]

Xudong Cao, Yichen Wei, Fang Wen, and Jian Sun. 2014. Face alignment by explicit shape regression. IJCV 107, 2 (2014), 177--190.

Digital Library

[5]

Dong Chen, Xudong Cao, Liwei Wang, Fang Wen, and Jian Sun. 2012. Bayesian face revisited: A joint formulation. In ECCV.

Digital Library

[6]

Dong Chen, Shaoqing Ren, Yichen Wei, Xudong Cao, and Jian Sun. 2014. Joint cascade face detection and alignment. In ECCV.

[7]

Sumit Chopra, Raia Hadsell, and Yann LeCun. 2005. Learn- ing a similarity metric discriminatively, with application to face verification. In CVPR.

Digital Library

[8]

Max Ehrlich, Timothy J Shields, Timur Almaev, and Mohamed R Amer. 2016. Facial attributes classification using multi-task representation learning. In CVPR Workshops.

[9]

Pedro F. Felzenszwalb, Ross B. Girshick, David McAllester, and Deva Ramanan. 2010. Object Detection with Discriminatively Trained Part-Based Models. IEEE TPAMI (2010).

Digital Library

[10]

Yanwei Fu, Timothy M. Hospedales, Tao Xiang, Yuan Yao, and Shaogang Gong. 2014. Interestingness Prediction by Robust Learning to Rank. In ECCV.

[11]

M. Guillaumin, J. Verbeek, and C. Schmid. 2009. Is that you? Metric Learning Approaches for Face Identification. In CVPR.

[12]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385 (2015).

[13]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. In CoRR, vol. abs/1502.01852 .

[14]

G. B. Huang, V. Jain, and E. Learned-Miller. 2007. Unsupervised Joint Alignment of Complex Images. In IEEE International Conference on Computer Vision. 1--8.

[15]

Gary B Huang, Manu Ramesh, Tamara Berg, and Erik Learned- Miller. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report.

[16]

Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Co- variate Shift. In ICML.

Digital Library

[17]

Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. 2014. Caffe: Convolutional architecture for fast feature embedding. arXiv (2014).

Digital Library

[18]

Ira Kemelmacher-Shlizerman, Steven M Seitz, Daniel Miller, and Evan Brossard. 2016. The MegaFace Benchmark: 1 Million Faces for Recognition at Scale. In CVPR.

[19]

Neeraj Kumar, Alexander Berg, Peter N Belhumeur, and Shree Nayar. 2011. Describable visual attributes for face verification and image search. IEEE TPMAI (2011).

Digital Library

[20]

Neeraj Kumar, Alexander C. Berg, Peter N. Belhumeur, and Shree K. Nayar. 2009. Attribute and Simile Classifiers for Face Verification. In ICCV.

[21]

Neeraj Kumar, Alexander C Berg, Peter N Belhumeur, and Shree K Nayar. 2009. Attribute and simile classifiers for face verification. In ICCV. IEEE, 365--372.

[22]

Jinguo Liu, Yafeng Deng, and Chang Huang. 2015. Targeting ultimate accuracy: Face recognition via deep embedding. arXiv preprint arXiv:1506.07310 (2015).

[23]

Xiaodong Liu, Jianfeng Gao, Xiaodong He, Li Deng, Kevin Duh, and Ye-Yi Wang. Representation learning using multi-task deep neural networks for semantic classification and information re- trieval.

[24]

Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2015. Deep learning face attributes in the wild. In ICCV.

Digital Library

[25]

David G. Lowe. 2004. Distinctive Image Features from Scale- Invariant Keypoints. International Journal of Computer Vision 60 (2004).

Digital Library

[26]

Hong-Wei Ng and Stefan Winkler. 2014. A data-driven approach to cleaning large face datasets. In ICIP.

[27]

Hieu V. Nguyen and Li Bai. 2010. Cosine Similarity Metric Learning for Face Verification. In ACCV.

Digital Library

[28]

Timo Ojala, Matti Pietikäinen, and Topi Mäenpää. 2002. Gray Scale and Rotation Invariant Texture Classification with Local Binary Patterns. IEEE TPAMI (2002).

Digital Library

[29]

Rajeev Ranjan, Vishal M Patel, and Rama Chellappa. 2016. Hy- perface: A deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. arXiv preprint arXiv:1603.01249 (2016).

[30]

Shaoqing Ren, Xudong Cao, Yichen Wei, and Jian Sun. 2014. Face alignment at 3000 fps via regressing local binary features. In CVPR. IEEE.

Digital Library

[31]

Ethan M. Rudd, Manuel Gunther, and Terrance E. Boult. 2016. MOON:A Mixed Objective Optimization Network for the Recognition of Facial Attributes. In ECCV.

[32]

Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. Facenet: A unified embedding for face recognition and clustering. In CVPR .

[33]

William Robson Schwartz, Huimin Guo, and Larry S Davis. 2010. A robust and scalable approach to face identification. In ECCV.

[34]

Yi Sun, Ding Liang, Xiaogang Wang, and Xiaoou Tang. 2015. Deepid3: Face recognition with very deep neural networks. arXiv preprint arXiv:1502.00873 (2015).

[35]

Yi Sun, Xiaogang Wang, and Xiaoou Tang. 2015. Deeply learned face representations are sparse, selective, and robust. In CVPR.

[36]

Yaniv Taigman, Ming Yang, Marc-Aurelio Ranzato, and Lior Wolf. 2014. Deepface: Closing the gap to human-level performance in face verification. In CVPR. 1701--1708.

Digital Library

[37]

Yaniv Taigman, Ming Yang, MarcAurelio Ranzato, and Lior Wolf. 2015. Web-scale training for face identification. In CVPR.

[38]

Bart Thomee, David A Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. 2015. The new data and new challenges in multimedia research. arXiv preprint arXiv:1503.01817 (2015).

[39]

Georgios Tzimiropoulos and Maja Pantic. 2014. Gauss-newton deformable part models for face alignment in-the-wild. In CVPR. IEEE, 1851--1858.

Digital Library

[40]

Paul Viola and Michael Jones. 2001. Rapid Object Detection using a Boosted Cascade of Simple Features. In CVPR.

[41]

P. Viola, J. Platt, and C. Zhang. 2005. Multiple instance boosting for object detection. In NIPS.

Digital Library

[42]

Jing Wang, Yu Cheng, and Rogerio Schmidt Feris. 2016. Walk and Learn: Facial Attribute Representation Learning from Egocentric Video and Contextual Data. arXiv preprint arXiv:1604.06433 (2016).

[43]

Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. 2016. A discriminative feature learning approach for deep face recognition. In ECCV.

[44]

Xiang Wu, Ran He, and Zhenan Sun. 2015. A Lightened CNN for Deep Face Representation. arXiv preprint arXiv:1511.02683 (2015).

[45]

Xiang Wu, Ran He, Zhenan Sun, and Tieniu Tan. 2016. A Light CNN for Deep Face Representation with Noisy Labels. arxiv (2016).

[46]

Xuehan Xiong and Fernando De la Torre. 2013. Supervised descent method and its applications to face alignment. In CVPR.

Digital Library

[47]

Heng Yang, Wenxuan Mou, Yichi Zhang, Ioannis Patras, Hatice Gunes, and Peter Robinson. 2015. Face Alignment Assisted by Head Pose Estimation. arXiv preprint arXiv:1507.03148 (2015).

[48]

Hao Ye, Weiyuan Shao, Hong Wang, Jianqi Ma, Li Wang, Yingbin Zheng, and Xiangyang Xue. 2016. Face Recognition via Active Annotation and Learning. In Proceedings of the 2016 ACM on Multimedia Conference. 1058--1062.

Digital Library

[49]

Dong Yi, Zhen Lei, Shengcai Liao, and Stan Z Li. 2014. Learning face representation from scratch. arXiv preprint arXiv:1411.7923 (2014).

[50]

Jie Zhang, Shiguang Shan, Meina Kan, and Xilin Chen. 2014. Coarse-to-fine auto-encoder networks (cfan) for real-time face alignment. In ECCV. Springer.

[51]

K. Zhang, Z. Zhang, Z. Li, and Y. Qiao. 2016. Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks. IEEE SP Letters (2016).

[52]

Zhanpeng Zhang, Ping Luo, Chen Change Loy, and Xiaoou Tang. 2014. Facial landmark detection by deep multi-task learning. In ECCV.

[53]

Yang Zhong, Josephine Sullivan, and Haibo Li. 2016. Face At- tribute Prediction Using Off-the-Shelf CNN Features. In arxiv.

[54]

Qiang Zhou, Gang Wang, Kui Jia, and Qi Zhao. 2013. Learning to share latent tasks for action recognition. In ICCV.

Digital Library

[55]

Shizhan Zhu, Cheng Li, Chen Change Loy, and Xiaoou Tang. 2015. Face Alignment by Coarse-to-Fine Shape Searching. In CVPR.

[56]

Xiangxin Zhu and Deva Ramanan. 2012. Face detection, pose estimation, and landmark localization in the wild. In CVPR.

Cited By

Lee YJang SBae HJeon TLee S(2024)Multitask Learning Strategy with Pseudo-Labeling: Face Recognition, Facial Landmark Detection, and Head Pose EstimationSensors10.3390/s2410321224:10(3212)Online publication date: 18-May-2024
https://doi.org/10.3390/s24103212
An WWu G(2024)Hybrid Spatial-Channel Attention Mechanism for Cross-Age Face RecognitionElectronics10.3390/electronics1307125713:7(1257)Online publication date: 28-Mar-2024
https://doi.org/10.3390/electronics13071257
Huang ZZhang JShan H(2023)When Age-Invariant Face Recognition Meets Face Age Synthesis: A Multi-Task Learning Framework and a New BenchmarkIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2022.321788245:6(7917-7932)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1109/TPAMI.2022.3217882
Show More Cited By

Index Terms

Multi-task Deep Neural Network for Joint Face Recognition and Facial Attribute Prediction
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Object identification
      2. Computer vision tasks
        Biometrics
  2. Machine learning
    1. Learning paradigms
      1. Multi-task learning
      2. Supervised learning
        Supervised learning by classification

Recommendations

A multi-task model for simultaneous face identification and facial expression recognition

Regarded as two independent tasks, both face identification and facial expression recognition perform poorly given small size training sets. To address this problem, we propose a multi-task facial inference model (MT-FIM) for simultaneous face ...
Deep face recognition using imperfect facial data
Abstract
Today, computer based face recognition is a mature and reliable mechanism which is being practically utilised for many access control scenarios. As such, face recognition or authentication is predominantly performed using ‘perfect’ ...
Highlights
- We show the performance of machine learning for face recognition using partial faces and other manipulations of the face such as rotation and zooming which ...
Unconstrained Still/Video-Based Face Verification with Deep Convolutional Neural Networks

Over the last 5 years, methods based on Deep Convolutional Neural Networks (DCNNs) have shown impressive performance improvements for object detection and recognition problems. This has been made possible due to the availability of large annotated ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMR '17: Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval

June 2017

524 pages

ISBN:9781450347013

DOI:10.1145/3078971

General Chairs:
Bogdan Ionescu
University Politehnica of Bucharest, Romania
,
Nicu Sebe
University of Trento, Italy
,
Program Chairs:
Jiashi Feng
National University of Singapore, Singapore
,
Martha Larson
Radboud University & Delft University of Technology, The Netherlands
,
Rainer Lienhart
University of Augsburg, Germany
,
Cees Snoek
University of Amsterdam & Qualcomm Research Netherlands, The Netherlands

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 June 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Shanghai Sailing Program
Shanghai Municipal Science and Technology Commission

Conference

ICMR '17

Sponsor:

SIGMM

ICMR '17: International Conference on Multimedia Retrieval

June 6 - 9, 2017

Bucharest, Romania

Acceptance Rates

ICMR '17 Paper Acceptance Rate 33 of 95 submissions, 35%;

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

45
Total Citations
View Citations
669
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)3

Reflects downloads up to 27 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lee YJang SBae HJeon TLee S(2024)Multitask Learning Strategy with Pseudo-Labeling: Face Recognition, Facial Landmark Detection, and Head Pose EstimationSensors10.3390/s2410321224:10(3212)Online publication date: 18-May-2024
https://doi.org/10.3390/s24103212
An WWu G(2024)Hybrid Spatial-Channel Attention Mechanism for Cross-Age Face RecognitionElectronics10.3390/electronics1307125713:7(1257)Online publication date: 28-Mar-2024
https://doi.org/10.3390/electronics13071257
Huang ZZhang JShan H(2023)When Age-Invariant Face Recognition Meets Face Age Synthesis: A Multi-Task Learning Framework and a New BenchmarkIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2022.321788245:6(7917-7932)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1109/TPAMI.2022.3217882
Talemi NKashiani HMalakshan SSaadabadi MNajafzadeh NAkyash MNasrabadi N(2023)AAFACE: Attribute-Aware Attentional Network for Face Recognition2023 IEEE International Conference on Image Processing (ICIP)10.1109/ICIP49359.2023.10222666(1940-1944)Online publication date: 8-Oct-2023
https://doi.org/10.1109/ICIP49359.2023.10222666
Li ZYin BYao TGuo JDing SChen SLiu C(2023)Sibling-Attack: Rethinking Transferable Adversarial Attacks against Face Recognition2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.02359(24626-24637)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.02359
Tan ZGuo G(2023)Face Recognition Research and DevelopmentHandbook of Face Recognition10.1007/978-3-031-43567-6_1(3-36)Online publication date: 30-Dec-2023
https://doi.org/10.1007/978-3-031-43567-6_1
Zhu JGuo QJuefei-Xu FHuang YLiu YPu G(2023)Masked Faces with Faced MasksComputer Vision – ECCV 2022 Workshops10.1007/978-3-031-25056-9_24(360-377)Online publication date: 15-Feb-2023
https://doi.org/10.1007/978-3-031-25056-9_24
Zhao JYan SFeng J(2022)Towards Age-Invariant Face RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2020.301142644:1(474-487)Online publication date: 1-Jan-2022
https://doi.org/10.1109/TPAMI.2020.3011426
Hassani AShair ZUd Duala Refat RMalik H(2022)Distilling Facial Knowledge with Teacher-Tasks: Semantic-Segmentation-Features For Pose-Invariant Face-Recognition2022 IEEE International Conference on Image Processing (ICIP)10.1109/ICIP46576.2022.9897793(741-745)Online publication date: 16-Oct-2022
https://doi.org/10.1109/ICIP46576.2022.9897793
Smirnov EGaraev NGalyuk VLukyanets E(2022)Prototype Memory for Large-Scale Face Representation LearningIEEE Access10.1109/ACCESS.2022.314605910(12031-12046)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3146059
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten