Skip to main content
SpringerLink
Log in

Deep learning face representation by fixed erasing in facial landmarks

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Face verification (FV) is a challenging problem, because occlusion, posture, illumination, aging will affect the accuracy of FV. Deep convolutional neural networks (DCNNs) have been widely used in many computer vision tasks. Due to the strong feature learning ability, DCNNs improve the accuracy of FV while they also bring some problems such as overfitting. Erasing has been proven to be an effective method for reducing overfitting, while the erasing position is seldom considered. We analyse the effect of different erasing positions and propose a novel data augmentation method especially for FV, called Fixed Erasing(FE). We randomly erase some face images with random size of rectangles centered on fixed facial landmarks such as the centers of eyes, tip of noses and the corners of mouths. Our method can alleviate the risk of overfitting and make the models learn more robust features. And our method can be easily merged with most DCNN-based FV models through a few lines of code. Extensive experiments demonstrate that FE can improve the performance of most recently proposed FV models on several popular benchmarks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Cao Q, Li S, Xie W, Parkhi OM, Zisserman A (2017) Vggface2: a dataset for recognising faces across pose and age. CoRR, arXiv:1710.08092

  2. Chen B, Deng W, Du J (2017) Noisy softmax: improving the generalization ability of DCNN via postponing the early softmax saturation. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pp 4021–4030

  3. Chen S, Liu Y, Gao X, Han Z (2018) Mobilefacenets: efficient cnns for accurate real-time face verification on mobile devices. CoRR, arXiv:1804.07573

  4. Deng J, Guo J, Zafeiriou S (2018) Arcface: additive angular margin loss for deep face recognition CoRR, arXiv:1801.07698

  5. Deng W, Hu J, Zhang N, Chen B, Guo J (2017) Fine-grained face verification: Fglfw database, baselines, and human-dcmn partnership. Pattern Recogn 66:63–73

    Article  Google Scholar 

  6. Ding C, Tao D (2018) Trunk-branch ensemble convolutional neural networks for video-based face recognition. IEEE Trans Pattern Anal Mach Intell 40(4):1002–1014

    Article  Google Scholar 

  7. Dong X, Yan Y, Tan M, Yi Y, Tsang IW (2018) Late fusion via subspace search with consistency preservation. IEEE Trans Image Process PP(99):1–1

    MATH  Google Scholar 

  8. Guo Y, Zhang L, Hu Y, He X, Gao J (2016) Ms-celeb-1m: a dataset and benchmark for large-scale face recognition. In: Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III, pp 87–102

  9. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition, CVPR Las Vegas, NV, USA, June 27-30, 2016. IEEE Computer Society, p 2016

  10. Huang GB, Ramesh M, Berg T, Learned-Miller E (2007) Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical Report 07-49. University of Massachusetts, Amherst

  11. Huang R, Zhang S, Li T, He R (2017) Beyond face rotation: global and local perception GAN for photorealistic and identity preserving frontal view synthesis. In: IEEE international conference on computer vision, ICCV 2017, Venice, Italy, October 22-29, 2017, pp 2458–2467

  12. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Bach FR, Blei DM (eds) Proceedings of the 32nd international conference on machine learning, ICML 2015, Lille, France, 6-11 July 2015, vol 37 of JMLR workshop and conference proceedings, pp 448–456, JMLR.org

  13. Jan A, Ding H, Meng H, Chen L, Li H (2018) Accurate facial parts localization and deep learning for 3d facial expression recognition. CoRR, arXiv:1803.05846

  14. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: Convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093

  15. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Bartlett PL, Pereira FCN, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems 25: 26th annual conference on neural information processing systems 2012. Proceedings of a meeting held December 3-6, 2012, Lake Tahoe, Nevada, United States, pp 1106–1114

  16. Liu J, Deng Y, Bai T, Huang C (2015) Targeting ultimate accuracy: face recognition via deep embedding. CoRR, arXiv:1506.07310

  17. Liu W, Wen Y, Yu Z, Li M, Raj B, Song L (2017) Sphereface: deep hypersphere embedding for face recognition. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017. IEEE Computer Society, pp 6738–6746

  18. Liu Y, Zhang L, Nie L, Yan Y, Rosenblum DS (2016) Fortune teller: predicting your career path. In: Proceedings of the thirtieth AAAI conference on artificial intelligence, February 12-17, 2016, Phoenix, Arizona, USA, pp 201–207

  19. Liu Y, Zheng Y, Liang Y, Liu S, Rosenblum DS (2016) Urban water quality prediction based on multi-task multi-view learning. In: Proceedings of the twenty-fifth international joint conference on artificial intelligence, IJCAI 2016, New York, NY, USA, 9-15 July 2016, pp 2576–2581

  20. Liu Y, Li H, Wang X (2017) Rethinking feature discrimination and polymerization for large-scale recognition. CoRR, arXiv:1710.00870

  21. Liu Y, Li H, Yan J, Wei F, Wang X, Tang X (2017) Recurrent scale approximation for object detection in CNN. In: IEEE International Conference on Computer Vision, ICCV 2017 Venice, Italy, October 22-29, 2017, pp 571–579

  22. Nguyen HV, Bai L (2010) Cosine similarity metric learning for face verification. In: Computer vision - ACCV 2010 - 10th Asian conference on computer vision, Queenstown, New Zealand, November 8-12 2010, Revised selected papers, Part II, pp 709–720

  23. Ranjan R, Castillo CD, Chellappa R (2017) L2-constrained softmax loss for discriminative face verification. CoRR, arXiv:1703.09507

  24. Salimans T, Kingma DP (2016) Weight normalization: a simple reparameterization to accelerate training of deep neural networks. In: Lee DD, Sugiyama M, von Luxburg U, Guyon I, Garnett R (eds) Advances in neural information processing systems 29: annual conference on neural information processing systems 2016, December 5-10, 2016, Barcelona, Spain, p 901

  25. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. CoRR, arXiv:1409.1556

  26. Sun Y, Wang X, Tang X (2014) Deep learning face representation by joint identification-verification. Adv Neural Inf Proces Syst 27:1988–1996

    Google Scholar 

  27. Sun Y, Wang X, Tang X (2014) Deep learning face representation from predicting 10,000 classes. In: Annual conference on neural information processing systems 2014. IEEE Computer Society, pp 1891–1898

  28. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In: Singh SP, Markovitch S (eds) Proceedings of the thirty-first AAAI conference on artificial intelligence, February 4-9 San Francisco, California, USA. AAAI Press, p 2017

  29. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: 2016 IEEE conference on computer vision and pattern recognition, CVPR Las Vegas, NV, USA, June 27-30, 2016. IEEE Computer Society, p 2016

  30. Taigman Y, Yang M, Ranzato M’A, Wolf L (2014) Deepface: closing the gap to human-level performance in face verification. In: 2014 IEEE conference on computer vision and pattern recognition, CVPR 2014, Columbus, OH, USA, June 23-28, 2014, pp 1701–1708

  31. Wan L, Zeiler MD, Zhang S, LeCun Y, Fergus R (2013) Regularization of neural networks using dropconnect. In: Proceedings of the 30th international conference on machine learning, ICML 2013, Atlanta, GA, USA, 16-21 June 2013, vol 28 of JMLR workshop and conference proceedings, pp 1058–1066, JMLR.org

  32. Wang F, Liu W, Liu H, Cheng J (2018) Additive margin softmax for face verification. CoRR, arXiv:1801.05599

  33. Wang F, Xiang X, Cheng J, Yuille AL (2017) Normface: L2 hypersphere embedding for face verification. In: Proceedings of the 2017 ACM on multimedia conference, MM 2017, Mountain View, CA, USA, October 23-27, 2017, pp 1041–1049

  34. Wang X, Shrivastava A, Gupta A (2017) A-fast-rcnn: hard positive generation via adversary for object detection. In: Conference on Computer Vision and Pattern Recognition (CVPR)

  35. Wen Y, Zhang K, Li Z, Qiao Y (2016) A discriminative feature learning approach for deep face recognition. In: Leibe B, Matas J, Sebe N, Welling M (eds) Computer vision - ECCV 2016 - 14th European conference, Amsterdam, The Netherlands, October 11-14, 2016 Proceedings, Part VII, vol 9911 of Lecture notes in computer science. Springer, pp 499–515

  36. Wu Y, He K (2018) Group normalization. CoRR, arXiv:1803.08494

  37. Xie L, Wang J, Wei Z, Wang M, Tian Q (2016) Disturblabel: regularizing CNN on the loss layer. In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pp 4753–4762

  38. Ye L, Nie L, Lei H, Zhang L, Rosenblum DS (2016) Action2activity: recognizing complex activities from sensor data

  39. Ye L, Nie L, Li L, Rosenblum D (2016) From action to activity: sensor-based activity recognition. Neurocomputing 181:108–115

    Article  Google Scholar 

  40. Yi D, Lei Z, Liao S, Li SZ (2014) Learning face representation from scratch. CoRR, arXiv:1411.7923

  41. Yu J, Rui Y, Tang YY, Tao D (2014) High-order distance-based multiview stochastic learning in image classification. IEEE Trans Cybernetics 44(12):2431–2442

    Article  Google Scholar 

  42. Yu J, Tao D, Li J, Cheng J (2014) Semantic preserving distance metric learning and applications. Inf Sci 281:674–686

    Article  MathSciNet  Google Scholar 

  43. Yuan C, Guo J, Feng P, Zhao Z, Xu C, Wang T, Choe G, Duan K (2019) A jointly learned deep embedding for person re-identification. Neurocomputing 330:127–137

    Article  Google Scholar 

  44. Zeiler MD, Fergus R (2013) Stochastic pooling for regularization of deep convolutional neural networks. CoRR, arXiv:1301.3557

  45. Zhang K, Zhang Z, Li Z, Yu Q (2016) Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process Lett 23(10):1499–1503

    Article  Google Scholar 

  46. Zhang S, Zhu X, Lei Z, Shi H, Wang X, Li SZ (2017) Faceboxes: a CPU real-time face detector with high accuracy. In: 2017 IEEE international joint conference on biometrics, IJCB 2017, Denver, CO, USA, October 1-4, 2017, pp 1–9

  47. Zhong Z, Zheng L, Kang G, Li S, Yi Y (2017) Random erasing data augmentation. CoRR, arXiv:1708.04896

Download references

Acknowledgements

This work was supported in part by the Natural Science Foundation of China under Grant U1536203, in part by the National key research and development program of China(2016QY01W0200), in part by the Major Scientific and Technological Project of Hubei Province (2018AAA068).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China

    Jie Lei, BaiYan Zhang & HeFei Ling

  2. School of Software, Nanchang University, Nanchang, 330069, China

    Jie Lei

Authors
  1. Jie Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. BaiYan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. HeFei Ling
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to HeFei Ling.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lei, J., Zhang, B. & Ling, H. Deep learning face representation by fixed erasing in facial landmarks. Multimed Tools Appl 78, 27703–27718 (2019). https://doi.org/10.1007/s11042-019-07892-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-07892-8

Keywords

Search

Navigation