Skip to main content
SpringerLink
Log in

WeightFace: weight adaptive scaling loss for face recognition

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The discriminability between distinct classes in the embedding space is improved by margin-based loss functions like SphereFace, CosFace and Arcface. More recently, face quality is introduced to face recognition to adaptively adjust the margin. However, these methods ignore the class imbalance problem, which affects the distribution of each class in the real embedding space and misleads the classification results of minority class. In this paper, a novel loss (WeightFace) is proposed to learn the scale parameter adaptively guided by class weight to address the class imbalance problem. We have proved that the minority class requires a larger scale parameter, since the minority class get a relatively smaller gradient so that the minority class has an intra-class variation comparable to the majority class. The weight magnitude is used to connect classes and scale parameters. This allows different classes distributing properly in real space and the test accuracy is boosted. Extensive experiments on popular benchmarks demonstrate the superiority of our WeightFace over state-of-the-arts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Algorithm 2

Similar content being viewed by others

Data Availability

Date and code will be made available on reasonable request.

References

  1. Boutros F, Damer N, Kirchbuchner F et al (2021) Self-restrained triplet loss for accurate masked face recognition. Pattern Recognit 124:473–108

    Google Scholar 

  2. Boutros F, Damer N, Kirchbuchner F et al (2022) Elasticface: Elastic margin loss for deep face recognition. In: 2022 IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW). IEEE, pp 1577–1586

  3. Chen S, Liu Y, Gao X et al (2018) Mobilefacenets: Efficient cnns for accurate real-time face verification on mobile devices. In: 13th chinese conference on biometric recognition (CCBR), pp 428–438

  4. Deng J, Guo J, Xue N et al (2019) Arcface: Additive angular margin loss for deep face recognition. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, pp 4685–4694

  5. Hadsell R, Chopra S, LeCun Y (2006) Dimensionality reduction by learning an invariant mapping. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06), vol 2. IEEE, pp 1735–1742

  6. Han D, Kim J, Kim J (2017) Deep pyramidal residual networks. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 6307–6315

  7. He H, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21:1263–1284

    Article  Google Scholar 

  8. He H, Bai Y, Garcia EA et al (2008) Adasyn: adaptive synthetic sampling approach for imbalanced learning. In: 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence). IEEE, pp 1322–1328

  9. He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 770–778

  10. Huang C, Li Y, Loy CC et al (2020a) Deep imbalanced learning for face recognition and attribute prediction. IEEE Trans Pattern Anal Mach Intell 42:2781–2794

    Article  Google Scholar 

  11. Huang GB, Mattar MA, Berg TL et al (2008) Labeled faces in the wild: a database forstudying face recognition in unconstrained environments

  12. Huang Y, Wang Y, Tai Y et al (2020b) Curricularface: adaptive curriculum learning loss for deep face recognition. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, pp 5900–5909

  13. Khalid SS, Awais M, Feng Z et al (2022) Npt-loss: Demystifying face recognition losses with nearest proxies triplet. IEEE Trans Pattern Anal Mach Intell:1–1

  14. Kim M, Jain AK, Liu X (2022) Adaface: quality adaptive margin for face recognition. In: 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR), vol 738. IEEE, pp 729–18

  15. Krawczyk B, Woźniak M, Schaefer G (2014) Cost-sensitive decision tree ensembles for effective imbalanced classification. Appl Soft Comput 14:554–562

    Article  Google Scholar 

  16. Li S, Xu J, Xu X et al (2021) Spherical confidence learning for face recognition. In: 2021 IEEE/CVF conference on computer vision and pattern recognition (CVPR), vol 632. IEEE, pp 624–15

  17. Li Y, Wu B, Zhao Y et al (2019) Handling missing labels and class imbalance challenges simultaneously for facial action unit recognition. Multimed Tools Appl:1–24

  18. Ling H, Wu J, Huang J et al (2019) Attention-based convolutional neural network for deep face recognition. Multimed Tools Appl 79:5595–5616

    Article  Google Scholar 

  19. Liu B, Deng W, Zhong Y et al (2019) Fair loss: Margin-aware reinforcement learning for deep face recognition. In: 2019 IEEE/CVF international conference on computer vision (ICCV), vol 060. IEEE, pp 051–10

  20. Liu W, Wen Y, Yu Z et al (2016) Large-margin softmax loss for convolutional neural networks. In: Proceedings of machine learning research. JMLR-JOURNAL MACHINE LEARNING RESEARCH, vol 48

  21. Liu W, Wen Y, Yu Z et al (2017) Sphereface: Deep hypersphere embedding for face recognition. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 6738–6746

  22. Maze B, Adams JC, Duncan JA et al (2018) Iarpa janus benchmark - c: Face dataset and protocol. In: 2018 international conference on biometrics (ICB), pp 158–165

  23. Meng Q, Zhao S, Huang Z et al (2021) Magface: A universal representation for face recognition and quality assessment. In: 2021 IEEE/CVF conference on computer vision and pattern recognition (CVPR), vol 229. IEEE, pp 220–14

  24. Moschoglou S, Papaioannou A, Sagonas C et al (2017) Agedb: The first manually collected, in-the-wild age database. In: 2017 IEEE conference on computer vision and pattern recognition Workshops (CVPRW). IEEE, pp 1997–2005

  25. Paszke A, Gross S, Chintala S et al (2017) Automatic differentiation in pytorch. In: 31st conference on neural information processing systems (NIPS)

  26. Schroff F, Kalenichenko D, Philbin J (2015) Facenet: A unified embedding for face recognition and clustering. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 815–823

  27. Sengupta S, Chen JC, Castillo CD et al (2016) Frontal to profile face verification in the wild. In: 2016 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 1–9

  28. Singh R, Om H (2017) Newborn face recognition using deep convolutional neural network. Multimed Tools Appl 76:9,005–19,015

    Article  Google Scholar 

  29. Sun Y, Chen Y, Wang X et al (2014a) Deep learning face representation by joint identification-verification. In: 28th conference on neural information processing systems (NIPS)

  30. Sun Y, Wang X, Tang X (2014b) Deep learning face representation from predicting 10,000 classes. In: 2014 IEEE conference on computer vision and pattern recognition. IEEE, pp 1891–1898

  31. Sun Y, Wang X, Tang X (2015) Deeply learned face representations are sparse, selective, and robust. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 2892–2900

  32. Taigman Y, Yang M, Ranzato M et al (2014) Deepface: Closing the gap to human-level performance in face verification. In: 2014 IEEE conference on computer vision and pattern recognition. IEEE, pp 1701–1708

  33. Tang Y, Zhang Y, Chawla N et al (2009) Svms modeling for highly imbalanced classification. IEEE Trans Syst Man Cybern Part B (Cybernetics) 39:281–288

    Article  Google Scholar 

  34. Wang F, Xiang X, Cheng J et al (2017) Normface: L2 hypersphere embedding for face verification. In: Proceedings of the 25th ACM international conference on Multimedia. ASSOC COMPUTING MACHINERY, pp 1041–1049

  35. Wang F, Cheng J, Liu W et al (2018a) Additive margin softmax for face verification. IEEE Signal Process Lett 25:926–930

    Article  Google Scholar 

  36. Wang H, Wang Y, Zhou Z et al (2018b) Cosface: Large margin cosine loss for deep face recognition. In: 2018 IEEE/CVF conference on computer vision and pattern recognition. IEEE, pp 5265–5274

  37. Wang X, Zhang S, Wang S et al (2020) Mis-classified vector guided softmax loss for face recognition. In: 34th AAAI conference on artificial intelligence, vol 248, pp 241–12

  38. Wen Y, Zhang K, Li Z et al (2016) A discriminative feature learning approach for deep face recognition. In: 14th European conference on computer vision (ECCV), pp 499–515

  39. Whitelam C, Taborsky E, Blanton A et al (2017) Iarpa janus benchmark-b face dataset. In: 2017 IEEE conference on computer vision and pattern recognition Workshops (CVPRW). IEEE, pp 592–600

  40. Wu Y, Liu H, Li JY et al (2017) Deep face recognition with center invariant loss. In: Proceedings of the thematic workshops of ACM multimedia, vol 2017, pp 408–414

  41. Yi D, Lei Z, Liao S et al (2014) Learning face representation from scratch. arXiv:1411.7923

  42. Yin X, Yu X, Sohn K et al (2018) Feature transfer learning for deep face recognition with long-tail data. arXiv:1803.09014

  43. Zhang X, Fang Z, Wen Y et al (2017) Range loss for deep face recognition with long-tailed training data. In: 2017 IEEE international conference on computer vision (ICCV). IEEE, pp 5419–5428

  44. Zhang X, Zhao R, Qiao Y et al (2019) Adacos: Adaptively scaling cosine logits for effectively learning deep face representations. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR), vol 824. IEEE, pp 815–10

  45. Zheng T (2018) Cross-pose lfw: a database for studying cross-pose face recognition in unconstrained environments

  46. Zheng T, Deng W, Hu J (2017) Cross-age lfw: a database for studying cross-age face recognition in unconstrained environments. arXiv:1708.08197

  47. Zhong Y, Deng W, Hu J et al (2021) Sface: Sigmoid-constrained hypersphere loss for robust face recognition. IEEE Trans Image Process 30:2587–2598

    Article  Google Scholar 

Download references

Acknowledgements

The experiments in this paper are conducted on the High Performance Computing Platform of Beihang University and the Supercomputing Platform of School of Mathematical Sciences. This work is supported by the National Natural Science Foundation of China under Grant 61671002.

Author information

Authors and Affiliations

  1. School of Mathematical Sciences, Beihang University, Beijing, 102206, China

    Huwei Ren & Xiaoyuan Yang

Authors
  1. Huwei Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoyuan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoyuan Yang.

Ethics declarations

Consent for Publication

All authors consent for this study to be published in MULTIMEDIA TOOLS AND APPLICATIONS Journal.

Conflict of Interests

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ren, H., Yang, X. WeightFace: weight adaptive scaling loss for face recognition. Multimed Tools Appl 82, 36633–36646 (2023). https://doi.org/10.1007/s11042-023-15085-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15085-7

Keywords

Search

Navigation