Skip to main content

Advertisement

SpringerLink
Log in

Fusion facial semantic feature and incremental learning mechanism for efficient face recognition

  • Data analytics and machine learning
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Efficient face recognition can realize fast and accurate face recognition and make it widely used in essential fields such as human–computer interaction and access control. At present, there are many face recognition methods whose recognition rate can reach high accuracy, but the training of the model and the recognition of samples take much time, which leads to insufficient real-time performance. This paper designs a fusion facial semantic feature (FFSF) and an incremental learning mechanism (ILM) for efficient face recognition. FFSF feature is a fusion of facial contour features and facial semantic component features, which can extract contour features and interior features of facial organs (eyes, mouth, nose, and eyebrow) according to facial organs’ position. FFSF features can ensure that the extracted features are concentrated in the face’s most discriminative region, making the extracted features have good discriminative characteristics. Then, we use a clustering algorithm to construct a hierarchical incremental learning tree (HIL-Tree) with a hierarchical structure and use the HIL-Tree to implement the ILM. ILM achieves fast and accurate sample classification by retrieving the nodes in HIL-Tree, and the training samples can be directly added to the HIL-Tree by retrieval instead of rebuilding the HIL-Tree during the training process. Extensive experiments on several public data sets demonstrate the proposed efficient face recognition method’s excellent accuracy and efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Availability of data and material

The data sets supporting the results of this paper are included within the article and its additional files.

Code availability

The software and code that support the findings of this study are available from the corresponding author upon reasonable request.

References

  • Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041

    Article  MATH  Google Scholar 

  • Bahroun S, Abed R, Zagrouba E (2021) KS-FQA: keyframe selection based on face quality assessment for efficient face recognition in video. IET Image Proc 15:77–90

    Article  Google Scholar 

  • Bashbaghi S, Granger E, Sabourin R, Bilodeau GA (2017) Dynamic ensembles of exemplar-SVMs for still-to-video face recognition. Pattern Recogn 69:61–81

    Article  Google Scholar 

  • Battaglia F, Iannizzotto G, Bello LL (2017) A person authentication system based on RFID tags and a cascade of face recognition algorithms. IEEE Trans Circuits Syst Video Technol 27(8):1676–1690

    Article  Google Scholar 

  • Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Article  Google Scholar 

  • Chen D, Cao X, Wen F, Sun J (2013) Blessing of dimensionality: high-dimensional feature and its efficient compression for face verification. In: Proceeding of the IEEE conference on computer vision and pattern recognition, Portland, pp 3025–3032

  • Cho M, Jeong Y (2017) Face recognition performance comparison between fake faces and live faces. Soft Comput 21:3429–3437

    Article  Google Scholar 

  • Cho H, Roberts R, Jung B, Choi O, Moo S (2014) An efficient hybrid face recognition algorithm using PCA and GABOR wavelets. Int J Adv Robot Syst 11(1):1–8

    Google Scholar 

  • Choi JY, Lee B (2020) Ensemble of deep convolutional neural networks with gabor face representations for face recognition. IEEE Trans Image Process 29:3270–3281

    Article  Google Scholar 

  • Dhekane M, Seal A, Khanna P (2017) Illumination and expression invariant face recognition. Int J Pattern Recogn Artif Intell 31(12):1–15

    Article  Google Scholar 

  • Du GY, Tian SL, Qiu YY, Xu CY (2016) Effective and efficient Grassfinch kernel for SVM classification and its application to recognition based on image set. Chaos Solitons Fract 89:295–303

    Article  MATH  Google Scholar 

  • Duan Y, Lu J, Feng J, Zhou J (2018) Context-aware local binary feature learning for face recognition. IEEE Trans Pattern Anal Mach Intell 40(5):1139–1153

    Article  Google Scholar 

  • Feng X, Pietikainen M, Hadid A (2007) Facial expression recognition based on local binary patterns. Pattern Recognit Image Anal 17(4):592–598

    Article  Google Scholar 

  • Georghiadesa AS, Belhumeur PN, Kriegman DJ (2001) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660

    Article  Google Scholar 

  • Guo C, Liang J, Zhan G, Liu Z, Pietikäinen M, Liu L (2019) Extended local binary patterns for efficient and robust spontaneous facial micro-expression recognition. IEEE Access 7:174517–174530

    Article  Google Scholar 

  • He XJ, Dai BQ (2016) A new traffic signs classification approach based on local and global features extraction. In: Proceeding of the international conference on information communication and management, Hatfield, pp 121–125

  • Heikkil M, Pietik M, Schmid C (2009) Description of interest regions with local binary patterns. Pattern Recognit 42(3):425–436

    Article  Google Scholar 

  • Hou J, Gao H, Xia Q, Qi N (2015) Feature combination and the kNN framework in object classification. IEEE Trans Neural Netw Learn Syst 27(6):1368–1378

    Article  Google Scholar 

  • Huang GB, Ramesh M, Berg T, Learned-Miller E (2007) Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Int J Comput vis 96(3):277–279

    Google Scholar 

  • Huang P, Li T, Gao GW, Yang G (2019) Feature extraction based on graph discriminant embedding and its applications to face recognition. Soft Comput 23(16):7015–7028

    Article  Google Scholar 

  • Jia H, Martinez AM (2009) Support vector machines in face recognition with occlusions. In: Proceeding of the IEEE conference on computer vision and pattern recognition, Miami, pp 136–141

  • Karczmarek P, Pedrycz W, Kiersztyn A, Rutka P (2017) A study in facial features saliency in face recognition: an analytic hierarchy process approach. Soft Comput 21(24):7503–7517

    Article  Google Scholar 

  • Lei Z, Pietikainen M, Li SZ (2013) Learning discriminant face descriptor. IEEE Trans Pattern Anal Mach Intell 36(2):289–302

    Google Scholar 

  • Liang J, Wang M, Chai Z, Wu Q (2014) Different lighting processing and feature extraction methods for efficient face recognition. IET Image Proc 8(9):528–538

    Article  Google Scholar 

  • Liang J, Tu H, Liu F, Zhao Q, Jain A (2020) 3D face reconstruction from mugshots: Application to arbitrary view face recognition. Neurocomputing 410(14):12–27

    Article  Google Scholar 

  • Liao S, Law MWK, Chung ACS (2009) Dominant local binary patterns for texture classification. IEEE Trans Image Process 18(5):1107–1118

    Article  MathSciNet  MATH  Google Scholar 

  • Liu QF, Liu CJ (2017) A novel locally linear KNN method with applications to visual recognition. IEEE Trans Neural Netw Learn Syst 28(9):2010–2021

    Article  MathSciNet  Google Scholar 

  • Liu L, Zhao L, Long Y, Kuang G, Fieguth P (2012) Extended local binary patterns for texture classification. Image Vis Comput 30(2):86–99

    Article  Google Scholar 

  • Liu R, Feng WG, Zhu M (2013) Expression and lighting invariant face recognition using fast tree-based matching. Electron Lett 49(22):1379–1381

    Article  Google Scholar 

  • Liu F, Zhao Q, Liu X, Zeng D (2020) Joint face alignment and 3D face reconstruction with application to face recognition. IEEE Trans Pattern Anal Mach Intell 42(3):664–678

    Article  Google Scholar 

  • Lu J, Liong VE, Zhou X, Zhou J (2015) Learning compact binary face descriptor for face recognition. IEEE Trans Pattern Anal Mach Intell 37(10):2041–2056

    Article  Google Scholar 

  • Lu J, Liong VE, Zhou J (2018) Simultaneous local binary feature learning and encoding for homogeneous and heterogeneous face recognition. IEEE Trans Pattern Anal Mach Intell 40(8):1979–1993

    Article  Google Scholar 

  • Macqueen J (1967) Some methods for classification and analysis of multivariate observations. In: Proceedings of the 5th Berkeley symposium on mathematical statistics and probability. University of California, pp 281–297.

  • Mahmood Z, Ali T, Khan SU (2016) Effects of pose and image resolution on automatic face recognition. IET Biom 5(2):111–119

    Article  Google Scholar 

  • Martinez AM, Benavente R (1998) The AR face database. CVC technical report 24

  • Roh SB, Oh SK, Yoon JH, Seo K (2019) Design of face recognition system based on fuzzy transform and radial basis function neural networks. Soft Comput 23(13):4969–4985

    Article  Google Scholar 

  • Sim T, Baker S, Bsat M (2003) The CMU pose, illumination, and expression database. IEEE Trans Pattern Anal Mach Intell 25(12):1615–1618

    Article  Google Scholar 

  • Sun Y, Wang X, Tang X (2014) Deep learning face representation from predicting 10,000 classes. In: Proceeding of the IEEE conference on computer vision and pattern recognition,Columbus, pp 1891–1898

  • Tan XY, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process 19(6):1635–1650

    Article  MathSciNet  MATH  Google Scholar 

  • Tan H, Yang B, Ma M (2014) Face recognition based on the fusion of global and local HOG features of face images. IET Comput Vis 8(3):224–234

    Article  Google Scholar 

  • Tang H, Yin B, Sun Y, Hu Y (2013) 3D face recognition using local binary patterns. Signal Process 93(8):2190–2198

    Article  Google Scholar 

  • Vu NS, Caplier A (2012) Enhanced patterns of oriented edge magnitudes for face recognition and image matching. IEEE Trans Image Process 21(3):1352–1365

    Article  MathSciNet  MATH  Google Scholar 

  • Weng JY, Hwang WS (2007) Incremental hierarchical discriminant regression. IEEE Trans Neural Netw 18(2):397–415

    Article  Google Scholar 

  • Xiong X, Torre FD (2013) Supervised descent method and its application to face alignment. In: Proceeding of the IEEE conference on computer vision and pattern recognition, Portland, pp 532–539

  • Xu J, Xie S, Zhu W (2017) Marginal patch alignment for dimensionality reduction. Soft Comput 21:2347–2356

    Article  Google Scholar 

  • Yang F, Mao KZ, Lee GK, Tang W (2015) Emphasizing minority class in LDA for feature subset selection on high-dimensional small-sized problems. IEEE Trans Knowl Data Eng 27(1):88–101

    Article  Google Scholar 

  • Yang WK, Wang ZY, Zhang BC (2016) Face recognition using adaptive local ternary patterns method. Neurocomputing 213:183–190

    Article  Google Scholar 

  • Zhao J, Han J, Shao L (2018) Unconstrained face recognition using a set-to-set distance measure on deep learned features. IEEE Trans Circuits Syst Video Technol 28(10):2679–2689

    Article  Google Scholar 

  • Zhao J, Xiong L, Li J, Xing J, Yan S, Feng J (2019) 3D-aided dual-agent GANs for unconstrained face recognition. IEEE Trans Pattern Anal Mach Intell 41(10):2380–2394

    Article  Google Scholar 

  • Zheng J, Ranjan R, Chen C, Chen J, Castillo CD, Chellappa R (2020) An automatic system for unconstrained video-based face recognition. IEEE Trans Biom Behav Identity Sci 2(3):194–209

    Article  Google Scholar 

  • Zhong Y, Deng W, Hu J, Zhao D, Li X, Wen D (2020) SFace: sigmoid-constrained hypersphere loss for robust face recognition. IEEE Trans Image Process 30:2587-2598

    Article  Google Scholar 

  • Zhu WJ, Yan YH, Peng YS (2017) Pair of projections based on sparse consistence with applications to efficient face recognition. Signal Process Image Commun 55:32–40

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (Grant Nos. 62073250, 62003249, and 82060328), Key Research and Development Program of China (Grant No. 2017YFC0806503-05), Science and Technology Research Project of Jiangxi Provincial Department of Education (Grant Nos. 180771 and 190742), Open Project of Key Laboratory of Jiangxi Province Numerical Simulation and Emulation Techniques, Educational Science Planning Project of Jiangxi Province (Grant No. 18ZD057), and Key Research and Development Program of Hubei Province (Grant No. 2020BAB021).

Author information

Authors and Affiliations

  1. School of Mathematics and Computer Science, Gannan Normal University, Ganzhou, 341000, China

    Rui Zhong & Qi Zhong

  2. Key Laboratory of Jiangxi Province for Numerical Simulation and Emulation Techniques, Gannan Normal University, Ganzhou, 341000, China

    Rui Zhong

  3. School of Information Science and Engineering, Wuhan University of Science and Technology, Wuhan, 430081, China

    Huaiyu Wu & Zhihuan Chen

  4. Institue of Robotics and Intelligent Systems, Wuhan Univerisity of Science and Technology, Wuhan, 430081, China

    Huaiyu Wu & Zhihuan Chen

Authors
  1. Rui Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huaiyu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhihuan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qi Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

ZR and WH conceived and designed the study. ZR and ZQ performed the experiments. ZR and CZ wrote the paper. CZ reviewed and edited the manuscript.

Corresponding author

Correspondence to Huaiyu Wu.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interests.

Ethics approval

This article does not contain any studies with human or animals performed by any of the authors.

Consent to participate

All authors have been personally and actively involved in substantive work leading to the manuscript, and will hold themselves jointly and individually responsible for its content.

Consent for publication

All authors consent for this study to be published in Soft Computing Journal.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (WMV 60878 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhong, R., Wu, H., Chen, Z. et al. Fusion facial semantic feature and incremental learning mechanism for efficient face recognition. Soft Comput 25, 9347–9363 (2021). https://doi.org/10.1007/s00500-021-05915-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-021-05915-x

Keywords

Search

Navigation