Skip to main content
Log in

A new approach for small sample face recognition with pose variation by fusing Gabor encoding features and deep features

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Small sample and pose variation are two critical technical problems in automatic face recognition (AFR). The combination of these two difficulties will seriously affect the performance of face recognition and restrict the wide application of AFR. To address this problem, we propose a new multi-feature fusion framework. First, we propose a pixel-level data augmentation algorithm based on manifold subspace partition, which constructs virtual samples in the original face image space to achieve training sample expansion and diversity enhancement. Then, we propose a feature-level data augmentation based on Gabor transformation, which can capture multi-level face features through multi-scale and multi-direction Gabor filters to realize the face expansion in feature space. To eliminate the data redundancy and interference information generated by Gabor feature augmentation, a Gabor feature encoding algorithm is proposed to construct the compressed Gabor feature vector. In addition, we propose a small scale adaptive deep CNN model, which is suitable for small sample datasets and can effectively extract nonlinear deep features of pose-varied faces. Finally, Gabor encoding features and nonlinear deep features are combined for small sample face recognition with pose variation. Experiment results based on two face datasets prove the effectiveness of the proposed multi-feature fusion framework.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell 28(12):2037–2041

    MATH  Google Scholar 

  2. Bao-cai Y, Zhuang Z, Yan-feng S, Cheng-zhang W (2007) Pose variant face recognition based on 3D Morphable model. J Beijing Univ Techn 33(3):320–325

    MATH  Google Scholar 

  3. Bengio Y, Lamblin P, Popovici D et al (2007) Greedy layer-wise training of deep networks. Adv Neural Inf Proces Syst:153–160

  4. Beom-Seok O, Toh K-A, Teoh ABJ et al (2018) An analytic Gabor feed forward network for single-sample and pose-invariant face recognition. IEEE Trans Image Process 27(6):2791–2805

    MathSciNet  MATH  Google Scholar 

  5. Cao B, Wang N, Li J et al (2018) Data augmentation-based joint learning for heterogeneous face recognition. IEEE Trans Neural Networks Learning Syst:1–13

  6. Ye Changiming, Jiang Jianguo, Zhan Shu, et al. 3D Facial Depth Map Recognition in Different Poses with Surface Contour Feature. PR & AI, 2013,26(2):219–224

  7. Cheng D, GuangDa S, Gang LX et al (2004) Synthesis of Face Image with Pose Variations. Journal of Optoelectronics·Laser 15(12):1498–1451

    Google Scholar 

  8. Choe J, Park S, Kim K et al (2017) Face Generation for Low-Shot Learning Using Generative Adversarial Networks. IEEE Int Conf Computer Vision (ICCV):1940–1948

  9. Daugman J (1985) Uncertainty relation for resolution in space, spatial, frequency, and orientation optimized by two-dimensional visual cortical filters. J Opt Soc Am 2(7):1160–1169

    Google Scholar 

  10. De Marsico M, Nappi M, Riccio D et al (2013) Robust face recognition for uncontrolled pose and illumination changes. IEEE Trans Syst Man, Cybernetics: Syst 43(1):149–163

    Google Scholar 

  11. Fan Deng-Ping, Zhang Shengchuan, Wu Yu-Huan et al. Scoot: A Perceptual Metric for Facial Sketches. 2019 IEEE Int Conference on Computer Vision(ICCV2019),2019:1–11.

  12. Ding C, Tao D (2015) Robust face recognition via multimodal deep face representation. IEEE Trans Multimedia 17(11):2049–2058

    Google Scholar 

  13. GuoFeng Z, GuiXia F, HaiTao L et al (2015) A survey of multi-pose face recognition. PR&AI 28(7):613–625

    Google Scholar 

  14. Guo-feng ZOU, Gui-xia FU, Ke-jun WANG, Ming-liang GAO, Jin SHEN (2017) Construction method of adaptive deep convolutional neural network model. J Beijing Univ Posts Telecomm 40(4):98–103

    Google Scholar 

  15. Huajie C, Wei W (2007) Multi pose face recognition based on correlative sub region mapping. Journal of Image and Graphics 12(7):1254–1260

    Google Scholar 

  16. KaiQi H, XiaoTang C, YunFeng K et al (2015) Intelligent Visual Surveillance: A Review. Chinese Journal Of Computer 38(6):1093–1115

    MathSciNet  Google Scholar 

  17. Ke-Jun W, Guo-Feng Z (2013) A Sub-Pattern Gabor Features Fusion Method for Single Sample Face Recognition. Pattern Recog Artificial Intelligent 26(1):50–56

    Google Scholar 

  18. LeCun Y, Bottou L, Bengio Y et al (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Google Scholar 

  19. Leng B, Yu K, Qin J (2017) Data augmentation for unbalanced face recognition training sets. Neurocomputing 235:10–14

    Google Scholar 

  20. Li L, Ge H, Tong Y, Zhang Y (2018) Face recognition using Gabor-based feature extraction and feature space transformation fusion method for single image per person problem. Neural Process Letter 47:1197–1217

    Google Scholar 

  21. Lian SF, Liu YH, Li LC (2014) Face recognition under unconstrained based on LBP and deep learning. J Commun 35(6):154–160

    Google Scholar 

  22. W Q Liang, G G Wang, J H Lai, et al. M2M-GAN: Many-to-Many Generative Adversarial Transfer Learning for Person Re-Identification.2018. https://arxiv.org/abs/1811.03768.

  23. Z. X. Lin, W. J. Yang, C. C. Ho et al. Fast Vertical-Pose- Invariant Face Recognition Module for Intelligent Robot Guard. The 4th International Conference on Autonomous Robots and Agents, 2009:613–617.

  24. Ziwei Liu,Ping Luo,Xiaogang Wang, et al. Deep Learning Face Attributes in the Wild, 2015 IEEE International Conference on Computer Vision: 3730–3738

  25. J. Liu, Y. Deng, C. Huang, Targeting Ultimate Accuracy: Face Recognition via Deep Embedding, arXiv preprint arxiv:hepth/1506.07310.

  26. Lowe DG (1999) Object recognition from local scale-invariant features, in: proceedings of the IEEE international conference on computer vision. Vol. 2:1150–1157

    Google Scholar 

  27. Luan S, Chen C, Zhang B et al (2018) Gabor convolutional networks. IEEE Trans Image Process 27(9):4357–4366

    MathSciNet  Google Scholar 

  28. Lv J-J, Shao X-H, Huang J-S et al (2017) Data augmentation for face recognition. Neurocomputing 230:184–196

    Google Scholar 

  29. J M Lv, W H Chen, Q Li, et al. Unsupervised Cross-dataset Person Re-identification by Transfer Learning of Spatial-Temporal Patterns. In 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018: 7948–7956.

  30. Iacopo Masi, Stephen Rawls, Gérard Medioni, Prem Natarajan. Pose-Aware Face Recognition in the Wild, 2016 .IEEE Int Conf Comp VisionPattern Recog:4838–4846

  31. Masi I, Chang F-J, Choi J et al (2019) Learning pose-aware models for pose-invariant face recognition in the wild. EEE Trans Pattern Anal Mach Intel 41(2):379–393

    Google Scholar 

  32. Moon H-M, Seo CH, Pan SB (2017) A face recognition system based on convolution neural network using multiple distance face. Soft Comput 21(17):4995–5002

    Google Scholar 

  33. Ng A. Sparse autoencoder . CS294A Lecture notes, 2011, 72(2011): 1–19.

  34. Guang Yu Nie, Ming Ming Cheng, Yun Liu, et al. Multi-Level Context Ultra-Aggregation for Stereo Matching. 2019 IEEE International Conference on Computer Vision and Pattern Recognition (CVPR 2019)2019:3283–3291.

  35. Nikolay P (1995) Biologically motivated computationally intensive approaches to image pattern recognition. Futur Gener Comput Syst 11(4/5):451–465

    Google Scholar 

  36. Nikolay P (1995) Biologically motivated computationally intensive approaches to image pattern recognition. Futur Gener Comput Syst 11(4/5):451–465

    Google Scholar 

  37. Yongri Piao, Wei Ji, Jingjing Li, et al. Depth-induced Multi-scale Recurrent Attention Network for Saliency Detection. 2019 IEEE International Conference on Computer Vision(ICCV2019),2019:7254–7263.

  38. Hongwei Qin, Junjie Yan, Xiu Li, et al. Joint Training of Cascaded CNN for Face Detection. 2016 IEEE International Conference on Computer Vision and Pattern Recognition:3456–3465.

  39. Sharma A, Al Haj M, Choi J et al (2012) Robust pose invariant face recognition using coupled latent space discriminant analysis. Comput Vis Image Underst 116:1095–1110

    Google Scholar 

  40. Y Sun, X Wang, X Tang. Hybrid deep learning for face verification. Proc. IEEE Int Conference Comp Vision (ICCV 2013), 2013:1489–1496.

  41. Y. Taigman, M. Yang, M. Ranzato, L. Wolf, Deepface: closing the gap to humanlevel performance in face verification, in: proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 1701–1708.

  42. Wang KJ, GuoFeng Z, GuiXia F et al (2013) An approach to fast eye location and face plane rotation correction. Journal of Computer-Aided Design &Computer Graphics 25(6):865–872

    Google Scholar 

  43. Xi M, Liang C, Polajnar D, Tong W (2016) Local binary pattern network: A deep learning approach for face recognition. 2016 IEEE International Conference on Image Processing (ICIP):3224–3228

  44. Xiao C, Ze L, Wang YL et al (2016) Face Recogniton with pose variation based on fusion of multi-scale MRF and SCT. Application Research of Computers 33(8):2519–2523

    Google Scholar 

  45. Xiao-hong Z Z-h H (2013) Linear locality preserving and discriminating projection for face recognition. J Elec Info Techn 35(2):463–467

    Google Scholar 

  46. Xiu-Juan C, Shi-Guang S, Xi-Lin C et al (2007) Local linear regression for pose invariant face recognition. IEEE Trans on Image Process 16(7):1716–1725

    MathSciNet  Google Scholar 

  47. Yin X, Liu X (2018) Multi-task convolutional neural network for pose-invariant face recognition. IEEE Trans Image Process 27(2):964–975

    MathSciNet  MATH  Google Scholar 

  48. Xi Yin, Xiang Yu, Kihyuk Sohn, et al. Feature Transfer Learning for Deep Face Recognition with Long-Tail Data. In Proceeding of IEEE Computer Vision and Pattern Recognition (CVPR 2019), Long Beach, CA, Jun. 2019. arXiv:1803.09014.

  49. Yong C, Ting ting H, Hua lin L et al (2016) Multi-pose face ensemble classification aided by Gabor features and deep belief nets. Optik 127(2):946–954

    Google Scholar 

  50. Zhang ZC, Hong WC (2019) Electric load forecasting by complete ensemble empirical model decomposition adaptive noise and support vector regression with quantum-based dragonfly algorithm. Nonlinear Dynamics 98:1107–1136

    Google Scholar 

  51. Zhang Y-N, Guo Z, Xia Y et al (2012) 2D representation of facial surfaces for multi-pose 3D face recognition. Pattern Recogn Lett 33:530–536

    Google Scholar 

  52. Zhang H, Nasrabadi NM, Zhang Y et al (2012) Joint dynamic sparse representation for multi-view face recognition. Pattern Recogn 45:1290–1298

    Google Scholar 

  53. Zhang Y, Zhao D, Sun J, Zou G, Li W (2016) Adaptive convolutional neural network and its application in face recognition. Neural Process Lett 43(2):389–399

    Google Scholar 

  54. Zhao J, Yu C, Xu Y et al (2018) Towards Pose Invariant Face Recognition in the Wild. IEEE Conf Comp Vision Pattern Recogn (CVPR):2207–2216

  55. Zou G-f, Gui-xia F, Gao M-l et al (2019) A novel construction method of convolutional neural network model based on data-driven. Multimed Tools Appl 78(6):6969–6987

    Google Scholar 

Download references

Funding

This research was funded by the Visiting Project Funds of Shandong University of Technology, the Integration Funds of Shandong University of Technology and Zhangdian District (No.118228), the National Natural Science Foundation of China (No. 61601266, No.61801272), the Natural Science Foundation of Shandong Province of China (No. ZR2015FL029, ZR2016FL14) .

Author information

Authors and Affiliations

Authors

Contributions

Guofeng Zou proposed the original idea and wrote this paper; Guixia Fu analyzed the data and correct the translation; Mingliang Gao and Jinfeng Pan reviewed this paper; Zheng Liu participated in the rationality of the demonstration experiment.

Corresponding author

Correspondence to Guofeng Zou.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zou, G., Fu, G., Gao, M. et al. A new approach for small sample face recognition with pose variation by fusing Gabor encoding features and deep features. Multimed Tools Appl 79, 23571–23598 (2020). https://doi.org/10.1007/s11042-020-09076-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09076-1

Keywords

Navigation