Skip to main content
SpringerLink
Log in

Fusing magnitude and phase features with multiple face models for robust face recognition

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

High accuracy face recognition is of great importance for a wide variety of real-world applications. Although significant progress has been made in the last decades, fully automatic face recognition systems have not yet approached the goal of surpassing the human vision system, even in controlled conditions. In this paper, we propose an approach for robust face recognition by fusing two complementary features: one is Gabor magnitude of multiple scales and orientations and the other is Fourier phase encoded by spatial pyramid based local phase quantization (SPLPQ). To reduce the high dimensionality of both features, block-wise fisher discriminant analysis (BFDA) is applied and further combined by score-level fusion. Moreover, inspired by the biological cognitive mechanism, multiple face models are exploited to further boost the robustness of the proposed approach. We evaluate the proposed approach on three challenging databases, i.e., FRGC ver2.0, LFW, and CFW-p, that address two face classification scenarios, i.e., verification and identification. Experimental results consistently exhibit the complementarity of the two features and the performance boost gained by the multiple face models. The proposed approach achieved approximately 96% verification rate when FAR was 0.1% on FRGC ver2.0 Exp.4, impressively surpassing all the best known results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Zhao WY, Chellappa R, Phillips P J, Rosenfeld A. Face recognition: a literature survey. ACM Computing Surveys, 2003, 35(4): 399–458

    Article  Google Scholar 

  2. Liu C J, Wechsler H. Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition. IEEE Transactions on Image Processing, 2002, 11(4): 467–476

    Article  Google Scholar 

  3. Zhang W C, Shan S G, Gao W, Chen X L, Zhang H M. Local gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition. In: Proceedings of International Conference on Computer Vision. 2005, 786–791

    Google Scholar 

  4. Su Y, Shan S G, Chen X L, Gao W. Hierarchical ensemble of global and local classifiers for face recognition. IEEE Transactions on Image Processing, 2009, 18(8): 1885–1896

    Article  MathSciNet  MATH  Google Scholar 

  5. Xie S F, Shan S G, Chen X L, Chen J. Fusing local patterns of gabor magnitude and phase for face recognition. IEEE Transactions on Image Processing, 2010, 19(5): 1349–1361

    Article  MathSciNet  MATH  Google Scholar 

  6. Li Y, Shan S G, Zhang H H, Lao S H, Chen X L. Fusing magnitude and phase features for robust face recognition. In: Proceedings of Asian Conference on Computer Vision. 2013, 601–612

    Google Scholar 

  7. Tan X Y, Triggs B. Fusing gabor and lbp feature sets for kernel-based face recognition. In: Proceedings of International Conference on Automatic Face and Gesture Recognition. 2007, 235–249

    Google Scholar 

  8. Chan C H, Kittler J, Tahir M A. Kernel fusion of multiple histogram descriptors for robust face recognition. In: Proceedings of Joint IAPR International Workshops on Statistical Techniques in Pattern Recognition and Structural, Syntactic, and Statistical Pattern Recognition, 2010, 718–727

    Google Scholar 

  9. Cai D, He X F, Han JW. Efficient kernel discriminant analysis via spectral regression. In: Proceedings of International Conference on Data Mining. 2007, 427–432

    Google Scholar 

  10. Deng W H, Hu J N, Guo J, Cai W, Feng D G. Emulating biological strategies for uncontrolled face recognition. Pattern Recognition, 2010, 43(6): 2210–2223

    Article  MATH  Google Scholar 

  11. Deng W H, Hu J N, Lu J W, Guo J. Transform-invariant PCA: a unified approach to fully automatic face-alignment, representation, and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(6): 1275–1284

    Article  Google Scholar 

  12. Gabor D. Theory of communication. Part 1: The analysis of information. Journal of the Institution of Electrical Engineers-Part III: Radio and Communication Engineering, 1946, 93(26): 429–441

    Article  Google Scholar 

  13. Ojansivu V, Heikkilä J. Blur insensitive texture classification using local phase quantization. In: Proceedings of International Conference on Image and Signal Processing, 2008, 236–243

    Google Scholar 

  14. Ojala T, Pietikäinen M, Harwood D. A comparative study of texture measures with classification based on featured distributions. Pattern Recognition, 1996, 29(1): 51–59

    Article  Google Scholar 

  15. Lowe D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2): 91–110

    Article  MathSciNet  Google Scholar 

  16. Bicego M, Lagorio A, Grosso E, Tistarelli M. On the use of sift features for face authentication. In: Proceedings of Computer Vision and Pattern Recognition Workshop. 2006

    Google Scholar 

  17. Luo J, Ma Y, Takikawa E, Lao S, Kawade M, Lu B L. Person-specific sift features for face recognition. In: Proceedings of International Conference on Acoustics, Speech and Signal Processing. 2007

    Google Scholar 

  18. Mian A S, Bennamoun M, Owens R. An efficient multimodal 2d-3d hybrid approach to automatic face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(11): 1927–1943

    Article  Google Scholar 

  19. Dalal N, Triggs B. Histograms of oriented gradients for human detection. In: Proceedings of Computer Vision and Pattern Recognition. 2005, 886–893

    Google Scholar 

  20. Cao Z M, Yin Q, Tang X O, Sun J. Face recognition with learning-based descriptor. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2010, 2707–2714

    Google Scholar 

  21. Albiol A, Monzo D, Martin A, Sastre J, Albiol A. Face recognition using HOG-EBGM. Pattern Recognition Letters, 2008, 29(10): 1537–1543

    Article  Google Scholar 

  22. Liu ZM, Liu C J. Robust face recognition using color information. Proceedings of International Conference on Biometrics, 2009, 122–131

    Google Scholar 

  23. Shan S G, Zhang W C, Su Y, Chen X L, Gao W. Ensemble of piecewise FDA based on spatial histograms of local (Gabor) binary patterns for face recognition. In: Proceedings of International Conference on Pattern Recognition. 2006, 606–609

    Google Scholar 

  24. Sinha P, Poggio T. I think I know that face. Nature, 1996, 384(6608): 404

    Article  Google Scholar 

  25. Davies G. Perceiving and Remembering Faces. London: Academic Press, 1981

    Google Scholar 

  26. Ellis H D. Aspects of Face Processing. Boston: Martinus Nijhoff Publishers, 1986

    Book  Google Scholar 

  27. Phillips P J, Flynn P J, Scruggs T, Bowyer K W, Chang J, Hoffman K, Marques J, Min J, Worek W. Overview of the face recognition grand challenge. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2005, 947–954

    Google Scholar 

  28. Huang G B, Mattar M, Berg T, Learned-Miller E. Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical Report 07–49, 2007

    Google Scholar 

  29. Zhang X, Zhang L, Wang X J, Shum H Y. Finding celebrities in billions of web images. IEEE Transactions on Multimedia. 2012, 14(4): 995–1007

    Article  Google Scholar 

  30. Lades M, Vorbruggen J C, Buhmann J, Lange J, Malsburg V D C, Wurtz R P, Konen W. Distortion invariant object recognition in the dynamic link architecture. IEEE Transactions on Computers, 1993, 42(3): 300–311

    Article  Google Scholar 

  31. Zhang B C, Shan S G, Chen X L, Gao W. Histogram of Gabor phase patterns (HGPP): a novel object representation approach for face recognition. IEEE Transactions on Image Processing, 2007, 16(1): 57–68

    Article  MathSciNet  Google Scholar 

  32. Lazebnik S, Schmid C, Ponce J. Beyond bags of features: spatial pyramid-matching for recognizing natural scene categories. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2006, 2169–2178

    Google Scholar 

  33. Grauman K, Darrell T. The pyramid match kernel: discriminative classification with sets of image features. In: Proceedings of International Conference on Computer Vision. 2005, 1458–1465

    Google Scholar 

  34. Hadjidemetriou E, Grossberg M D, Nayar S K. Multiresolution histograms and their use for recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(7): 831–847

    Article  Google Scholar 

  35. Fisher R A. The use of multiple measurements in taxonomic problems. Annals of Human Genetics, 1936, 7(2): 179–188

    Google Scholar 

  36. Kumar N, Berg A C, Belhumeur P N, Nayar S K. Attribute and simile classifiers for face verification. In: Proceedings of International Conference on Computer Vision. 2009, 365–372

    Google Scholar 

  37. Tan X Y, Triggs B. Enhanced local texture feature sets for face recognition under difficult lighting conditions. In: Proceedings of International Conference on Automatic Face and Gesture Recognition. 2007, 168–182

    Google Scholar 

  38. Yang J, Liu C J. Color image discriminant models and algorithms for face recognition. IEEE Transactions on Neural Networks, 2008, 19(12): 2088–2098

    Article  Google Scholar 

  39. Yang J, Liu C J. Horizontal and vertical 2DPCA-based discriminant analysis for face verification on a large-scale database. IEEE Transactions on Information Forensics and Security, 2007, 2(4): 781–792

    Article  MathSciNet  Google Scholar 

  40. Shih P, Liu C J. Improving the face recognition grand challenge baseline performance using color configurations across color spaces. In: Proceedings of International Conference on Image Processing. 2006, 1001–1004

    Google Scholar 

  41. Taigman Y, Wolf L, Hassner T. Multiple one-shots for utilizing class label information. In: Proceedings of British Machine Vision Conference. 2009, 1–12

    Google Scholar 

  42. Yang Y, Song J K, Huang Z, Ma Z G, Sebe N, Hauptmann A G. Multi-feature fusion via hierarchical regression for multimedia analysis. IEEE Transaction on Multimedia, 2013, 15(3): 572–581

    Article  Google Scholar 

  43. Ma Z G, Yang Y, Sebe N, Hauptmann A G. Multiple features but few labels? a symbiotic solution exemplified for video analysis. In: Proceedings of the 22nd ACM International Conference on Multimedia. 2014, 77–86

    Google Scholar 

  44. Kumar B, Savvides M, Xie C Y. Correlation pattern recognition for face recognition. Proceedings of the IEEE, 2006, 94(11): 1963–1976

    Article  Google Scholar 

  45. Liu C J. The bayes decision rule induced similarity measures. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(6): 1086–1090

    Article  MathSciNet  Google Scholar 

  46. Hwang W, Park G, Lee J, Kee S C. Multiple face model of hybrid fourier feature for large face image set. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2006, 1574–1581

    Google Scholar 

  47. Liu C J. Capitalize on dimensionality increasing techniques for improving face recognition grand challenge performance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(5): 725–737

    Article  Google Scholar 

  48. Parkhi O M, Vedaldi A, Zisserman A. Deep face recognition. In: Proceedings of British Machine Vision Conference. 2015, 1–12

    Google Scholar 

  49. Li P, Fu Y, Mohammed U, Elder J H, Prince S J. Probabilistic models for inference about identity. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(1): 144–157

    Article  Google Scholar 

  50. Berg T, Belhumeur P N. Tom-vs-Pete classifiers and identity-preserving alignment for face verification. In: Proceedings of BritishMachine Vision Conference. 2012

    Google Scholar 

  51. Chen D, Cao X D, Wen F, Sun J. Blessing of dimensionality: high-dimensional feature and its efficient compression for face verification. In: Proceedings of Computer Vision and Pattern Recognition. 2013, 3025–3032

    Google Scholar 

  52. Taigman Y, Yang M, Ranzato M, Wolf L. Deepface: closing the gap to human-level performance in face verification. In: Proceedings of Computer Vision and Pattern Recognition. 2014, 1701–1708

    Google Scholar 

  53. Sun Y, Wang X G, Tang X O. Deep learning face representation from predicting 10,000 classes. In: Proceedings of Computer Vision and Pattern Recognition. 2014, 1891–1898

    Google Scholar 

  54. Schroff F, Kalenichenko D, Philbin J. Facenet: a unified embedding for face recognition and clustering. In: Proceedings of Computer Vision and Pattern Recognition. 2015, 815–823

    Google Scholar 

  55. Yi D, Lei Z, Liao S C, Li S Z. Learning face representation from scratch. 2014, arXiv preprint arXiv:1411.7923v1

    Google Scholar 

Download references

Acknowledgements

This work is partially supported by the National Basic Research Program of China (2015CB351802), the National Natural Science Foundation of China (Grant Nos. 61390511, 61222211, 61379083, 61271445), the Strategic Priority Research Program of the CAS (XDB02070004), and the Youth Innovation Promotion Association CAS (2015085).

Author information

Authors and Affiliations

  1. Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology (ICT), CAS, Beijing, 100190, China

    Yan Li, Shiguang Shan, Ruiping Wang & Xilin Chen

  2. University of Chinese Academy of Sciences, Beijing, 100049, China

    Yan Li, Shiguang Shan, Ruiping Wang & Xilin Chen

  3. School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, 210094, China

    Zhen Cui

Authors
  1. Yan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shiguang Shan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ruiping Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhen Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xilin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shiguang Shan.

Additional information

Yan Li received the BS degree in computer science and technology from Nankai University, China in 2010. He is currently pursuing the PhD degree with the Institute of Computing Technology (ICT), Chinese Academy of Sciences (CAS), China. He also spent nine months working as a Research Scholar with Lane Department of Computer Science and Electrical Engineering in the Benjamin M.Statler College of Engineering, and Mineral Resources at West Virginia University, USA. His research interests include computer vision, pattern recognition, image processing, and in particular, image and video face recognition, face retrieval, and binary code learning.

Shiguang Shan receivedMS degree in computer science from the Harbin Institute of Technology, China in 1999, and PhD degree in computer science from the Institute of Computing Technology (ICT), Chinese Academy of Sciences (CAS), China, in 2004. He joined ICT, CAS in 2002 and has been a professor since 2010. His research interests cover computer vision, pattern recognition, and machine learning. He especially focuses on face recognition related research topics. He has published more than 200 papers in refereed journals and proceedings in the areas of computer vision and pattern recognition. He is a recipient of the China’s State Natural Science Award in 2015, and the China’s State S&T Progress Award in 2005 for his research.

Ruiping Wang received the BS degree in applied mathematics from Beijing Jiaotong University, China in 2003, and the PhD degree in computer science from the Institute of Computing Technology (ICT), Chinese Academy of Sciences (CAS), China in 2010. He was a postdoctoral researcher with the Department of Automation, Tsinghua University, China from 2010 to 2012. He also spent one year working as a research associate with the Institute for Advanced Computer Studies, at the University of Maryland, College Park, from Nov. 2010 to Oct. 2011. He has been with the faculty of the ICT, CAS since July 2012, where he is currently an associate professor. His research interests include computer vision, pattern recognition, and machine learning.

Zhen Cui received the BS, MS, and PhD degrees from Shandong Normal University, Sun Yat-sen University, and Institute of Computing Technology (ICT), Chinese Academy of Sciences, China in 2004, 2006, and 2014, respectively. He was a research fellow in the Department of Electrical and Computer Engineering at National University of Singapore (NUS), Singapore from 2014 to 2015. He also spent half a year as a Research Assistant on Nanyang Technological University (NTU) from Jun. 2012 to Dec. 2012. Currently, he is an associate professor of Southeast University, China. His research interests cover computer vision, pattern recognition and machine learning, especially focusing on deep learning, manifold learning, sparse coding, face detection/alignment/recognition, object tracking, image super resolution, emotion analysis, etc.

Xilin Chen received the BS, MS, and PhD degrees in computer science from the Harbin Institute of Technology, Harbin, China in 1988, 1991, and 1994, respectively. He was a professor with the Harbin Institute of Technology from 1999 to 2005. He has been a professor with the Institute of Computing Technology, Chinese Academy of Sciences (CAS), China. Since August 2004. He has published one book and over 200 papers in refereed journals and proceedings in the areas of computer vision, pattern recognition, image processing, and multimodal interfaces. He served as an Organizing Committee / Program Committee Member for more than 50 conferences. He is a fellow of IEEE, and a fellow of China Computer Federation (CCF).

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Shan, S., Wang, R. et al. Fusing magnitude and phase features with multiple face models for robust face recognition. Front. Comput. Sci. 12, 1173–1191 (2018). https://doi.org/10.1007/s11704-017-6275-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-017-6275-6

Keywords

Search

Navigation