Skip to main content
SpringerLink
Log in

Two-dimensional joint local and nonlocal discriminant analysis-based 2D image feature extraction for deep learning

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

This paper proposes a new two-dimensional manifold learning algorithm called two-dimensional joint local/nonlocal discriminant analysis (2DJLNDA) for 2D image feature extraction, which directly extracts projective vectors from 2D image matrices rather than image vectors. Different from other typical 2D methods, e.g., two-dimensional principal component analysis (2DPCA), two-dimensional linear discriminative analysis (2DLDA), two-dimensional locality-preserving projection (2DLPP), 2DJLNDA preserves not only local/nonlocal intrinsic structure but also local/nonlocal penalization structure of the image data in the high-dimensional space, which can be powerful in extracting intrinsic information of the image data in the low-dimensional space. The experimental results on the ORL, Yale, AR and UMIST face datasets indicate that 2DJLNDA is capable of extracting effective image features and outperforms 2DPCA, 2DLDA and 2DLPP. The 2D image features extracted by 2DJLNDA further improve the performance of deep neural networks (DNNs), e.g., stacked denoising autoencoder, and convolutional neural network (CNN) significantly. These studying results illustrate that the feature face images will provide more discriminant features than the original face images for DNNs. Therefore, 2DJLNDA-based 2D feature image extraction can be used as an effective preprocessing of DNNs (e.g., CNN) for face recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

References

  1. Du S, Ward RK (2010) Adaptive region-based image enhancement method for robust face recognition under variable illumination conditions. IEEE Trans Circuits Syst Video Technol 20(9):1165–1175

    Google Scholar 

  2. Deepa GM, Keerthi R, Meghana N, Manikantan K (2012) Face recognition using spectrum-based feature extraction. Appl Soft Comput 12(9):2913–2923

    Google Scholar 

  3. Luh GC, Lin CY (2011) PCA based immune networks for human face recognition. Appl Soft Comput 11(2):1743–1752

    Google Scholar 

  4. Arashloo SR, Kittler J (2014) Class-specific kernel fusion of multiple descriptors for face verification using multiscale binarised statistical image features. IEEE Trans Inf Forensics Secur 9(12):2100–2109

    Google Scholar 

  5. He X, Yan S, Hu Y, Niyogi P, Zhang HJ (2005) Face recognition using Laplacianfaces. IEEE Trans Pattern Anal Mach Intell 27(3):328–340

    Google Scholar 

  6. Nanni L, Lumini A, Brahnam S (2017) Ensemble of texture descriptors for face recognition obtained by varying feature transforms and preprocessing approaches. Appl Soft Comput 61:8–16

    Google Scholar 

  7. Sirovich L, Kirby M (1987) Low-dimensional procedure for the characterization of human faces. J Opt Soc Am A 4(3):519–524

    Google Scholar 

  8. Turk M, Pentland AP (1991) Face recognition using eigenfaces. In: IEEE conference on computer vision and pattern Recognition, pp 586–591

  9. Wold S, Esbensen K, Geladi P (1987) Principal component analysis. Chemom Intell Lab Syst 2(1–3):37–52

    Google Scholar 

  10. Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Google Scholar 

  11. Mika S, Ratsch G, Weston J, Scholkopf B, Mullers KR (1999) Fisher discriminant analysis with kernels. In: Proceedings of the 1999 IEEE signal processing society workshop, pp 41–48

  12. Chang Y, Hu C, Turk M (2003) Manifold of facial expression. In: IEEE international workshop on analysis and modeling of faces and gestures, pp 28–35

  13. Lee KC, Ho J, Yang MH, Kriegman D (2003) Video-based face recognition using probabilistic appearance manifolds. Computer vision and pattern recognition. In: IEEE computer society conference on computer vision and pattern recognition, pp 313–320

  14. Tenenbaum JB, de Silva V, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319–2323

    Google Scholar 

  15. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Google Scholar 

  16. Belkin M, Niyogi P (2001) Laplacian Eigenmaps and spectral techniques for Embedding and clustering. In: Proceedings conference of advances in neural information processing system, vol 14, pp 585–591

  17. He X, Niyogi P (2003) Locality preserving projections. In: Proceedings conference of advances in neural information processing system, pp 1–6

  18. Yu W, Teng X, Liu C (2006) Face recognition using discriminant locality preserving projections. Image Vis Comput 24(3):239–248

    Google Scholar 

  19. Kokiopoulou E, Saad Y (2005) Orthogonal neighborhood preserving projections. In: Fifth IEEE international conference on data mining, pp 1–8

  20. Gui J, Jia W, Zhu L, Wang SL, Huang DS (2010) Locality preserving discriminant projections for face and palmprint recognition. Neurocomputing 73(13):2696–2707

    Google Scholar 

  21. Krzanowski WJ, Jonathan P, McCarthy WV, Thomas MR (1995) Discriminant analysis with singular covariance matrices: methods and applications to spectroscopic data. Appl Stat 44(1):101–115

    MATH  Google Scholar 

  22. Xu Y, Song F, Feng G, Zhao Y (2010) A novel local preserving projection scheme for use with face recognition. Expert Syst Appl 37(9):6718–6721

    Google Scholar 

  23. Ding J, Wen C, Li G, Chua C (2016) Locality sensitive batch feature extraction for high-dimensional data. Neurocomputing 171:664–672

    Google Scholar 

  24. Yu J (2012) Semiconductor manufacturing process monitoring using gaussian mixture model and Bayesian method with local and nonlocal information. IEEE Trans Semicond Manuf 25(3):480–493

    Google Scholar 

  25. Yu J, Lu X (2016) Wafer map defect detection and recognition using joint local and nonlocal linear discriminant analysis. IEEE Trans Semicond Manuf 29(1):33–43

    Google Scholar 

  26. Bian W, Tao D (2011) Max–min distance analysis by using sequential SDP relaxation for dimension reduction. IEEE Trans Pattern Anal Mach Intell 33(5):1037–1050

    Google Scholar 

  27. Xu C, Tao D, Ru Y (2014) Large-margin weakly supervised dimensionality reduction. In: Proceedings of the 31st international conference on machine learning (ICML-14), pp 865–873

  28. Zhang Y, Zhou ZH (2010) Multilabel dimensionality reduction via dependence maximization. ACM Trans Knowl Discov Data (TKDD) 4(3):14–22

    Google Scholar 

  29. Xu C, Tao D, Xu C (2015) Multi-view intact space learning. IEEE Trans Pattern Anal Mach Intell 37(12):2531–2544

    Google Scholar 

  30. Luo Y, Tao D, Ramamohanarao K, Xu C, Wen Y (2015) Tensor canonical correlation analysis for multi-view dimension reduction. IEEE Trans Knowl Data Eng 27(11):3111–3124

    Google Scholar 

  31. Qiao L, Chen S, Tan X (2010) Sparsity preserving projections with applications to face recognition. Pattern Recogn 43(1):331–341

    MATH  Google Scholar 

  32. Hu D, Feng G, Zhou Z (2007) Two-dimensional locality preserving projections (2DLPP) with its application to palmprint recognition. Pattern Recogn 40(1):339–342

    MATH  Google Scholar 

  33. Yang J, Zhang D, Frangi AF, Yang J (2004) Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans Pattern Anal Mach Intell 26(1):131–137

    Google Scholar 

  34. Zhang H, Ho JK, Wu QJ, Ye Y (2013) Multidimensional latent semantic analysis using term spatial information. IEEE Trans Cybern 43(6):1625–1640

    Google Scholar 

  35. Li M, Yuan B (2005) 2D-LDA: a statistical linear discriminant analysis for image matrix. Pattern Recogn Lett 26(5):527–532

    Google Scholar 

  36. Chen S, Zhao H, Kong M, Luo B (2007) 2D-LPP: a two-dimensional extension of locality preserving projections. Neurocomputing 70(4):912–921

    Google Scholar 

  37. Gao Q, Hao X, Zhao Q, ShenW Ma J (2013) Feature extraction using two-dimensional neighborhood margin and variation embedding. Comput Vis Image Underst 117(5):525–531

    Google Scholar 

  38. Pan X, Ruan QQ (2008) Palmprint recognition with improved two-dimensional locality preserving projections. Image Vis Comput 26(9):1261–1268

    Google Scholar 

  39. Shikkenawis G, Mitra SK (2016) 2D orthogonal locality preserving projection for image denoising. IEEE Trans Image Process 25(1):262–273

    MathSciNet  MATH  Google Scholar 

  40. Zhang H, Wu QM, Chow WS, Zhao M (2012) A two-dimensional Neighborhood Preserving Projection for appearance-based face recognition. Pattern Recogn 45(5):1866–1876

    MATH  Google Scholar 

  41. Jing X, Sui Z, Yao Y, Sun J (2012) Two-dimensional sparse preserving projection for face recognition. In: International conference on automatic control and artificial intelligence (ACAI 2012) pp 2223–2226

  42. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444

    Google Scholar 

  43. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  44. Wang Y, Wang X, Liu W (2016) Unsupervised local deep feature for image recognition. Inf Sci 351:67–75

    Google Scholar 

  45. Zhan S, Tao QQ, Li XH (2016) Face detection using representation learning. Neurocomputing 187:19–26

    Google Scholar 

  46. Zhang H, Cao X, Ho JK, Chow TW (2017) Object-level video advertising: an optimization framework. IEEE Trans Ind Inf 13(2):520–531

    Google Scholar 

  47. Siniscalchi SM, Yu D, Deng L, Lee CH (2013) Exploiting deep neural networks for detection-based speech recognition. Neurocomputing 106:148–157

    Google Scholar 

  48. TaigmanY, Yang M, Ranzato MA, Wolf L (2014) Deepface: closing the gap to human-level performance in face verification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1701–1708

  49. Zhu Z, Luo P, Wang X, Tang X (2014) Recover canonical-view faces in the wild with deep neural networks, pp 1–10. arXiv preprint arXiv:1404.3543

  50. SunY, LiangD, WangX, TangX (2015) Deepid3: face recognition with very deep neural networks. arXiv preprint arXiv:1502.00873

  51. Schroff F, Kalenichenko D, Philbin J (2015) Facenet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 815–823

  52. Al WSE, Banerjee S (2016) Report on the BTAS 2016 video person recognition evaluation. In: 2016 IEEE 8th international conference on biometrics theory, applications and systems (BTAS), IEEE, pp 1–8

  53. Ding C, Choi J, Tao D, Davis L (2016) Multi-directional multi-level dual-cross patterns for robust face recognition. IEEE Trans Pattern Anal Mach Intell 38(3):518–531

    Google Scholar 

  54. Lu C, Tang X (2015) Surpassing human-level face verification performance on LFW with Gaussian Face. In: Twenty-ninth AAAI conference on artificial intelligence, pp 3811–3819

  55. Ding C, Xu C, Tao D (2015) Multi-task pose-invariant face recognition. IEEE Trans Image Process 24(3):980–993

    MathSciNet  MATH  Google Scholar 

  56. Samaria F, Harter AC (1994) Parameterisation of a stochastic model for human face identification. In: IEEE workshop on applications of computer vision, pp 138–142. Sarasota, USA

  57. Graham DB, Allinson NM (1998) Characterizing virtual eigensignatures for general purpose face recognition. Face Recognit Theory Appl 163:446–456

    Google Scholar 

  58. Martinez AM, Benavente R (1998) The AR face database. CVC Technical Report, vol 24. Hammond, IN

  59. Juliana VA, Andrés ÁM, Genaro DS, Germán DC (2009) Automatic choice of the number of nearest neighbors in locally linear embedding. In: Proceedings 14th CIARP, vol 5856, pp 77–84

  60. Kouropteva O, Okun O, Pietikainen M (2002) Selection of the optimal parameter value for the locally linear embedding algorithm. In: Proceedings 1st International conference fuzzy system knowledge discovery, pp 359–363

Download references

Acknowledgements

This research was financially supported by National Natural Science Foundation of China (No. 71777173) and Fundamental Research Funds for the Central Universities.

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, Tongji University, Shanghai, 200084, China

    Jianbo Yu, Haiqiang Liu & Xiaoyun Zheng

Authors
  1. Jianbo Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haiqiang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoyun Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianbo Yu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, J., Liu, H. & Zheng, X. Two-dimensional joint local and nonlocal discriminant analysis-based 2D image feature extraction for deep learning. Neural Comput & Applic 32, 6009–6024 (2020). https://doi.org/10.1007/s00521-019-04085-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04085-0

Keywords

Search

Navigation