Skip to main content
SpringerLink
Log in

Large-scale image recognition based on parallel kernel supervised and semi-supervised subspace learning

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Kernel discriminant subspace learning technique is effective to exploit the structure of image dataset in the high-dimensional nonlinear space. However, for large-scale image recognition applications, this technique usually suffers from large computational burden. Although some kernel accelerating methods have been presented, how to greatly reduce computing time and simultaneously keep favorable recognition accuracy is still challenging. In this paper, we introduce the idea of parallel computing into kernel subspace learning and build a parallel kernel discriminant subspace learning framework. In this framework, we firstly design a random non-overlapping equal data division strategy to divide the whole training set into several subsets and assign each computational node a subset. Then, we separately learn kernel discriminant subspaces from these subsets without mutual communications and finally select the most appropriate subspace to classify test samples. Under the built framework, we propose two novel kernel subspace learning approaches, i.e., parallel kernel discriminant analysis (PKDA) and parallel kernel semi-supervised discriminant analysis (PKSDA). We show the superiority of the proposed approaches in terms of time complexity as compared with related methods, and provide the fundamental supports for our framework. For experiment, we establish a parallel computing environment and employ three public large-scale image databases as experiment data. Experimental results demonstrate the efficiency and effectiveness of the proposed approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Liu W, Zhang H, Tao D, Wang Y, Lu K (2014) Large-scale paralleled sparse principal component analysis. Multimedia Tools Appl. doi:10.1007/s11042-014-2004-4

    Google Scholar 

  2. Belhumeur PN, Hespanda J, Kiregeman D (1997) Eigenfaces versus fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Article  Google Scholar 

  3. Pang SN, Ban T, Kadobayashi Y, Kasabov NK (2012) LDA merging and splitting with applications to multiagent cooperative learning and system alteration. IEEE Trans Syst Man Cybern Part B 42(2):552–564

    Article  Google Scholar 

  4. Ye JP (2007) Least squares linear discriminant analysis. In: International conference on machine learning, pp 1087–1093

  5. Su Y, Shan SG, Chen XL, Gao W (2008) Classifiability-based optimal discriminatory projection pursuit. In: IEEE conference on computer vision and pattern recognition, pp 1–7

  6. Zhang TH, Huang KQ, Li XL, Yang J, Tao DC (2010) Discriminative orthogonal neighborhood-preserving projections for classification. IEEE Trans Syst Man Cybern Part B 40(1):253–263

    Article  Google Scholar 

  7. Li X, Hu WM, Wang HZ, Zhang ZF (2010) Linear discriminant analysis using rotational invariant L1 norm. Neurocomputing 73(13–15):2571–2579

    Article  Google Scholar 

  8. Zhao C, Miao D, Lai Z, Gao C, Liu C, Yang J (2013) Two-dimensional color uncorrelated discriminant analysis for face recognition. Neurocomputing 113:251–261

    Article  Google Scholar 

  9. Zhong FJ, Zhang JS (2013) Linear discriminant analysis based on L1-norm maximization. IEEE Trans Image Process 22(8):3018–3027

    Article  MathSciNet  Google Scholar 

  10. Yang M, Sun S (2014) Multi-view uncorrelated linear discriminant analysis with applications to handwritten digit recognition. In: International joint conference on neural networks, pp 4175–4181

  11. Zhao C, Lai Z, Miao D, Wei Z, Liu C (2014) Graph embedding discriminant analysis for face recognition. Neural Comput Appl 24(7–8):1697–1706

    Article  Google Scholar 

  12. Wang Z, Ruan Q, An G (2015) Projection-optimal local fisher discriminant analysis for feature extraction. Neural Comput Appl 26(3):589–601

    Article  Google Scholar 

  13. Wu F, Jing XY, Yao YF, Yue D, Chen J (2015) Group recursive discriminant subspace learning with image set decomposition. Neural Comput Appl. doi:10.1007/s00521-015-1966-0

    Google Scholar 

  14. Li S, Fu Y (2014) Robust subspace discovery through supervised low-rank constraints. SIAM international conference on data mining, pp 163–171

  15. Li S, Fu Y (2015) Learning robust and discriminative subspace with low-rank constraints. IEEE Trans Neural Netw Learn Syst (in press)

  16. Zhang DQ, Zhou ZH, Chen SC (2007) Semi-supervised dimensionality reduction. In: International conference on data mining, pp 629–634

  17. Cai D, He XF, Han JW (2007) Semi-supervised discriminant analysis. In: International conference on computer vision, pp 1–7

  18. Fan MY, Gu NN, Qiao H, Zhang B (2011) Sparse regularization for semi-supervised classification. Pattern Recognit 44(8):1777–1784

    Article  MATH  Google Scholar 

  19. Zhang TT, Ji RR, Liu W, Tao DC, Hua G (2013) Semi-supervised learning with manifold fitted graphs. In: International joint conference artificial intelligence, pp 1896–1902

  20. Tu W, Sun S (2013) Semi-supervised feature extraction for EEG classification. Pattern Anal Appl 16(2):213–222

    Article  MathSciNet  Google Scholar 

  21. Zhao M, Zhang Z, Chow TW, Li B (2014) A general soft label based linear discriminant analysis for semi-supervised dimensionality reduction. Neural Netw 55:83–97

    Article  MATH  Google Scholar 

  22. Gao Q, Huang Y, Gao X, Shen W, Zhang H (2015) A novel semi-supervised learning for face recognition. Neurocomputing 152:69–76

    Article  Google Scholar 

  23. Kim M (2015) Greedy approaches to semi-supervised subspace learning. Pattern Recognit 48(4):1559–1566

    Article  Google Scholar 

  24. Taylor JS, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University Press, Cambridge

    Book  Google Scholar 

  25. Mika S, Ratsch G, Weston J, Scholkopf B, Muller K (1999) Fisher discriminant analysis with kernels. In: IEEE signal processing society workshop on neural networks for signal processing IX, pp 41–48

  26. Baudat G, Anouar F (2000) Generalized discriminant analysis using a kernel approach. Neural Comput 12(10):2385–2404

    Article  Google Scholar 

  27. Jing XY, Yao YF, Zhang D, Yang JY, Li M (2007) Face and palmprint pixel level fusion and KDCV–RBF classifier for small sample biometric recognition. Pattern Recognit 40(11):3209–3224

    Article  MATH  Google Scholar 

  28. Chen B, Yuan L, Liu H, Bao Z (2007) Kernel subclass discriminant analysis. Neurocomputing 71(1–3):455–458

    Article  Google Scholar 

  29. Zheng WM, Lin ZC, Tang XO (2010) A rank-one update algorithm for fast solving kernel Foley–Sammon optimal discriminant vectors. IEEE Trans Neural Netw 21(3):393–403

    Article  Google Scholar 

  30. Li JB, Peng Y, Liu D (2013) Quasiconformal kernel common locality discriminant analysis with application to breast cancer diagnosis. Inf Sci 223:256–269

    Article  MathSciNet  MATH  Google Scholar 

  31. Zheng W, Lin Z, Wang H (2014) L1-norm kernel discriminant analysis via Bayes error bound optimization for robust feature extraction. IEEE Trans Neural Netw Learn Syst 25(4):793–805

    Article  Google Scholar 

  32. Iosifidis A, Tefas A, Pitas I (2014) Kernel reference discriminant analysis. Pattern Recognit Lett 49:85–91

    Article  Google Scholar 

  33. Tahir MA, Kittler J, Bouridane A (2015) Multi-label classification using stacked spectral kernel discriminant analysis. Neurocomputing. doi:10.1016/j.neucom.2015.06.023

    Google Scholar 

  34. Hu EL, Chen SC, Zhang DQ, Yin XS (2010) Semisupervised kernel matrix learning by kernel propagation. IEEE Trans Neural Netw 21(11):1831–1841

    Article  Google Scholar 

  35. Zhang Y, Yeung DY (2011) Semisupervised generalized discriminant analysis. IEEE Trans Neural Netw 22(8):1207–1217

    Article  Google Scholar 

  36. Zhao M, Li B, Wu Z, Zhan C (2015) Image classification via least square semi-supervised discriminant analysis with flexible kernel regression for out-of-sample extension. Neurocomputing 153:96–107

    Article  Google Scholar 

  37. Faußer S, Schwenker F (2014) Semi-supervised clustering of large data sets with kernel methods. Pattern Recognit Lett 37:78–84

    Article  Google Scholar 

  38. Xiao JX, Hays J, Ehinger KA, Oliva A, Torralba A (2010) SUN database: large-scale scene recognition from abbey to zoo. In: IEEE conference on computer vision and pattern recognition, pp 3485–3492

  39. Tipping ME (2000) Sparse kernel principal component analysis. Adv Neural Inf Process Syst 13:633–639

    Google Scholar 

  40. Jiang XH, Snapp RR, Motai YC, Zhu XQ (2006) Accelerated kernel feature analysis. In: IEEE conference on computer vision and pattern recognition, pp 109–116

  41. Franc V, Hlavac V (2006) Greedy kernel principal component analysis. Lect Notes Comput Sci 3948:87–105

    Article  Google Scholar 

  42. Xu Y, Zhang D, Jin Z, Li M, Yang JY (2006) A fast kernel-based nonlinear discriminant analysis for multi-class problems. Pattern Recognit 39(6):1026–1033

    Article  MATH  Google Scholar 

  43. Zeng WJ, Li XL, Zhang XD, Cheng E (2010) Kernel-based nonlinear discriminant analysis using minimum squared errors criterion for multiclass and undersampled problems. Signal Process 90(8):2333–2343

    Article  MATH  Google Scholar 

  44. Cai D, He XF, Han JW (2011) Speed up kernel discriminant analysis. Int J Very Large Data Bases 20(1):21–33

    Article  Google Scholar 

  45. Wang HX, Hu ZL, Zhao YE (2007) An efficient algorithm for generalized discriminant analysis using incomplete Cholesky decomposition. Pattern Recognit Lett 28(2):254–259

    Article  Google Scholar 

  46. Li M, Bi W, Kwok JT, Lu BL (2015) Large-scale Nyström kernel matrix approximation using randomized SVD. IEEE Trans Neural Netw Learn Syst 26(1):152–164

    Article  MathSciNet  Google Scholar 

  47. Sun P, Yao X (2010) Sparse approximation through boosting for learning large scale kernel machines. IEEE Trans Neural Netw 21(6):883–894

    Article  Google Scholar 

  48. Rahimi A, Recht B (2009) Random features for large-scale kernel machines. Adv Neural Inf Process Syst: 1–10

  49. Zhou ZH, Chindaro S, Deravi F (2009) A classification framework for large-scale face recognition systems. Lect Notes Comput Sci 5558:337–346

    Article  Google Scholar 

  50. Vedaldi A, Zisserman A (2012) Sparse kernel approximations for efficient classification and detection. In: IEEE conference on computer vision and pattern recognition, pp 2320–2327

  51. Macua SV, Belanovic P, Zazo S (2011) Distributed linear discriminant analysis. In: International conference on acoustics, speech and signal processing, pp 3288–3291

  52. Jing XY, Li S, Zhang D, Yang J, Yang JY (2012) Supervised and unsupervised parallel subspace learning for large-scale image recognition. IEEE Trans Circuits Syst Video Technol 22(10):1497–1511

    Article  Google Scholar 

  53. Fu JS, Yang WL (2011) Distributed kernel Fisher discriminant analysis for radar image recognition. In: International conference on mechanic automation and control engineering, pp 1241–1244

  54. Ma ZY, Leijion A (2009) Bata mixture models and the application to image classification. In: International conference on image processing, pp 2045–2048

  55. Mizukami Y, Tadamura K, Warrell J, Li P, Prince S (2010) CUDA implementation of deformable pattern recognition and its application to MNIST handwritten digit database. In: International conference on pattern recognition, pp 2001–2004

  56. Lee KC, Ho J, Kriegman D (2005) Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans Pattern Anal Mach Intell 27(5):684–698

    Article  Google Scholar 

  57. Chawla NV, Karakoulas GI (2005) Learning from labeled and unlabeled data: an empirical study across techniques and domains. J Artif Intell Res 23:331–366

    MATH  Google Scholar 

  58. Turk MA, Pentland AP (1991) Face recognition using Eigenfaces. IEEE Conference on computer vision and pattern recognition, pp 586–591

  59. Kittler J, Hatef M, Duin RPW, Matas J (1998) On combining classifiers. IEEE Trans Pattern Anal Mach Intell 20(3):226–239

    Article  Google Scholar 

Download references

Acknowledgments

This work was fully supported by the National Natural Science Foundation of China under Project Nos. 61272273 and 61233011, the Major Science and Technology Innovation Plan of Hubei Province under Project No. 2013AAA020, the Research Project of Nanjing University of Posts and Telecommunications under Project No. XJKY14016, and the Postgraduate Scientific Research and Innovation Plan of Jiangsu Province Universities under Project No. CXLX13_465.

Author information

Authors and Affiliations

  1. College of Communication and Information Engineering, Nanjing University of Posts and Telecommunications, Nanjing, 210003, China

    Fei Wu

  2. College of Automation, Nanjing University of Posts and Telecommunications, Nanjing, 210003, China

    Xiao-Yuan Jing, Qian Liu & Song-Song Wu

  3. State Key Laboratory of Software Engineering, School of Computer, Wuhan University, Wuhan, 430072, China

    Xiao-Yuan Jing & Guo-Liang He

Authors
  1. Fei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiao-Yuan Jing
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qian Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Song-Song Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guo-Liang He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiao-Yuan Jing.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, F., Jing, XY., Liu, Q. et al. Large-scale image recognition based on parallel kernel supervised and semi-supervised subspace learning. Neural Comput & Applic 28, 483–498 (2017). https://doi.org/10.1007/s00521-015-2081-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-015-2081-y

Keywords

Search

Navigation