Skip to main content
Springer Nature Link
Log in

Incremental maximum margin criterion based on eigenvalue decomposition updating algorithm

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Dimensionality reduction has been proven to be a critical data processing step for face recognition. Maximum margin criterion (MMC) is one of the popular supervised dimensionality reduction algorithms. However, the original implementation of MMC is not suitable for incremental learning problem. In this paper, we first propose an eigenvalue decomposition updating algorithm (EVDU) for symmetric matrix. Then, based on our proposed EVDU technique, we propose an incremental MMC (EVDU-IMMC) method which can update the discriminant vectors of MMC when new samples are inserted into the training set. Experiments on ORL and PIE face databases show that the proposed EVDU-IMMC gives the same performance as the batch MMC with much lower computational complexity. The experimental results also show that our proposed EVDU-IMMC gives better performance than other IMMC method in terms of recognition accuracy and computational efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Cheng, H., Vu, K., Hua, K.A.: SubSpace projection: a unified framework for a class of partition-based dimension reduction techniques. Inf. Sci. 179, 1234–1248 (2009)

    Article  Google Scholar 

  2. Garcia-Pedrajas, N., Maudes-Raedo, J., Garcia-Osorio, C., Rodriguez-Diez, J.J.: Supervised subspace projections for constructing ensembles of classifiers. Inf. Sci. 193(6), 1–21 (2012)

    Article  Google Scholar 

  3. Tang, H., Maitre, H., Boujemaa, N., Jiang, W.: On the relevance of linear discriminative features. Inf. Sci. 180(18), 3422–3433 (2010)

    Article  Google Scholar 

  4. Yin, H.J., Huang, W.L.: Adaptive nonlinear manifolds and their applications to pattern recognition. Inf. Sci. 180(14), 2649–2662 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  5. Connolly, J.-F., Granger, E., Sabourin, R.: An adaptive classification system for video-based face recognition. Inf. Sci. 192(6), 50–70 (2012)

    Article  Google Scholar 

  6. Lu, G.-F., Jin, Z., Zou, J.: Face recognition using discriminant sparsity neighborhood preserving embedding. Knowl.-Based Syst. 31(7), 119–127 (2012)

    Article  Google Scholar 

  7. Duda, R.O., Hart, P.E., Stork. D.G.: Pattern Classification, 2nd edn. Wiley, New York (2000)

  8. Howland, P., Wang, J., Park, H.: Solving the small sample size problem in face recognition using generalized discriminant analysis. Pattern Recognit. 39(2), 227–287 (2006)

    Article  Google Scholar 

  9. Belhumeour, P.N., Hespanha, J.P., Kriegman, D.J.: Eigenfaces vs. Fisherfaces: recognition using class specific linear projection, IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 711–720 (1997)

  10. Yu, H., Yang, J.: A direct LDA algorithm for high-dimensional data with application to face recognition. Pattern Recognit. 33(1), 1726–1731 (2000)

    Google Scholar 

  11. Chen, L.F., Liao, H.Y.M., Ko, M.T., Yu, G.J.: A new LDA-based face recognition system which can solve the small sample size problem. Pattern Recognit. 33(10), 1713–1726 (2000)

  12. Lu, G.-F., Wang, Y.: Feature extraction using a fast null space based linear discriminant analysis algorithm. Inf. Sci. 193(6), 72–80 (2012)

    Article  Google Scholar 

  13. Wang, X., Tang, X.: Dual-space linear discriminant analysis for face recognition. In: IEEE Computer Society Conf. on Computer Vision and Pattern Recognition (CVPR’04), Washington, pp. 564–569 (2004)

  14. Ye, J., Janardan, R., Park, C.H., Park, H.: An optimization criterion for generalized discriminant analysis on undersampled problems. IEEE Trans. Pattern Anal. Mach. Intell. 26(8), 982–994 (2004)

    Article  Google Scholar 

  15. Li, H., Jiang, T., Zhang, K.: Efficient and robust feature extraction by maximum margin criterion. IEEE Trans. Neural Netw. 17(1), 1157–1165 (2006)

    Article  Google Scholar 

  16. Song, F., Zhang, D., Mei, D., Guo, Z.: A multiple maximum scatter difference discriminant criterion for facial feature extraction. IEEE Trans. Syst. Man Cybern. Part B Cybern. 37(6), 1599–1606 (2007)

    Article  Google Scholar 

  17. Hall, P.M., Marshall, A.D., Martin, R.R.: Merging and splitting eigenspace models. IEEE Trans. Pattern Anal. Mach. Intell. 22(9), 1042–1049 (2000)

  18. Ross, D.A., Lim, J., Lin, R.-S.: Incremental learning for robust visual tracking. Int. J. Comput. Vis. 77(1–3), 125–141 (2008)

  19. Ghashami, M., Phillips, J.M.: Relative errors for deterministic low-rank matrix approximations. In: Proceedings of the Twenty-Fifth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 707–717 (2014)

  20. Ghashami, M., Liberty, E., Phillips, J.M., Woodruff, D.P.: Frequent directions: simple and deterministic matrix sketching 1–28 (2015). arXiv:1501.01711v2

  21. Feldman, D., Schmidt, M., Sohler, C.: Turning big data into tiny data: constant-size coresets for \(k\)-means, PCA and projective clustering. In: Proceedings of the Twenty-Fourth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 1434–1453 (2013)

  22. Cohen, M.B., Elder, S., Musco, C., Musco, C., Persu, M.: Dimensionality reduction for \(k\)-means clustering and low rank approximation. 1–37 (2014). arXiv:1410.6801v3

  23. Mao, J., Jain, K.: Artifical neural networks for feature extraction and multivariate data projection. IEEE Trans. Neural Netw. 6(2), 296–317 (1995)

    Article  Google Scholar 

  24. Pang, S., Ozawa, S., Kasabov, N.: Incremental linear discriminant analysis for classification of data streams. IEEE Trans. Syst. Man Cybern. Part B Cybern. 35(5), 905–914 (2005)

    Article  MATH  Google Scholar 

  25. Zha, H., Simon, H.D.: On updating problems in latent semantic indexing. SIAM J. Sci. Comput. 21(2), 782–791 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  26. Zhao, H., Yuen, P.C.: Incremental linear discriminant analysis for face recognition. IEEE Trans. Syst. Man Cybern. Part B Cybern. 38(1), 210–221 (2008)

  27. Kim, T.-K., Wong, S.F., Stenger, B., Kittler, J., Cipolla, R.: Incremental linear discriminant analysis using sufficient spanning set approximations. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR’07), Minneapolis, pp. 1–7 (2007)

  28. Kim, T.-K., Stenger, B., Kittler, J., Cipolla, R.: Incremental linear discriminant analysis using sufficient spanning sets and its applications. Int. J. Comput. Vis. 91(2), 216–232 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  29. Ye, J., Li, Q., Xiong, H., Park, H.: IDR/QR: an incremental dimension reduction algorithm via QR decomposition. IEEE Trans. Knowl. Data Eng. 17(9), 1208–1222 (2005)

    Article  Google Scholar 

  30. Zheng, W., Tang, X.: Fast algorithm for updating the discriminnat vectors of dual-space LDA. IEEE Trans. Inf. Forensics Secur. 4(3), 418–427 (2009)

    Article  MathSciNet  Google Scholar 

  31. Lu, G.-F., Zou, J., Wang, Y.: Incremental complete LDA for face recognition. Pattern Recognit. 45(7), 2510–2521 (2012)

    Article  MATH  Google Scholar 

  32. Lu, G.-F., Zou, J., Wang, Y.: Incremental learning of complete linear discriminant analysis for face recognition. Knowl.-Based Syst. 31(7), 19–27 (2012)

    Article  Google Scholar 

  33. Lu, G.-F., Zou, J., Wang, Y.: Incremental learning of discriminant common vectors for feature extraction. Appl. Math. Comput. 218(22), 11269–11278 (2012)

    Article  MATH  Google Scholar 

  34. Weng, J., Zhang, Y., Hwang, W.S.: Candid covariance-free incremental principal component analysis. IEEE Trans. Pattern Anal. Mach. Intell. 25(8), 1034–1040 (2003)

  35. Yan, J., Zhang, B., Yan, S., Yang, Q., Li, H., Chen, Z., Xi, W., Fan, W., Ma, W.-Y., Cheng, Q.: IMMC: incremental maximum margin criterion. In: Proceedings of Int’l Conf. Knowledge Discovery and Data Mining (KDD’04), pp. 1–6. ACM, Seattle (2004)

  36. Yan, J., Zhang, B., Yan, S., Liu, N., Yang, Q., Cheng, Q., Li, H., Chen, Z., Ma, W.-Y.: A scalable supervised algorithm for dimensionality reduction on streaming data. Inf. Sci. 176(14), 2042–2065 (2006)

    Article  Google Scholar 

  37. Zhao, H., Yuen, P.C., Kwok, J.T.: A novel incremental principal component analysis and its application for face recognition. IEEE Trans. Syst. Man Cybern. Part B Cybern. 36(4), 873–886 (2006)

    Article  Google Scholar 

  38. Zheng, W., Zou, C., Zhao, L.: Weighted maximum margin discriminant analysis with kernels. Neurocomputing 67, 357–362 (2005)

    Article  Google Scholar 

  39. Liu, J., Chen, S.C., Tan, X., Zhang, D.: Comments on “Efficient and robust maximal margin criterion”. IEEE Trans. Neural Netw. 18(6), 1862–1864 (2007)

    Article  Google Scholar 

  40. Lu, G.-F., Lin, Z., Jin, Z.: Face recognition using discriminant locality preserving projections based on maximum margin criterion. Pattern Recognit. 43(10), 3572–3579 (2010)

    Article  MATH  Google Scholar 

Download references

Acknowledgments

This research is supported by Anhui Provincial Natural Science Foundation (No. 1308085MF95), the Pre-research Foundation of NSFC of Anhui Polytechnic University (zryy1305), the Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information (Nanjing University of Science and Technology), Ministry of Education (Grant No. 30920130122005), China Postdoctoral Science Foundation (2013M531251), NSFC of China (Nos. 61231002, 61073137, 61203243), the Natural Science Foundation of the Anhui Higher Education Institutions of China (No. KJ2013B031) and Jiangxi Provincial Natural Science Foundation of China (20122BAB211025).

Author information

Authors and Affiliations

  1. School of Computer and Information, Anhui Polytechnic University, Wuhu, 241000, Anhui, China

    Gui-Fu Lu & Jian Zou

Authors
  1. Gui-Fu Lu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jian Zou
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Gui-Fu Lu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, GF., Zou, J. Incremental maximum margin criterion based on eigenvalue decomposition updating algorithm. Machine Vision and Applications 26, 807–817 (2015). https://doi.org/10.1007/s00138-015-0691-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-015-0691-0

Keywords