Skip to main content
SpringerLink
Log in

Sparse PCA by iterative elimination algorithm

  • Published:
Advances in Computational Mathematics Aims and scope Submit manuscript

Abstract

In this paper we proposed an iterative elimination algorithm for sparse principal component analysis. It recursively eliminates variables according to certain criterion that aims to minimize the loss of explained variance, and reconsiders the sparse principal component analysis problem until the desired sparsity is achieved. Two criteria, the approximated minimal variance loss (AMVL) criterion and the minimal absolute value criterion, are proposed to select the variables eliminated in each iteration. Deflation techniques are discussed for multiple principal components computation. The effectiveness is illustrated by both simulations on synthetic data and applications on real data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Cadima, J., Jolliffe, I.T.: Loadings and correlations in the interpretation of principal components. J. Appl. Stat. 22(2), 203–215 (1995)

    Article  MathSciNet  Google Scholar 

  2. d’Aspremont, A., El Ghaoui, L., Jordan, M.I., Lanckriet, G.R.G.: A direct formulation for sparse PCA using semidefinite programming. SIAM Rev. 49(3), 434–448 (electronic) (2007)

    Article  MATH  MathSciNet  Google Scholar 

  3. d’Aspremont, A., Bach, F., El Ghaoui, L.: Optimal solutions for sparse principal component analysis. J. Mach. Learn. Res. 9, 1269–1294 (2008)

    MATH  MathSciNet  Google Scholar 

  4. Golub, G., Loan, C.V.: Matrix Computations. Johns Hopkins University Press, Baltimore (1983)

    MATH  Google Scholar 

  5. Guyon, I., Weston, J., Barnhill, S., Vapnik, V.: Gene selection for cancer classification using support vector machines. Mach. Learn. 46, 389–422 (2002)

    Article  MATH  Google Scholar 

  6. Jeffers, J.: Two case studies in the application of principal component. Appl. Stat. 16, 225–236 (1967)

    Article  Google Scholar 

  7. Jolliffe, I.: Principal Component Analysis. Springer, Berlin (2002)

    MATH  Google Scholar 

  8. Jolliffe, I.T., Trendafilov, N.T., Uddin, M.: A modified principal component technique based on the LASSO. J. Comput. Graph. Stat. 12(3), 531–547 (2003)

    Article  MathSciNet  Google Scholar 

  9. Journée, M., Nesterov, Y., Richtárik, P., Sepulchre, R.: Generalized power method for sparse principal component analysis. J. Mach. Learn. Res. 11, 517–553 (2010)

    MathSciNet  Google Scholar 

  10. Mackey, L.: Deflation methods for sparse PCA. In: Koller, D., Schuurmans, D., Bengio, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems, vol. 21, pp. 1017–1024 (2009)

  11. Moghaddam, B., Weiss, Y., Avidan, S.: Spectral bounds for sparse PCA: exact and greedy algorithms. In: Weiss, Y., Schölkopf, B., Platt, J. (eds.) Advances in Neural Information Processing Systems, vol. 18, pp. 915–922. MIT Press, Cambridge (2006)

    Google Scholar 

  12. Ramaswamy, S., Tamayo, P., Rifkin, R., Mukheriee, S., Yeang, C., Angelo, M., Ladd, C., Reich, M., Latulippe, E., Mesirov, J., Poggio, T., Gerald, W., Loda, M., Lander, E., Golub, T.: Multiclass cancer diagnosis using tumor gene expression signature. Proc. Natl. Acad. Sci. 98, 15149–15154 (2001)

    Article  Google Scholar 

  13. Shen, H., Huang, J.Z.: Sparse principal component analysis via regularized low rank matrix approximation. J. Multivar. Anal. 99(6), 1015–1034 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  14. Sriperumbudur, B., Torres, D., Lanckriet, G.: Sparse eigen methods by dc programming. In: Proceedings of the 24th International Conference on Machine Learning, pp. 831–838 (2007)

  15. Sriperumbudur, B., Torres, D., Lanckriet, G.: A D.C. Programming Approach to the Sparse Generalized Eigenvalue Problem (2009). Available on arXiv:0901.1504v2

  16. Zass, R., Shashua, A.: Nonnegative sparse PCA. Adv. Neural Inf. Process. Syst. 19, 1561 (2007)

    Google Scholar 

  17. Zou, H., Hastie, T., Tibshirani, R.: Sparse principal component analysis. J. Comput. Graph. Stat. 15(2), 265–286 (2006)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Michigan State University, East Lansing, MI, 48824, USA

    Yang Wang

  2. Department of Mathematical Sciences, Middle Tennessee State University, Murfreesboro, TN, 37132, USA

    Qiang Wu

Authors
  1. Yang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qiang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qiang Wu.

Additional information

Communicated by Charles Micchelli.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wang, Y., Wu, Q. Sparse PCA by iterative elimination algorithm. Adv Comput Math 36, 137–151 (2012). https://doi.org/10.1007/s10444-011-9186-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10444-011-9186-3

Keywords

Mathematics Subject Classifications (2010)

Search

Navigation