Skip to main content
SpringerLink
Log in

Principal component analysis using QR decomposition

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

In this paper we present QR based principal component analysis (PCA) method. Similar to the singular value decomposition (SVD) based PCA method this method is numerically stable. We have carried out analytical comparison as well as numerical comparison (on Matlab software) to investigate the performance (in terms of computational complexity) of our method. The computational complexity of SVD based PCA is around \( 14dn^{2} \) flops (where d is the dimensionality of feature space and n is the number of training feature vectors); whereas the computational complexity of QR based PCA is around \( 2dn^{2} \, + \,2dth \) flops (where t is the rank of data covariance matrix and h is the dimensionality of reduced feature space). It is observed that the QR based PCA is more efficient in terms of computational complexity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. To deal with higher order matrices, other variants of PCA have been proposed in the literature. See the following references for details [23–27]

References

  1. Alsberg BK, Kvalheim OM (1994) Speed improvement of multivariate algorithms by the method of postponed basis matrix multiplication Part I. Principal component analysis. Chemom Intell Lab Syst 24:31–42

    Article  Google Scholar 

  2. Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE T PAMI 19(7):711–720

    Article  Google Scholar 

  3. Chen T-C, Chen K, Liu W, Chen L-G (2009) One-chip principal component analysis with a mean pre-estimation method for spike sorting. In: Proceedings of IEEE international symposium on circuits and systems, pp 3310–3113

  4. Chen T-C, Liu W, Chen L-G (2008) VLSI architecture of leading eigenvector generation for on-chip principal component analysis spike sorting system. In: Proceedings of 30th Annual International Conference IEEE engineering in medicine and biology society EMBS’08, pp 3192–3195

  5. Fukunaga K (1990) Introduction to Statistical Pattern Recognition. Academic Press, USA

    MATH  Google Scholar 

  6. Golub GH, Loan CFV (1996) Matrix Computations. The John Hopkins University Press, Baltimore

    MATH  Google Scholar 

  7. Jian H, Yuen PC, Wen-Sheng C (2004) A novel subspace LDA algorithms for recognition of face images with illumination and pose variations. Proc Int Conf Mach Learn Cybern 6:3589–3594

    Google Scholar 

  8. Kiers HAL, Harshman RA (1997) Relating two proposed methods for speed up of algorithms for fitting two- and three- way principal component and related multilinear models. Chemom Intell Lab Syst 36:31–40

    Article  Google Scholar 

  9. Paliwal KK, Sharma A (2010) Improved direct LDA and its application to DNA microarray gene expression data. Pattern Recogn Lett 31:2489–2492

    Article  Google Scholar 

  10. Roweis S (1997) EM Algorithms for PCA and SPCA. Neural Inf Process Syst 10:626–632

    Google Scholar 

  11. Sajid I, Ahmed MM, Taj I (2008) Design and implementation of a face recognition system using fast PCA. In: Proceedings of the International Symposium on Computer Science and its Applications CSA, pp 126–130

  12. Schilling RJ, Harris SL (2000) Applied Numerical Methods for Engineers Using Matlab and C. Brooks/Cole Publishing, Boston

    Google Scholar 

  13. Sharma A, Paliwal KK (2010) Regularisation of eigenfeatures by extrapolation of scatter-matrix in face-recognition problem. Electron Lett IEE 46(10):682–683

    Article  Google Scholar 

  14. Sharma A, Paliwal KK (2007) Fast principal component analysis using fixed-point algorithm. Pattern Recogn Lett 28(10):1151–1155

    Article  Google Scholar 

  15. Sirovich L (1987) Turbulence and the dynamics of coherent structures. Quart Appl Math XLV 45:561–571

    MathSciNet  MATH  Google Scholar 

  16. Song F, Zhang D, Wang J, Liu H, Tao Q (2007) A parameterized direct LDA and its application to face recognition. Neurocomputing 71:191–196

    Article  Google Scholar 

  17. Swets DL, Weng J (1996) Using discriminative eigenfeatures for image retrieval. IEEE T PAMI 18(8):831–836

    Article  Google Scholar 

  18. Wu X-J, Kittler J, Yang J-Y, Messerm K, Wang S (2004) A new direct LDA (D-LDA) algorithm for feature extraction in face recognition. Proc Int Conf Pattern Recognit 4:545–548

    Google Scholar 

  19. Xiaogang W, Xiaoou T (2004) A unified framework for subspace face recognition. IEEE T PAMI 26(9):1222–1228

    Article  Google Scholar 

  20. Ye J, Xiong T (2006) Computational and theoretical analysis of null space and orthogonal linear discriminant analysis. J Mach Learn Res 7:1183–1204

    MathSciNet  MATH  Google Scholar 

  21. Zhao W, Chellappa R, Nandhakumar N (1998) Empirical performance analysis of linear discriminant classifiers. In: Proceedings of IEEE Conferences on Computer Vision and Pattern Recognition, pp 164–169

  22. Sharma A, Paliwal KK (2008) A Gradient linear discriminant analysis for small sample sized problem. Neural Process Lett 27(1):17–24

    Article  Google Scholar 

  23. Alsberg BK, Kvalheim OM (1994) Speed improvement of multivariate algorithms by the method of postponed basis matrix multiplication Part II. Three-mode principal component analysis. Chemom Intell Lab Syst 24:43–54

    Article  Google Scholar 

  24. Carroll JD, Pruzansky S, Kruskal JB (1980) CANELINC: a general approach to multidimensional analysis of many-way arrays with linear constraints on parameters. Psychometrika 45:3–24

    Article  MathSciNet  MATH  Google Scholar 

  25. Giordani P, Kiers HAL (2004) Three-way component analysis of interval-valued data. J Chemom 18:253–264

    Article  Google Scholar 

  26. Kiers HAL, Harshman RA (2009) An efficient algorithm for Parafac with uncorrelated mode-A components applied to large I × J × K datasets with JK. J Chemom 23:442–447

    Article  Google Scholar 

  27. Kiers HAL, Kroonenberg PM, Berge JMFT (1992) An efficient algorithm for Tuckals3 on data with large numbers of observation units. Psychometrika 57:415–422

    Article  Google Scholar 

  28. Poon B, Amin MA, Yan H (2011) Performance evaluation and comparison of PCA Based human face recognition methods for distorted images. Int J Mach Learn Cybern 2(4):245–259

    Article  Google Scholar 

  29. Sharma A, Paliwal KK (2012) A two-stage linear discriminant analysis for face-recognition. Pattern Recogn Lett 33(9):1157–1162

    Article  Google Scholar 

  30. Pei D, Mi J-S (2011) Attribute reduction in decision formal context based on homomorphism. Int J Mach Learn Cybern 2(4):289–293

    Article  Google Scholar 

  31. Wang X, Dong L, Yan J (2012) Maximum ambiguity based sample selection in fuzzy decision tree induction. IEEE Trans Knowl Data Eng 24(8):1491–1505

    Article  Google Scholar 

  32. Wang X, Dong C (2009) Improving generalization of fuzzy if-then rules by maximizing fuzzy entropy. IEEE Trans Fuzzy Syst 17(3):556–567

    Article  Google Scholar 

  33. Sharma A, Imoto S, Miyano S (2012) A filter based feature selection algorithm using null space of covariance matrix for DNA microarray gene expression data. Curr Bioinform 7(3):289–294

    Article  Google Scholar 

  34. Sharma A, Imoto S, Miyano S, Sharma V (2011) Null space based feature selection method for gene expression data. Int J Mach Learn Cybern. doi:10.1007/s13042-011-0061-9

    Google Scholar 

Download references

Acknowledgments

We thank the Reviewers and the Editor for their constructive comments which appreciably improved the presentation quality of the paper.

Author information

Authors and Affiliations

  1. Laboratory of DNA Information Analysis, Human Genome Center, Institute of Medical Science, University of Tokyo, 4-6-1 Shirokanedai, Minato-ku, Tokyo, 108-8639, Japan

    Alok Sharma, Seiya Imoto & Satoru Miyano

  2. Signal Processing Lab, School of Engineering, Griffith University, Nathan, Australia

    Kuldip K. Paliwal

  3. School of Engineering and Physics, University of the South Pacific, Suva, Fiji

    Alok Sharma

Authors
  1. Alok Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kuldip K. Paliwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Seiya Imoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Satoru Miyano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alok Sharma.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sharma, A., Paliwal, K.K., Imoto, S. et al. Principal component analysis using QR decomposition. Int. J. Mach. Learn. & Cyber. 4, 679–683 (2013). https://doi.org/10.1007/s13042-012-0131-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-012-0131-7

Keywords

Search

Navigation