Abstract
We present a range of new incremental (single-pass streaming) algorithms for incremental principal components analysis (IPCA) and show that they are more effective than exiting ones. IPCA algorithms process the columns of a matrix A one at a time and attempt to build a basis for a low-dimensional subspace that spans the dominant subspace of A. We present a unified framework for IPCA algorithms, show that many existing ones are parameterizations of it, propose new sophisticated algorithms, and show that both the new algorithms and many existing ones can be implemented more efficiently than was previously known. We also show that many existing algorithms can fail even in easy cases and we show experimentally that our new algorithms outperform existing ones.
This research is supported by grants 965/15 and 863/15 from the Israel Science Foundation (founded by the Israel Academy of Sciences and Humanities) and by a grant from the United States-Israel Bi-national Science Foundation (BSF).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Brand, M.: Incremental singular value decomposition of uncertain data with missing values. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 707–720. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-47969-4_47
Brand, M.: Fast low-rank modifications of the thin singular value decomposition. Linear Algebra Appl. 415(1), 20–30 (2006)
Chahlaoui, Y., Gallivan, K.A., Dooren, P.V.: An incremental method for computing dominant singular spaces. In: Computational Information Retrieval, pp. 53–62 (2001)
Chahlaoui, Y., Gallivan, K.A., Van Dooren, P.: Recursive calculation of dominant singular subspaces. SIAM J. Matrix Anal. Appl. 25(2), 445–463 (2003)
Chandrasekaran, S., Manjunath, B., Wang, Y.-F., Winkeler, J., Zhang, H.: An eigenspace update algorithm for image analysis. Graph. Models Image Process. 59(5), 321–332 (1997)
Desai, A., Ghashami, M., Phillips, J.M.: Improved practical matrix sketching with guarantees. IEEE Trans. Knowl. Data Eng. 28(7), 1678–1690 (2016)
Ghashami, M., Liberty, E., Phillips, J.M., Woodruff, D.P.: Frequent directions: simple and deterministic matrix sketching. SIAM J. Comput. 45(5), 1762–1792 (2016)
Halpern, T.: Fast and robust algorithms for large-scale streaming PCA. Master’s thesis, Tel Aviv University, July 2017. http://www.tau.ac.il/~stoledo/Pubs/MSc_Tal_Halpern.pdf
Levey, A., Lindenbaum, M.: Sequential Karhunen-Loeve basis extraction and its application to images. IEEE Trans. Image Process. 9(8), 1371–1374 (2000)
Liberty, E.: Simple and deterministic matrix sketching. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 581–588. ACM (2013)
Manjunath, B., Chandrasekaran, S., Wang, Y.-F.: An eigenspace update algorithm for image analysis. In: Proceedings International Symposium on Computer Vision, pp. 551–556. IEEE (1995)
O’Brien, G.W.: Information management tools for updating an SVD-encoded indexing scheme. Master’s thesis, University of Tennessee, Knoxville (1994)
Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Sci. 290(5500), 2323–2326 (2000)
Tenenbaum, J.B., De Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Sci. 290(5500), 2319–2323 (2000)
Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The Caltech-UCSD Birds-200-2011 Dataset. Technical report CNS-TR-2011-001, California Institute of Technology (2011). http://www.vision.caltech.edu/visipedia/CUB-200-2011.html
Zha, H., Simon, H.D.: On updating problems in latent semantic indexing. SIAM J. Sci. Comput. 21(2), 782–791 (1999)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Halpern, T., Toledo, S. (2018). Advances in Incremental PCA Algorithms. In: Wyrzykowski, R., Dongarra, J., Deelman, E., Karczewski, K. (eds) Parallel Processing and Applied Mathematics. PPAM 2017. Lecture Notes in Computer Science(), vol 10777. Springer, Cham. https://doi.org/10.1007/978-3-319-78024-5_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-78024-5_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-78023-8
Online ISBN: 978-3-319-78024-5
eBook Packages: Computer ScienceComputer Science (R0)