skip to main content
review-article
Free access

RandNLA: randomized numerical linear algebra

Published: 23 May 2016 Publication History

Abstract

Randomization offers new benefits for large-scale linear algebra computations.

References

[1]
Achlioptas, D., Karnin, Z., Liberty, E. Near-optimal entrywise sampling for data matrices. In Annual Advances in Neural Information Processing Systems 26: Proceedings of the 2013 Conference, 2013.
[2]
Achlioptas, D., McSherry, F. Fast computation of low-rank matrix approximations. J. ACM 54, 2 (2007), Article 9.
[3]
Ailon, N., Chazelle, B. Faster dimension reduction. Commun. ACM 53, 2 (2010), 97--104.
[4]
Avron, H., Maymounkov, P., Toledo, S. Blendenpik: Supercharging LAPACK's least-squares solver. SIAM J. Sci. Comput. 32 (2010), 1217--1236.
[5]
Batson, J., Spielman, D.A., Srivastava, N., Teng, S.-H. Spectral sparsification of graphs: Theory and algorithms. Commun. ACM 56, 8 (2013), 87--94.
[6]
Belkin, M., Niyogi, P. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15, 6 (2003), 1373--1396.
[7]
Boutsidis, C., Mahoney, M.W., Drineas, P. An improved approximation algorithm for the column subset selection problem. In Proceedings of the 20th Annual ACM-SIAM Symposium on Discrete Algorithms (2009), 968--977.
[8]
Candes, E.J., Recht, B. Exact matrix completion via convex optimization. Commun. ACM 55, 6 (2012), 111--119.
[9]
Chatterjee, S., Hadi, A.S. Influential observations, high leverage points, and outliers in linear regression. Stat. Sci. 1, 3 (1986), 379--393.
[10]
Chen, Y., Bhojanapalli, S., Sanghavi, S., Ward, R. Coherent matrix completion. In Proceedings of the 31st International Conference on Machine Learning (2014), 674--682.
[11]
Clarkson, K. Subgradient and sampling algorithms for l<sup>1</sup> regression. In Proceedings of the 16th Annual ACM-SIAM Symposium on Discrete Algorithms (2005), 257--266.
[12]
Clarkson, K.L., Woodruff, D.P. Low rank approximation and regression in input sparsity time. In Proceedings of the 45th Annual ACM Symposium on Theory of Computing (2013), 81--90.
[13]
Drineas, P., Kannan, R., Mahoney, M.W. Fast Monte Carlo algorithms for matrices I: approximating matrix multiplication. SIAM J. Comput. 36 (2006), 132--157.
[14]
Drineas, P., Magdon-Ismail, M., Mahoney, M.W., Woodruff, D.P. Fast approximation of matrix coherence and statistical leverage. J. Mach. Learn. Res. 13 (2012), 3475--3506.
[15]
Drineas, P., Mahoney, M.W., Muthukrishnan, S. Sampling algorithms for l<sup>2</sup> regression and applications. In Proceedings of the 17th Annual ACM-SIAM Symposium on Discrete Algorithms (2006), 1127--1136.
[16]
Drineas, P., Mahoney, M.W., Muthukrishnan, S. Relative-error CUR matrix decompositions. SIAM J. Matrix Anal. Appl. 30 (2008), 844--881.
[17]
Drineas, P., Mahoney, M.W., Muthukrishnan, S., Sarlós, T. Faster least squares approximation. Numer. Math. 117, 2 (2010), 219--249.
[18]
Drineas, P., Zouzias, A. A note on element-wise matrix sparsification via a matrix-valued Bernstein inequality. Inform. Process. Lett. 111 (2011), 385--389.
[19]
Frieze, A., Kannan, R., Vempala, S. Fast Monte-Carlo algorithms for finding low-rank approximations. J. ACM 51, 6 (2004), 1025--1041.
[20]
Füredi, Z., Komlós, J. The eigenvalues of random symmetric matrices. Combinatorica 1, 3 (1981), 233--241.
[21]
Gittens, A. Mahoney, M.W. Revisiting the Nyström method for improved large-scale machine learning. J. Mach. Learn Res. In press.
[22]
Golub, G.H., Van Loan, C.F. Matrix Computations. Johns Hopkins University Press, Baltimore, 1996.
[23]
Gross, D. Recovering low-rank matrices from few coefficients in any basis. IEEE Trans. Inform. Theory 57, 3 (2011), 1548--1566.
[24]
Gu, M. Subspace iteration randomization and singular value problems. Technical report, 2014. Preprint: arXiv:1408.2208.
[25]
Halko, N., Martinsson, P.-G., Tropp, J.A. Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM Rev. 53, 2 (2011), 217--288.
[26]
Koren, Y., Bell, R., Volinsky, C. Matrix factorization techniques for recommender systems. IEEE Comp. 42, 8 (2009), 30--37.
[27]
Koutis, I., Miller, G.L., Peng, R. A fast solver for a class of linear systems. Commun. ACM 55, 10 (2012), 99--107.
[28]
Kundu, A., Drineas, P. A note on randomized elementwise matrix sparsification. Technical report, 2014. Preprint: arXiv:1404.0320.
[29]
Le, Q.V., Sarlós, T., Smola, A.J. Fastfood---approximating kernel expansions in loglinear time. In Proceedings of the 30th International Conference on Machine Learning, 2013.
[30]
Ma, P., Mahoney, M.W., Yu, B. A statistical perspective on algorithmic leveraging. J. Mach. Learn. Res. 16 (2015), 861--911.
[31]
Mackey, L., Talwalkar, A., Jordan, M.I. Distributed matrix completion and robust factorization. J. Mach. Learn. Res. 16 (2015), 913--960.
[32]
Mahoney, M.W. Randomized Algorithms for Matrices and Data. Foundations and Trends in Machine Learning. NOW Publishers, Boston, 2011.
[33]
Mahoney, M.W., Drineas, P. CUR matrix decompositions for improved data analysis. Proc. Natl. Acad. Sci. USA 106 (2009), 697--702.
[34]
Meng, X., Mahoney, M.W. Low-distortion subspace embeddings in input-sparsity time and applications to robust linear regression. In Proceedings of the 45th Annual ACM Symposium on Theory of Computing (2013), 91--100.
[35]
Meng, X., Saunders, M.A., Mahoney, M.W. LSRN: A parallel iterative solver for strongly over- or underdetermined systems. SIAM J. Sci. Comput. 36, 2 (2014), C95--C118.
[36]
Nelson, J., Huy, N.L. OSNAP: Faster numerical linear algebra algorithms via sparser subspace embeddings. In Proceedings of the 54th Annual IEEE Symposium on Foundations of Computer Science (2013), 117--126.
[37]
Oliveira, R.I. Sums of random Hermitian matrices and an inequality by Rudelson. Electron. Commun. Prob. 15 (2010) 203--212.
[38]
Paschou, P., Ziv, E., Burchard, E.G., Choudhry, S., Rodriguez-Cintron, W., Mahoney, M.W., Drineas, P. PCA-correlated SNPs for structure identification in worldwide human populations. PLoS Genet. 3 (2007), 1672--1686.
[39]
Rahimi, A., Recht, B. Random features for large-scale kernel machines. In Annual Advances in Neural Information Processing Systems 20: Proceedings of the 2007 Conference, 2008.
[40]
Recht, B. A simpler approach to matrix completion. J. Mach. Learn. Res. 12 (2011), 3413--3430.
[41]
Rokhlin, V., Szlam, A., Tygert, M. A randomized algorithm for principal component analysis. SIAM J. Matrix Anal. Appl. 31, 3 (2009), 1100--1124.
[42]
Rudelson, M., Vershynin, R. Sampling from large matrices: an approach through geometric functional analysis. J. ACM 54, 4 (2007), Article 21.
[43]
Sarlós, T., Improved approximation algorithms for large matrices via random projections. In Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science (2006), 143--152.
[44]
Smale, S. Some remarks on the foundations of numerical analysis. SIAM Rev. 32, 2 (1990), 211--220.
[45]
Spielman, D.A., Srivastava, N. Graph sparsification by effective resistances. SIAM J. Comput. 40, 6 (2011), 1913--1926.
[46]
Stigler, S.M. The History of Statistics: The Measurement of Uncertainty before 1900. Harvard University Press, Cambridge, 1986.
[47]
Tropp, J.A. User-friendly tail bounds for sums of random matrices. Found. Comput. Math. 12, 4 (2012), 389--434.
[48]
Turing, A.M. Rounding-off errors in matrix processes. Quart. J. Mech. Appl. Math. 1 (1948), 287--308.
[49]
von Neumann, J., Goldstine, H.H. Numerical inverting of matrices of high order. Bull. Am. Math. Soc. 53 (1947), 1021--1099.
[50]
Wigner, E.P. Random matrices in physics. SIAM Rev. 9, 1 (1967), 1--23.
[51]
Woodruff, D.P. Sketching as a Tool for Numerical Linear Algebra. Foundations and Trends in Theoretical Computer Science. NOW Publishers, Boston, 2014.
[52]
Yang, J., Meng, X., Mahoney, M.W. Implementing randomized matrix algorithms in parallel and distributed environments. Proc. IEEE 104, 1 (2016), 58--92.
[53]
Yang, J., Rübel, O., Prabhat, Mahoney, M.W., Bowen, B.P. Identifying important ions and positions in mass spectrometry imaging data using CUR matrix decompositions. Anal. Chem. 87, 9 (2015), 4658--4666.
[54]
Yip, C.-W., Mahoney, M.W., Szalay, A.S., Csabai, I., Budavari, T., Wyse, R.F.G., Dobos, L. Objective identification of informative wavelength regions in galaxy spectra. Astron. J. 147, 110 (2014), 15.

Cited By

View all
  • (2025)Fast stochastic subspace identification of densely instrumented bridges using randomized SVDMechanical Systems and Signal Processing10.1016/j.ymssp.2024.112264225(112264)Online publication date: Feb-2025
  • (2025)Dominant subspace and low-rank approximations from block Krylov subspaces without a prescribed gapLinear Algebra and its Applications10.1016/j.laa.2024.11.021708(112-149)Online publication date: Mar-2025
  • (2024)A provably accurate randomized sampling algorithm for logistic regressionProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i10.29042(11597-11605)Online publication date: 20-Feb-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Communications of the ACM
Communications of the ACM  Volume 59, Issue 6
June 2016
106 pages
ISSN:0001-0782
EISSN:1557-7317
DOI:10.1145/2942427
  • Editor:
  • Moshe Y. Vardi
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 May 2016
Published in CACM Volume 59, Issue 6

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Review-article
  • Popular
  • Refereed

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1,443
  • Downloads (Last 6 weeks)128
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Fast stochastic subspace identification of densely instrumented bridges using randomized SVDMechanical Systems and Signal Processing10.1016/j.ymssp.2024.112264225(112264)Online publication date: Feb-2025
  • (2025)Dominant subspace and low-rank approximations from block Krylov subspaces without a prescribed gapLinear Algebra and its Applications10.1016/j.laa.2024.11.021708(112-149)Online publication date: Mar-2025
  • (2024)A provably accurate randomized sampling algorithm for logistic regressionProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i10.29042(11597-11605)Online publication date: 20-Feb-2024
  • (2024)Hyperdimensional computing: A fast, robust, and interpretable paradigm for biological dataPLOS Computational Biology10.1371/journal.pcbi.101242620:9(e1012426)Online publication date: 24-Sep-2024
  • (2024)Recent and Upcoming Developments in Randomized Numerical Linear Algebra for Machine LearningProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671461(6470-6479)Online publication date: 25-Aug-2024
  • (2024)Distributed-Memory Randomized Algorithms for Sparse Tensor CP DecompositionProceedings of the 36th ACM Symposium on Parallelism in Algorithms and Architectures10.1145/3626183.3659980(155-168)Online publication date: 17-Jun-2024
  • (2024)Optimal Embedding Dimension for Sparse Subspace EmbeddingsProceedings of the 56th Annual ACM Symposium on Theory of Computing10.1145/3618260.3649762(1106-1117)Online publication date: 10-Jun-2024
  • (2024)Solving Dense Linear Systems Faster Than via PreconditioningProceedings of the 56th Annual ACM Symposium on Theory of Computing10.1145/3618260.3649694(1118-1129)Online publication date: 10-Jun-2024
  • (2024)Efficient Error and Variance Estimation for Randomized Matrix ComputationsSIAM Journal on Scientific Computing10.1137/23M155853746:1(A508-A528)Online publication date: 8-Feb-2024
  • (2024)A Framework for Compressed Weighted Nonnegative Matrix FactorizationIEEE Transactions on Signal Processing10.1109/TSP.2024.346983072(4798-4811)Online publication date: 1-Jan-2024
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Digital Edition

View this article in digital edition.

Digital Edition

Magazine Site

View this article on the magazine site (external)

Magazine Site

Login options

Full Access

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media