Skip to main content
SpringerLink
Log in
Book cover

International Conference on Numerical Methods and Applications

NMA 2010: Numerical Methods and Applications pp 14–28Cite as

Stochastic Algorithms in Linear Algebra - beyond the Markov Chains and von Neumann - Ulam Scheme

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 6046))

Abstract

Sparsified Randomization Monte Carlo (SRMC) algorithms for solving systems of linear algebraic equations introduced in our previous paper [34] are discussed here in a broader context. In particular, I present new randomized solvers for large systems of linear equations, randomized singular value (SVD) decomposition for large matrices and their use for solving inverse problems, and stochastic simulation of random fields. Stochastic projection methods, which I call here “random row action” algorithms, are extended to problems which involve systems of equations and constrains in the form of systems of linear inequalities.

The author thanks the organizers of the conference, and acknowledges the support of the RFBR under Grants N 06-01-00498, 09-01-12028-ofi-m, and a joint BMBF and Bortnik Funds Grant N 7326.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Achlioptas, D., McSherry, F.: Fast computation of low rank matrix approximations. In: Proceedings of the 33rd Annual Symposium on Theory of Computing (2001)

    Google Scholar 

  2. Ailon, N., Chazelle, B.: The fast JohnsonLinderstrauss transform and approximate nearest neighbors. SIAM J. Comput. 39(1), 302–322 (2009)

    Article  MATH  Google Scholar 

  3. Belongie, S., Fowlkes, C., Chung, F., Malik, J.: Spectral Partitioning with Indefinite Kernels Using the Nystreom Extension. In: Heyden, A., et al. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 531–542. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  4. Beylkin, G., Mohlenkam, M.J.: Algorithms for numerical analysis in high dimension. SIAM Journal on Scientific Computing 26(6), 2133–2159 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  5. Dimov, I., Philippe, B., Karaivanova, A., Weihrauch, C.: Robustness and Applicability of Markov Chain Monte Carlo Algorithms for Eigenvalue Problem. Journal of Applied Mathematical Modelling 32, 1511–1529 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  6. Dimov, I., Alexandrov, V., Papancheva, R., Weihrauch, C.: Monte Carlo Numerical Treatment of Large Linear Algebra Problems. In: Shi, Y., et al. (eds.) ICCS 2007. LNCS, vol. 4487, pp. 747–754. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  7. Dimov, I.T.: Monte Carlo Methods for Applied Scientists, p. 291. World Scientific, Singapore (2008)

    MATH  Google Scholar 

  8. Drineas, P., Frieze, A., Kannan, R., Vempala, S., Vinay, V.: Clustering Large Graphs via the Singular Value Decomposition. Machine Learning 56(13), 9–33 (2004)

    Article  MATH  Google Scholar 

  9. Drineas, P., Kannan, R.: Pass Efficient Algorithms for Approximating Large Matrices. In: Proceedings of the 14th Annual Symposium on Discrete Algorithms (Baltimore, MD), pp. 223–232 (2003)

    Google Scholar 

  10. Drineas, P., Drinea, E., Huggins, P.S.: An experimental evaluation of a Monte Carlo algorithm for singular value decomposition. In: Manolopoulos, Y., Evripidou, S., Kakas, A.C. (eds.) PCI 2001. LNCS, vol. 2563, pp. 279–296. Springer, Heidelberg (2003) ISSN 0302-9743

    Chapter  Google Scholar 

  11. Drineas, P., Kannan, R.: Fast Monte Carlo algorithms for approximate matrix multiplication. In: Proceedings of the 42nd IEEE Symposium on Foundations of Computer Science, p. 452 (2001) ISBN: 0-7695-1390-5

    Google Scholar 

  12. Drineas, P., Kannan, R., Mahoney, M.W.: Fast Monte Carlo algorithms for matrices I: approximating matrix multiplication. SIAM J. Comput. 36(1), 132–157 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  13. Eberly, W., Kaltofen, E.: On Randomized Lanczos Algorithms. In: International Conference on Symbolic and Algebraic Computation Archive Proceedings of the 1997 International Symposium on Symbolic and Algebraic Computation, pp. 176–183 (1997)

    Google Scholar 

  14. Eckhart, C., Young, G.: A principal axis transformation for non-Hermitian matrices. Bulletin of the American Mathematical Siciety 45, 118–121 (1939)

    Article  MathSciNet  MATH  Google Scholar 

  15. Ermakov, S.M., Mikhailov, G.A.: Statistical modeling. Nauka, Moscow (1982) (in Russian)

    Google Scholar 

  16. Ermakov, S.M.: Monte Carlo Method in Computational Mathematics. An Introductory course. BINOM publisher, St. Pitersburg (2009) (in Russian)

    Google Scholar 

  17. Ermakov, S.M., Sipin, A.S.: A new Monte Carlo scheme for solving problems of mathematical physics. Soviet Dokl. 285(3) (1985) (Russian)

    Google Scholar 

  18. Frieze, A., Kannan, R., Vempala, S.: Fast Monte Carlo algorithms for finding low-rank approximations. J. ACM 51( 6), 1025–1041 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  19. Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. Johns Hopkins University Press, Baltimore (1996)

    MATH  Google Scholar 

  20. Hammersley, J.M., Handscomb, D.C.: Monte Carlo Methods. Chapman and Hall, London (1964)

    Book  MATH  Google Scholar 

  21. Johnson, W.B., Lindenstrauss, J.: Extensions of Lipschitz maps into a Hilbert space. Contemp. Math. 26, 189–206 (1984)

    Article  MATH  Google Scholar 

  22. Kaczmarz, S.: Angenaeherte Aufloesung von Systemen linearer Gleichungen. Bull. Acad. Polon. Sciences et Lettres, A, 355–357 (1937)

    Google Scholar 

  23. Kobayashi, M., Dupret, G., King, O., Samukawa, H.: Estimation of singular values of very large matrices using random sampling. Computers and Mathematics with Applications 42, 1331–1352 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  24. Lanczos, C.: An iteration method for the solution of the eigenvalue problem of linear differential and integral operators. Journal of Research of the National Bureau of Standards 45(4), 255–282 (1950)

    Article  MathSciNet  Google Scholar 

  25. Liberty, E., Woolfe, F., Martinsson, P.-G., Rokhlin, V., Tygert, M.: Randomized algorithms for the low-rank approximation of matrices. Yale Dept. of Computer Science Technical Report 1388

    Google Scholar 

  26. Muller, N., Magaia, L., Herbst, B.M.: Singular Value Decomposition, Eigenfaces, and 3D Reconstructions. SIAM Review 46(3), 518–545 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  27. Phoon, K.K., Huang, H.W., Quek, S.T.: Simulation of strongly non-Gaussian processes using Karhunen-Loeve expansion. Probabilistic engineering Mechanics 20, 188–198 (2005)

    Article  Google Scholar 

  28. Rokhlin, V.: Rapid solution of integral equations of classical potential theory. J. Comp. Phys. 60, 187–207 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  29. Rokhlin, V., Szlam, A., Tygert, M.: A randomized algorithm for principal component analysis. SIAM J. Matrix Anal. Appl., arxiv.org (2009)

    Google Scholar 

  30. Expansion of random boundary excitations for some elliptic PDEs. Monte Carlo Methods and Applications 13(5-6), 403–451 (2007)

    Google Scholar 

  31. Sabelfeld, K.K.: Monte Carlo Methods in Boundary Value Problems. Springer, Heidelberg (1991)

    Google Scholar 

  32. Sabelfeld, K.K., Simonov, N.A.: Random Walks on Boundary for Solving PDEs. VSP, The Netherlands, Utrecht (1994)

    Google Scholar 

  33. Sabelfeld, K., Loshina, N.: Fast stochastic iterative projection methods for very large linear systems. In: Seventh IMACS Seminar on Monte Carlo Methods (MCM 2009), Brussels, September 6-11 (2009)

    Google Scholar 

  34. Sabelfeld, K., Mozartova, N.: Sparsified Randomization Algorithms for large systems of linear equations and a new version of the Random Walk on Boundary method. Monte Carlo Methods and Applications 15(3), 257–284 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  35. Sabelfeld, K., Mozartova, N.: Sparsified Randomization Algorithms for low rank approximations and applications to integral equations and inhomogeneous random field simulation. Mathematics and Computers in Simulation (2010) (submitted)

    Google Scholar 

  36. Sabelfeld, K., Shalimova, I., Levykin, A.: Random Walk on Fixed Spheres for Laplace and Lamé equations. Monte Carlo Methods and Applications 12(1), 55–93 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  37. Sobol, I.M.: Numerical Monte Carlo Methods. Nauka, Moscow (1973) (in Russian)

    Google Scholar 

  38. Strang, G.: The fundamental Theorem of linear algebra. The American Mathematical Monthly 100(9), 848–855 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  39. Strohmer, T., Vershynin, R.: A randomized Kaczmarz algorithm with exponential convergence. Journal of Fourier Analysis and Applications 15, 262–278 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  40. Stewart, G.W.: On the Early History of the Singular Value Decomposition. SIAM Review 35(4) (1993)

    Google Scholar 

  41. Vempala, S.S.: The Random projection method. AMS (2004)

    Google Scholar 

  42. Vorobiev, Ju.V.: Stochastic iteration process. J. Comp. Math. and Math. Physics 4(6), 5(5), 1088–1092, 787-795 (1964) (in Russian)

    Google Scholar 

  43. Walker, A.J.: New fast method for generating discrete random numbers with arbitrary friquency distributions. Electronic Letters 10, 127–128 (1974)

    Article  Google Scholar 

  44. Woolfe, F., Liberty, E., Rokhlin, V., Tygert, M.: A fast randomized algorithm for the approximation of matrices. Applied and Computational Harmonic Analysis 25, 335–366 (2008)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute Comp. Math. & Math. Geoph., Novosibirsk, Lavrentiev str, 6, 630090, Novosibirsk, Russia

    Karl Sabelfeld

Authors
  1. Karl Sabelfeld
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute of Computer and Communication Technologies, Bulgarian Academy of Sciences, Acad. G. Bonchev 25 A, 1113, Sofia, Bulgaria

    Ivan Dimov

  2. Faculty of Mathematics and Informatics, Department Numerical Methods and Algorithms, University of Sofia "St. Kliment Ohridski",, blvd. James Bourchier 5, 1164, Sofia, Bulgaria

    Stefka Dimova

  3. Institute of Mathematics and Informatics, Bulgarian Academy of Sciences, Acad. Bonchev St.,Bl.8, 1113, Sofia, Bulgaria

    Natalia Kolkovska

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Sabelfeld, K. (2011). Stochastic Algorithms in Linear Algebra - beyond the Markov Chains and von Neumann - Ulam Scheme. In: Dimov, I., Dimova, S., Kolkovska, N. (eds) Numerical Methods and Applications. NMA 2010. Lecture Notes in Computer Science, vol 6046. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-18466-6_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-18466-6_2

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-18465-9

  • Online ISBN: 978-3-642-18466-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation