Skip to main content
SpringerLink
Log in

Upper bounds for convergence rates of acceleration methods with initial iterations

  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

GMRES(n,k), a version of GMRES for the solution of large sparse linear systems, is introduced. A cycle of GMRES(n,k) consists of n Richardson iterations followed by k iterations of GMRES. Such cycles can be repeated until convergence is achieved. The advantage in this approach is in the opportunity to use moderate k, which results in time and memory saving. Because the number of inner products among the vectors of iteration is about k2/2, using a moderate k is particularly attractive on message-passing parallel architectures, where inner products require expensive global communication. The present analysis provides tight upper bounds for the convergence rates of GMRES(n,k) for problems with diagonalizable coefficient matrices whose spectra lie in an ellipse in 0. The advantage of GMRES(n,k) over GMRES(k) is illustrated numerically.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. M. Abramowitz and I.A. Stegun, Handbook of Mathematical Functions, National Bureau of Standards, Applied Mathematics Series 55 (Government Printing Office, Washington, DC, 1964).

    Google Scholar 

  2. W.E. Arnoldi, The principle of minimized iterations in the solution of the matrix eigenvalue problem, Quart. Appl. Math. 9 (1951) 17-29.

    Google Scholar 

  3. O. Axelsson, Conjugate gradient type methods for unsymmetric and inconsistent systems of linear equations, Linear Algebra Appl. 29 (1980) 1-16.

    Google Scholar 

  4. J. Baglama, D. Calvetti, G.H. Golub and L. Reichel, Technical Report SCCN-96-15, Stanford University, Stanford, CA (1996).

    Google Scholar 

  5. A. Brandt and I. Yavneh, Accelerated multigrid convergence and high Reynolds recirculating flows, SIAM J. Sci. Statist. Comput. 14 (1993) 607-626.

    Google Scholar 

  6. S. Cabay and L.W. Jackson, A polynomial extrapolation method for finding limits and antilimits of vector sequences, SIAM J. Numer. Anal. 13 (1976) 734-752.

    Google Scholar 

  7. E.W. Cheney, Introduction to Approximation Theory, 2nd ed. (Chelsea, New York, 1982).

    Google Scholar 

  8. P. Concus and G.H. Golub, A generalized conjugate gradient method for nonsymmetric systems of linear equations, in: Proc. of the 2nd Internat. Symp. on Computing Methods in Applied Sciences and Engineering, eds. R. Glowinski and J.L. Lions, IRIA, Paris (December 1975), Lecture Notes in Economics and Mathematical Systems 134 (Springer, Berlin, 1976) pp. 56-65.

    Google Scholar 

  9. E. De Sturler, Nested Krylov methods based on GCR, J. Comput. Appl. Math. 67 (1996) 15-41.

    Google Scholar 

  10. R.P. Eddy, Extrapolating to the limit of a vector sequence, in: Information Linkage between Applied Mathematics and Industry, ed. P.C.C. Wang (Academic Press, New York, 1979) pp. 387-396.

    Google Scholar 

  11. S.C. Eisenstat, H.C. Elman and M.H. Schultz, Variational iterative methods for nonsymmetric systems of linear equations, SIAM J. Numer. Anal. 20 (1983) 345-357.

    Google Scholar 

  12. R. Freund and S. Ruscheweyh, On a class of Chebyshev approximation problems which arise in connection with a conjugate gradient type method, Numer. Math. 48 (1986) 525-542.

    Google Scholar 

  13. W. Gander, G.H. Golub and D. Gruntz, Solving linear equations by extrapolation, Manuscript NA-89-11, Stanford University, Stanford, CA (October 1989).

    Google Scholar 

  14. M. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. N.B.S. 49 (1952) 409-436.

    Google Scholar 

  15. S. Kaniel and J. Stein, Least-square acceleration of iterative methods for linear equations, J. Optim. Theory Appl. 14 (1974) 431-437.

    Google Scholar 

  16. S.A. Kharchenko and A.Y. Yeremin, Eigenvalue translation based preconditioners for the GMRES(k) method, Numer. Linear Algebra Appl. 2 (1995) 51-77.

    Google Scholar 

  17. G.G. Lorentz, Approximation by incomplete polynomials (problems and results), in: Padé and Rational Approximations: Theory and Applications, eds. E.B. Saff and R.S. Varga (Academic Press, New York, 1977) pp. 289-302.

    Google Scholar 

  18. M. Mešina, Convergence acceleration for the iterative solution of the equations X = AX + f, Comput. Methods Appl. Mech. Engrg. 10 (1977) 165-173.

    Google Scholar 

  19. R.B. Morgan, A restarted GMRES method augmented with eigenvectors, SIAM J. Matrix Anal. Appl. 16 (1995) 1154-1171.

    Google Scholar 

  20. Y. Saad, Krylov subspace methods for solving large unsymmetric linear systems, Math. Comp. 37 (1981) 105-126.

    Google Scholar 

  21. Y. Saad and M.H. Schultz, A generalized minimal residual algorithm for solving nonsymmetric linear systems, SIAM J. Sci. Statist. Comput. 7 (1986) 856-869.

    Google Scholar 

  22. E.B. Saff and R.S. Varga, On incomplete polynomials, in: Numerische Methoden der Approximationstheorie, Band 4, International Series of Numerical Mathematics, Vol. 42, eds. L. Collatz, G. Meinardus and H. Werner (Birkhäuser, Basel, 1978) pp. 281-298.

    Google Scholar 

  23. E.B. Saff and R.S. Varga, Uniform approximation by incomplete polynomials, Internat. J. Math. Math. Sci. 1 (1978) 407-420.

    Google Scholar 

  24. E.B. Saff and R.S. Varga, The sharpness of Lorentz’s theorem on incomplete polynomials, Trans. Amer. Math. Soc. 249 (1979) 163-186.

    Google Scholar 

  25. E.B. Saff and R.S. Varga, Incomplete polynomials II, Pacific J. Math. 92 (1981) 161-172.

    Google Scholar 

  26. A. Sidi, Convergence and stability properties of minimal polynomial and reduced rank extrapolation algorithms, SIAM J. Numer. Anal. 23 (1986) 197-209.

    Google Scholar 

  27. A. Sidi, Extrapolation vs. projection methods for linear systems of equations, J. Comput. Appl. Math. 22 (1988) 71-88.

    Google Scholar 

  28. A. Sidi, Efficient implementation of minimal polynomial and reduced rank extrapolation methods, J. Comput. Appl. Math. 36 (1991) 305-337.

    Google Scholar 

  29. A. Sidi, Convergence of intermediate rows of minimal polynomial and reduced rank extrapolation tables, Numer. Algorithms 6 (1994) 229-244.

    Google Scholar 

  30. A. Sidi and J. Bridger, Convergence and stability analyses for some vector extrapolation methods in the presence of defective iteration matrices, J. Comput. Appl. Math. 22 (1988) 35-61.

    Google Scholar 

  31. A. Sidi and Y. Shapira, Upper bounds for convergence rates of vector extrapolation methods on linear systems with initial iterations, TR # 701, Computer Science Department, Technion (December 1991). Also NASA TM 105608, ICOMP-92-09 (July 1992). For an extended abstract, see Proc. of the Cornelius Lanczos Internat. Centenary Conf., eds. J.D. Brown, M.T. Chu, D.C. Ellison and R.J. Plemmons (1994) pp. 285-287.

    Google Scholar 

  32. Y. Shapira, Convergence criteria for diagonalizable non-normal equations, Technical Report LAUR-98-1993, Los Alamos National Laboratory (1998).

  33. E.L. Stiefel, Relaxationsmethoden bester Strategie zur Lösung linearer Gleichungssystems, Comment. Math. Helv. 29 (1955) 157-179.

    Google Scholar 

  34. G. Szegö, Orthogonal Polynomials, Vol. 23 (Amer. Math. Soc., Providence, RI, 1939).

    Google Scholar 

  35. R.S. Varga, Matrix Iterative Analysis (Prentice-Hall, Englewood Cliffs, NJ, 1962).

    Google Scholar 

  36. P.K.W. Vinsome, Orthomin, an iterative method for solving sparse sets of simultaneous linear equations, in: Proc. 4th Symp. on Reservoir Simulation (Society of Petroleum Engineers of AIME, 1976) pp. 149-159.

  37. O. Widlund, A Lanczos method for a class of nonsymmetric systems of linear equations, SIAM J. Numer. Anal. 15 (1978) 801-812.

    Google Scholar 

  38. D.M. Young and K.C. Jea, Generalized conjugate gradient acceleration of nonsymmetrizable iterative methods, Linear Algebra Appl. 34 (1980) 159-194.

    Google Scholar 

Download references

Author information

Author notes

  1. Avram Sidi

    Present address: Institute for Computational Mechanics in Propulsion, NASA – Lewis Research Center, Cleveland, OH, 44135, USA

Authors and Affiliations

  1. Computer Science Department, Technion – Israel Institute of Technology, Haifa, 32000, Israel and

    Avram Sidi

  2. Mathematics Department, Technion –, Israel Institute of Technology, Haifa, 32000, Israel

    Yair Shapira

Authors
  1. Avram Sidi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yair Shapira
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sidi, A., Shapira, Y. Upper bounds for convergence rates of acceleration methods with initial iterations. Numerical Algorithms 18, 113–132 (1998). https://doi.org/10.1023/A:1019113314010

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1019113314010

Keywords

Search

Navigation