Abstract
We capitalize upon the known relationship between pairs of orthogonal and minimal residual methods (or, biorthogonal and quasi-minimal residual methods) in order to estimate how much smaller the residuals or quasi-residuals of the minimizing methods can be compared to those of the corresponding Galerkin or Petrov–Galerkin method. Examples of such pairs are the conjugate gradient (CG) and the conjugate residual (CR) methods, the full orthogonalization method (FOM) and the generalized minimal residual (GMRES) method, the CGNE and BiCG versions of applying CG to the normal equations, as well as the biconjugate gradient (BiCG) and the quasi-minimal residual (QMR) methods. Also the pairs consisting of the (bi)conjugate gradient squared (CGS) and the transpose-free QMR (TFQMR) methods can be added to this list if the residuals at half-steps are included, and further examples can be created easily.
The analysis is more generally applicable to the minimal residual (MR) and quasi-minimal residual (QMR) smoothing processes, which are known to provide the transition from the results of the first method of such a pair to those of the second one. By an interpretation of these smoothing processes in coordinate space we deepen the understanding of some of the underlying relationships and introduce a unifying framework for minimal residual and quasi-minimal residual smoothing. This framework includes the general notion of QMR-type methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
P.N. Brown, A theoretical comparison of the Arnoldi and GMRES algorithms, SIAM J. Sci. Statist. Comput. 12 (1991) 58–78.
T.F. Chan, E. Gallopoulos, V. Simoncini, T. Szeto and C.H. Tong, A quasi-minimal residual variant of the Bi-CGSTAB algorithm for nonsymmetric systems, SIAM J. Sci. Comput. 15 (1994) 338–347.
J. Cullum, Peaks, plateaus, numerical instabilities in a Galerkin/minimal residual pair of methods for solving Ax = b, Appl. Numer. Math. 19 (1995) 255–278.
J. Cullum and A. Greenbaum, Relations between Galerkin and norm-minimizing iterative methods for solving linear systems, SIAM J. Matrix Anal. Appl. 17 (1996) 223–247.
M. Eiermann and O. Ernst, Geometric aspects in the theory of Krylov space methods, Acta Numerica 10 (2001) 251–312.
R. Fletcher, Conjugate gradient methods for indefinite systems, in: Numerical Analysis, ed. G.A. Watson, Dundee, 1975, Lecture Notes in Mathematics, Vol. 506 (Springer, Berlin, 1976) pp. 73-89.
R.W. Freund, A transpose-free quasi-minimal residual algorithm for non-Hermitian linear systems, SIAM J. Sci. Comput. 14 (1993) 470–482.
R.W. Freund, G.H. Golub and N.M. Nachtigal, Iterative solution of linear systems, Acta Numerica (1992) 57–100.
R.W. Freund, M.H. Gutknecht and N.M. Nachtigal, An implementation of the look-ahead Lanczos algorithm for non-Hermitian matrices, SIAM J. Sci. Comput. 14 (1993) 137–158.
R.W. Freund and N.M. Nachtigal, QMR: A quasi-minimal residual method for non-Hermitian linear systems, Numer. Math. 60 (1991) 315–339.
A. Greenbaum, Estimating the attainable accuracy of recursively computed residual methods, SIAM J. Matrix Anal. Appl. 18 (1997) 535–551.
M.H. Gutknecht, A completed theory of the unsymmetric Lanczos process and related algorithms, Part I, SIAM J. Matrix Anal. Appl. 13 (1992) 594–639.
M.H. Gutknecht, Changing the norm in conjugate gradient type algorithms, SIAM J. Numer. Anal. 30 (1993) 40–56.
M.H. Gutknecht, A completed theory of the unsymmetric Lanczos process and related algorithms, Part II, SIAM J. Matrix Anal. Appl. 15 (1994) 15–58.
M.R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. National. Bureau Standards 49 (1952) 409–435.
C. Lanczos, Solution of systems of linear equations by minimized iterations, J. Res. Nat. Bureau Standards 49 (1952) 33–53.
C.C. Paige and M.A. Saunders, Solution of sparse indefinite systems of linear equations, SIAM J. Numer. Anal. 12 (1975) 617–629.
Y. Saad, Iterative Methods for Sparse Linear Systems (PWS, Boston, 1996).
Y. Saad and M.H. Schultz, Conjugate gradient-like algorithms for solving nonsymmetric linear systems, Math. Comp. 44 (1985) 417–424.
H. Sadok, Analysis of the convergence of the minimal and the orthogonal residual methods, Preprint (1997).
W. Schönauer, Scientific Computing on Vector Computers (Elsevier, Amsterdam, 1987).
P. Sonneveld, CGS, a fast Lanczos-type solver for nonsymmetric linear systems, SIAM J. Sci. Statist. Comput. 10 (1989) 36–52.
E. Stiefel, Relaxationsmethoden bester Strategie zur Lösung linearer Gleichungssysteme, Comm. Math. Helv. 29 (1955) 157–179.
H.A. van der Vorst, Bi-CGSTAB: A fast and smoothly converging variant of Bi-CG for the solution of nonsymmetric linear systems, SIAM J. Sci. Statist. Comput. 13 (1992) 631–644.
H.F. Walker, Residual smoothing and peak/plateau behavior in Krylov subspace methods, Appl. Numer. Math. 19 (1995) 279–286.
R. Weiss, Convergence behavior of generalized conjugate gradient methods, PhD thesis, University of Karlsruhe (1990).
R. Weiss, Properties of generalized conjugate gradient methods, J. Numer. Linear Algebra Appl. 1 (1994) 45–63.
R. Weiss, Parameter-Free Iterative Linear Solvers, Mathematical Research, Vol. 97 (Akademie Verlag, Berlin, 1996).
L. Zhou and H.F. Walker, Residual smoothing techniques for iterative methods, SIAMJ. Sci. Comput. 15 (1994) 297–312.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Gutknecht, M.H., Rozložník, M. By How Much Can Residual Minimization Accelerate the Convergence of Orthogonal Residual Methods?. Numerical Algorithms 27, 189–213 (2001). https://doi.org/10.1023/A:1011889705659
Issue Date:
DOI: https://doi.org/10.1023/A:1011889705659