Abstract
The numerical solution of linear elliptic partial differential equations most often involves a finite element or finite difference discretization. To preserve sparsity, the arising system is normally solved using an iterative solution method, commonly a preconditioned conjugate gradient method. Preconditioning is a crucial part of such a solution process. In order to enable the solution of very large-scale systems, it is desirable that the total computational cost will be of optimal order, i.e. proportional to the degrees of freedom of the approximation used, which also induces mesh independent convergence of the iteration. This paper surveys the equivalent operator approach, which has proven to provide an efficient general framework to construct such preconditioners. Hereby one first approximates the given differential operator by some simpler differential operator, and then chooses as preconditioner the discretization of this operator for the same mesh. In this survey we give a uniform presentation of this approach, including theoretical foundation and several practically important applications for both symmetric and nonsymmetric equations and systems, and some nonlinear examples in the context of Newton linearization.
Similar content being viewed by others
References
Antal, I., Karátson, J.: A mesh independent superlinear algorithm for some nonlinear nonsymmetric elliptic systems. Comput. Math. Appl. (2008, in press)
Ashby, S.F., Manteuffel, T.A., Saylor, P.E.: A taxonomy for conjugate gradient methods. SIAM J. Numer. Anal. 27(6), 1542–1568 (1990)
Axelsson, O.: A generalized conjugate gradient least square method. Numer. Math. 51, 209–227 (1987)
Axelsson, O.: Iterative Solution Methods. Cambridge University Press, Cambridge (1994)
Axelsson, O.: On iterative solvers in structural mechanics; separate displacement orderings and mixed variable methods. Math. Comput. Simul. 50(1–4), 11–30 (1999)
Axelsson, O., Barker, V.A., Neytcheva, M., Polman, B.: Solving the Stokes problem on a massively parallel computer. Math. Model. Anal. 6(1), 7–27 (2001)
Axelsson, O., Blaheta, R., Neytcheva, M.: Preconditioning of boundary value problems using elementwise Schur complements. Technical Report 2006-048, Department of Information Technology, Uppsala University, November (2006)
Axelsson, O., Blaheta, R., Neytcheva, M.: A black-box generalized conjugate gradient minimum residual method based on variable preconditioners and local element approximations. TR 2007-033, Institute for Information Technology, Uppsala University, December (2007)
Axelsson, O., Faragó, I., Karátson, J.: Sobolev space preconditioning for Newton’s method using domain decomposition. Numer. Linear Algebr. Appl. 9, 585–598 (2002)
Axelsson, O., Gololobov, S.V.: A combined method of local Green’s functions and central difference method for singularly perturbed convection-diffusion problems. J. Comput. Appl. Math. 161(2), 245–257 (2003)
Axelsson, O., Gustafsson, I.: Iterative methods for the solution of the Navier equations of elasticity. Comput. Methods Appl. Mech. Eng. 15(2), 241–258 (1978)
Axelsson, O., Kaporin, I.: On the sublinear and superlinear rate of convergence of conjugate gradient methods. Mathematical journey through analysis, matrix theory and scientific computation (Kent, OH, 1999). Numer. Algorithms 25(1–4), 1–22 (2000)
Axelsson, O., Karátson, J.: Symmetric part preconditioning for the conjugate gradient method in Hilbert space. Numer. Funct. Anal. 24(5–6), 455–474 (2003)
Axelsson, O., Karátson J.: Conditioning analysis of separate displacement preconditioners for some nonlinear elasticity systems. Math. Comput. Simul. 64(6), 649–668 (2004)
Axelsson, O., Karátson J.: Superlinearly convergent CG methods via equivalent preconditioning for nonsymmetric elliptic operators. Numer. Math. 99(2), 197–223 (2004)
Axelsson, O., Karátson J.: Symmetric part preconditioning of the CGM for Stokes type saddle-point systems. Numer. Funct. Anal. 28(9–10), 1027–1049 (2007)
Axelsson, O., Karátson J.: Mesh independent superlinear PCG rates via compact-equivalent operators. SIAM J. Numer. Anal. 45(4), 1495–1516 (2007) (electronic)
Axelsson, O., Kolotilina, L.: Diagonally compensated reduction and related preconditioning methods. Numer. Linear Algebr. Appl. 1(2), 155–177 (1994)
Axelsson, O., Maubach, J.: On the updating and assembly of the Hessian matrix in finite element methods. Comput. Methods Appl. Mech. Eng. 71, 41–67 (1988)
Axelsson, O., Neytcheva, M.: Scalable algorithms for the solution of Navier’s equations of elasticity. J. Comput. Appl. Math. 63(1–3), 149–178 (1995)
Axelsson, O., Neytcheva, M.: An iterative solution method for Schur complement systems with inexact inner solver. In: Iliev, O., Kaschiev, M., Margenov, S., Sendov, B., Vassilevski, P.S. (eds.) Recent Advances in Numerical Methods and Applications II, pp. 795–803. World Scientific, Singapore (1999)
Axelsson, O., Vassilevski, P.S.: Algebraic multilevel preconditioning methods I. Numer. Math. 56, 157–177 (1989)
Axelsson, O., Vassilevski, P.S.: Algebraic multilevel preconditioning methods II. SIAM J. Numer. Anal. 27, 1569–1590 (1990)
Axelsson, O., Vassilevski, P.S.: Variable-step multilevel preconditioning methods. I. Self-adjoint and positive definite elliptic problems. Numer. Linear Algebr. Appl. 1(1), 75–101 (1994)
Babuška, I.: The finite element method with Lagrangian multipliers. Numer. Math. 20, 179–192 (1972/73)
Bängtsson, E., Neytcheva, M.: Finite element block-factorized preconditioners. Technical Reports from the Department of Information Technology, Uppsala University, No. 2007-008, March (2007)
Bank, R.E.: Marching algorithms for elliptic boundary value problems. II. The variable coefficient case. SIAM J. Numer. Anal. 14(5), 950–970 (1977)
Benzi, M., Golub, G.H., Liesen, J.: Numerical solution of saddle point problems. Acta Numer. 14, 1–137 (2005)
Blaheta, R.: Displacement decomposition—incomplete factorization preconditioning techniques for linear elasticity problems. Numer. Linear Algebr. Appl. 1(2), 107–128 (1994)
Blaheta, R.: Multilevel Newton methods for nonlinear problems with applications to elasticity. Copernicus 940820, Technical Report. Ostrava (1997)
Börgers, C., Widlund, O.B.: On finite element domain imbedding methods. SIAM J. Numer. Anal. 27(4), 963–978 (1990)
Bramble, J.H., Pasciak, J.E.: Preconditioned iterative methods for nonselfadjoint or indefinite elliptic boundary value problems. In: Unification of Finite Element Methods. North-Holland Math. Stud. vol. 94, pp. 167–184. North-Holland, Amsterdam (1984)
Brezinski, C., Sadok, H.: Lanczos-type algorithms for solving systems of linear equations. Appl. Numer. Math. 11(6), 443–473 (1993)
Brezzi, F.: On the existence, uniqueness and approximation of saddle-point problems arising from Lagrangian multipliers. Rev. Française Automat. Informat. Recherche Opérationnelle Sér. Rouge 8(R-2), 129–151 (1974)
Brezzi, F., Fortin, M.: Mixed and Hybrid Finite Element Methods. Springer, New York (1991)
Brezzi, F., Raviart, P.-A.: Mixed finite element methods for 4th order elliptic equations. In: Topics in Numerical Analysis III, pp. 33–56. Proc. Roy. Irish Acad. Conf., Trinity Coll., Dublin (1976)
Cao, W., Haynes, R.D., Trummer, M.R.: Preconditioning for a class of spectral differentiation matrices. J. Sci. Comput. 24(3), 343–371 (2005)
Carey, G.F., Jiang, B.-N.: Nonlinear preconditioned conjugate gradient and least-squares finite elements. Comput. Methods Appl. Mech. Eng. 62, 145–154 (1987)
Chen, H., Strikwerda, J.C.: Preconditioning for regular elliptic systems. SIAM J. Numer. Anal. 37(1), 131–151 (1999)
Ciarlet, P.G.: The Finite Element Method for Elliptic Problems. North-Holland, Amsterdam (1978)
Ciarlet, P: Mathematical Elasticity. Vol. I. Three-dimensional elasticity. Studies in Mathematics and its Applications, vol. 20. North-Holland, Amsterdam (1988)
Ciarlet, P.G., Raviart, P.-A.: Maximum principle and uniform convergence for the finite element method. Comput. Methods Appl. Mech. Eng. 2, 17–31 (1973)
Concus, P., Golub, G.H.: Use of fast direct methods for the efficient numerical solution of nonseparable elliptic equations. SIAM J. Numer. Anal. 10, 1103–1120 (1973)
Concus, P., Golub, G.H.: A generalized conjugate method for non-symmetric systems of linear equations. In: Glowinski, R., Lions, J.-L. (eds.) Lect. Notes Math. Syst. vol. 134, pp. 56–65. Springer, New York (1976)
Czách, L.: The steepest descent method for elliptic differential equations (in Russian). C.Sc. thesis (1955)
Dryja, M.: A priori estimates in \(W^{2}_{2}\) in a convex domain for systems of elliptic difference equations (Russian). Ž. Vyčisl. Mat. i Mat. Fiz. 12, 1595–1601, 1632 (1972)
Dryja, M.: An iterative substructuring method for elliptic mortar finite element problems with discontinuous coefficients. In: Domain Decomposition Methods 10 (Boulder, CO, 1997). Contemp. Math. vol. 218, pp. 94–103. AMS, Providence (1998)
D’yakonov, E.G.: On an iterative method for the solution of finite difference equations (in Russian). Dokl. Akad. Nauk SSSR 138, 522–525 (1961)
D’yakonov, E.G.: The construction of iterative methods based on the use of spectrally equivalent operators. USSR Comput. Math. Math. Phys. 6, 14–46 (1965)
Eisenstat, S.C., Elman, H.C., Schultz. M.H.: Variational iterative methods for nonsymmetric systems of linear equations. SIAM J. Numer. Anal. 20(2), 345–357 (1983)
Elman, H.C., Schultz. M.H.: Preconditioning by fast direct methods for nonself-adjoint nonseparable elliptic equations. SIAM J. Numer. Anal. 23, 44–57 (1986)
Elman, H.C., Silvester, D.J., Wathen, A.J.: Finite Elements and Fast Iterative Solvers: with Applications in Incompressible Fluid Dynamics, Numerical Mathematics and Scientific Computation. Oxford University Press, New York (2005)
Ewing, R.E., Margenov, S.D., Vassilevski, P.S.: Preconditioning the biharmonic equation by multilevel iterations. Math. Balk. (N.S.) 10(1), 121–132 (1996)
Faber, V., Manteuffel, T.: Necessary and sufficient conditions for the existence of a conjugate gradient method. SIAM J. Numer. Anal. 21(2), 352–362 (1984)
Faber, V., Manteuffel, T., Parter, S.V.: On the theory of equivalent operators and application to the numerical solution of uniformly elliptic partial differential equations. Adv. Appl. Math. 11, 109–163 (1990)
Faragó, I., Karátson, J.: Numerical Solution of Nonlinear Elliptic Problems via Preconditioning Operators. Theory and Applications. Advances in Computation, vol. 11. NOVA Science, New York (2002)
Funaro, D.: Polynomial Approximation of Differential Equations, Lecture Notes in Physics, New Series, Monographs, vol. 8. Springer, New York (1992)
Goldstein, C.I., Manteuffel, T.A., Parter, S.V.: Preconditioning and boundary conditions without H 2 estimates: L 2 condition numbers and the distribution of the singular values. SIAM J. Numer. Anal. 30(2), 343–376 (1993)
Golub, G.H., O’Leary, D.P.: Some history of the conjugate gradient and Lanczos algorithms: 1948–1976. SIAM Rev. 31(1), 50–102 (1989)
Golub, G.H., Van Loan, C.F.: Matrix Computations. Johns Hopkins University Press, Baltimore (1983)
Golub, G.H., Wathen, A.J.: An iteration for indefinite systems and its application to the Navier–Stokes equations. SIAM J. Sci. Comput. 19(2), 530–539 (1998)
Golub, G.H., Ye, Q.: Inexact preconditioned conjugate gradient method with inner–outer iteration. SIAM J. Sci. Comput. 21(4), 1305–1320 (1999/00)
Graham, I.G., Hagger, M.J.: Unstructured additive Schwarz-conjugate gradient method for elliptic problems with highly discontinuous coefficients. SIAM J. Sci. Comput. 20, 2041–2066 (1999) (electronic)
Greenbaum, A.: Diagonal scalings of the Laplacian as preconditioners for other elliptic differential operators. SIAM J. Matrix Anal. Appl. 13, 826–846 (1992)
Greenbaum, A.: Iterative Methods for Solving Linear Systems. SIAM, Philadelphia (1997)
Guillard, H., Désidéri, J.-A.: Iterative methods with spectral preconditioning for elliptic equations. Comput. Methods Appl. Mech. Eng. 80(1–3), 305–312 (1990)
Gunn, J.E.: The numerical solution of \(\nabla\cdot a\nabla u=f\) by a semi-explicit alternating direction iterative method. Numer. Math. 6, 181–184 (1964)
Gunn, J.E.: The solution of elliptic difference equations by semi-explicit iterative techniques. SIAM J. Numer. Anal. Ser. B 2, 24–45 (1965)
Gustafsson, I.: A class of first order factorization methods. BIT 18(2), 142–156 (1978)
Hackbusch, W.: Multigrid Methods and Applications, Springer Series in Computational Mathematics, vol. 4. Springer, Berlin (1985)
Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand., B 49(6), 409–436 (1952)
Horgan, C.O.: Korn’s inequalities and their applications in continuum mechanics. SIAM Rev. 37(4), 491–511 (1995)
Joubert, W., Manteuffel, T.A., Parter, S., Wong, S.-P.: Preconditioning second-order elliptic operators: experiment and theory. SIAM J. Sci. Statist. Comput. 13(1), 259–288 (1992)
Kantorovich, L.V., Akilov, G.P.: Functional Analysis. Pergamon, Oxford (1984)
Kaporin, I.E.: New convergence results and preconditioning strategies for the conjugate gradient method. Numer. Linear Algebr. Appl. 1(2), 179–210 (1994)
Karátson, J.: Mesh independent superlinear convergence estimates of the conjugate gradient method for some equivalent self-adjoint operators. Appl. Math. (Prague) 50(3), 277–290 (2005)
Karátson, J.: On the superlinear convergence rate of the preconditioned CGM for some nonsymmetric elliptic problems. Numer. Funct. Anal. 28(9–10), 1153–1164 (2007)
Karátson, J.: Superlinear PCG algorithms: symmetric part preconditioning and boundary conditions. Numer. Funct. Anal. 29, 590–611 (2008)
Karátson, J., Faragó I.: Variable preconditioning via quasi-Newton methods for nonlinear problems in Hilbert space. SIAM J. Numer. Anal. 41(4), 1242–1262 (2003)
Karátson, J., Kurics, T.: Superlinearly convergent PCG algorithms for some nonsymmetric elliptic systems. J. Comp. Appl. Math. 212(2), 214–230 (2008)
Karátson, J., Kurics, T.: Superlinear PCG Methods for FDM Discretizations of Convection-Diffusion Equations. Preprint, ELTE Dept. Appl. Anal. Comp. Math. http://www.cs.elte.hu/applanal/preprints; 2006-13. Lecture Notes Comp. Sci., Springer (to appear)
Karátson, J., Kurics, T., Lirkov, I.: A parallel algorithm for systems of convection–diffusion equations. In: Boyanov, T., et al. (eds.) NMA 2006, Lecture Notes Comp. Sci., vol. 4310, pp. 65–73, Springer, New York (2007)
Kim, S.D., Parter, S.V.: Semicirculant preconditioning of elliptic operators. SIAM J. Numer. Anal. 41(2), 767–795 (2003)
Klawonn, A., Widlund, O.B.: New results on FETI methods for elliptic problems with discontinuous coefficients. In: Numerical analysis 1999 (Dundee), Res. Notes Math., vol. 420, pp. 191–209. Chapman & Hall/CRC, Boca Raton (2000)
Knyazev, A, Lashuk, I.: Steepest descent and conjugate gradient methods with variable preconditioning. Electronic. Math. NA/0605767, arXiv. org. http://arxiv.org/abs/math/0605767 (2006–2007)
Kuznetsov, Y.A., Rossi, T.: Fast direct method for solving algebraic systems with separable symmetric band matrices. East-West J. Numer. Math. 4(1), 53–68 (1996)
Kraus, J.: Algebraic multilevel preconditioning of finite element matrices using local Schur complements. Numer. Linear Algebr. Appl. 13, 49–70 (2006)
Křížek, M., Lin Q.: On diagonal dominance of stiffness matrices in 3D. East-West J. Numer. Math. 3, 59–69 (1995)
Langer, U., Queck, W.: On the convergence factor of Uzawa’s algorithm. J. Comput. Appl. Math. 15(2), 191–202 (1986)
Loghin, D.: Green’s functions for preconditioning. DPhil Thesis, Oxford (1999)
Manteuffel, T.: The Tchebychev iteration for nonsymmetric linear systems. Numer. Math. 28(3), 307–327 (1977)
Manteuffel, T., Otto, J.: Optimal equivalent preconditioners. SIAM J. Numer. Anal. 30, 790–812 (1993)
Manteuffel, T., Parter, S.V.: Preconditioning and boundary conditions. SIAM J. Numer. Anal. 27(3), 656–694 (1990)
Mayo, A., Greenbaum, A.: Fast parallel iterative solution of Poisson’s and the biharmonic equations on irregular regions. SIAM J. Sci. Statist. Comput. 13(1), 101–118 (1992)
Mikhlin, S.G.: The Numerical Performance of Variational Methods. Walters–Noordhoff, Alphen aan den Rijn (1971)
Mikhlin, S.G.: Constants in Some Inequalities of Analysis (translated from the Russian by R. Lehmann). Wiley, Chichester (1986)
Nečas, J., Hlaváček, I.: Mathematical Theory of Elastic and Elasto-plastic Bodies: an Introduction, Studies in Applied Mechanics, vol. 3. Elsevier, Amsterdam (1980)
Nevanlinna, O.: Convergence of Iterations for Linear Equations. Birkhäuser, Basel (1993)
Neuberger, J.W.: Sobolev Gradients and Differential Equations, Lecture Notes in Math., No. 1670. Springer, New York (1997)
Nitsche, J., Nitsche, J.C.C.: Error estimates for the numerical solution of elliptic differential equations. Arch. Ration. Mech. Anal. 5, 293–306 (1960)
Repin, S., Sauter, S., Smolianski, A.: A posteriori error estimation for the Poisson equation with mixed Dirichlet/Neumann boundary conditions. J. Comput. Appl. Math. 164–165, 601–612 (2004)
Rossi, T., Toivanen, J.: Parallel fictitious domain method for a non-linear elliptic Neumann boundary value problem, Czech-US workshop in iterative methods and parallel computing, Part I (Milovy, 1997). Numer. Linear Algebr. Appl. 6(1), 51–60 (1999)
Rossi, T., Toivanen, J.: A parallel fast direct solver for block tridiagonal systems with separable matrices of arbitrary dimension. SIAM J. Sci. Comput. 20(5), 1778–1796 (1999)
Saad, Y., Schultz, M.H.: GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Statist. Comput. 7, 856–869 (1986)
Saad, Y.: Iterative Methods for Sparse Linear Systems. PWS Publ. Co., Boston (1996)
Saad, Y.: A flexible inner-outer preconditioned GMRES algorithm. SIAM J. Sci. Comput. 14(2), 461–469 (1993)
Simoncini V., Szyld D.B.: Recent computational developments in Krylov subspace methods for linear systems. Numer. Linear Algebr. Appl. 14(1), 1–59 (2007)
Simoncini V., Szyld D.B.: Flexible inner-outer Krylov subspace methods. SIAM J. Numer. Anal. 40, 2219–2239 (2003)
Sundqvist, P.: Numerical computations with fundamental solutions. PhD thesis, Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology, Uppsala University (2005)
Swarztrauber, P.N.: A direct method for the discrete solution of separable elliptic equations. SIAM J. Numer. Anal. 11, 1136–1150 (1974)
Swarztrauber, P.N.: The methods of cyclic reduction, Fourier analysis and the FACR algorithm for the discrete solution of Poisson’s equation on a rectangle. SIAM Rev. 19(3), 490–501 (1977)
Temam, R.: Navier-Stokes Equations. Theory and Numerical Analysis, Studies in Mathematics and its Applications, vol. 2. North-Holland, Amsterdam (1977)
Vassilevski, P.S.: Fast algorithm for solving a linear algebraic problem with separable variables. C. R. Acad. Bulgare Sci. 37(3), 305–308 (1984)
Vladimirov, V.S.: Equations of Mathematical Physics (translated from the Russian by E. Yankovsky). Mir, Moscow (1984)
Winter, R.: Some superlinear convergence results for the conjugate gradient method. SIAM J. Numer. Anal. 17, 14–17 (1980)
Widlund, O.: On the use of fast methods for separable finite difference equations for the solution of general elliptic problems. In: Rose, D.J., Willoughby, R.A. (eds.) Sparse Matrices and Applications, pp. 121–134. Plenum, New York (1972)
Widlund, O.: A Lanczos method for a class of non-symmetric systems of linear equations. SIAM J. Numer. Anal. 15, 801–812 (1978)
Young, D.M.: Iterative Solution of Large Linear Systems. Academic, New York (1971)
Zlatev, Z.: Computer Treatment of Large Air Pollution Models. Kluwer Academic, Dordrecht (1995)
Author information
Authors and Affiliations
Corresponding author
Additional information
Dedicated to the memory of Gene Golub for his friendly manner and for his broad interest and significant impact on numerical analysis.
Rights and permissions
About this article
Cite this article
Axelsson, O., Karátson, J. Equivalent operator preconditioning for elliptic problems. Numer Algor 50, 297–380 (2009). https://doi.org/10.1007/s11075-008-9233-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11075-008-9233-4