Skip to main content
Springer Nature Link
Log in

A New Projected Variant of the Deflated Block Conjugate Gradient Method

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

The deflated block conjugate gradient (D-BCG) method is an attractive approach for the solution of symmetric positive definite linear systems with multiple right-hand sides. However, the orthogonality between the block residual vectors and the deflation subspace is gradually lost along with the process of the underlying algorithm implementation, which usually causes the algorithm to be unstable or possibly have delayed convergence. In order to maintain such orthogonality to keep certain level, full reorthogonalization could be employed as a remedy, but the expense required is quite costly. In this paper, we present a new projected variant of the deflated block conjugate gradient (PD-BCG) method to mitigate the loss of this orthogonality, which is helpful to deal with the delay of convergence and thus further achieve the theoretically faster convergence rate of D-BCG. Meanwhile, the proposed PD-BCG method is shown to scarcely have any extra computational cost, while having the same theoretical properties as D-BCG in exact arithmetic. Additionally, an automated reorthogonalization strategy is introduced as an alternative choice for the PD-BCG method. Numerical experiments demonstrate that PD-BCG is more efficient than its counterparts especially when solving ill-conditioned linear systems or linear systems suffering from rank deficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Arioli, M., Duff, I.S., Ruiz, D., Sadkane, M.: Block Lanczos techniques for accelerating the block Cimmino method. SIAM J. Sci. Comput. 16(6), 1478–1511 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  2. Abdel-Rehim, A.M., Morgan, R.B., Nicely, D.A., Wilcox, W.: Deflated and restarted symmetric Lanczos methods for eigenvalues and linear equations with multiple right-hand sides. SIAM J. Sci. Comput. 32(1), 129–149 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  3. Agullo, E., Giraud, L., Jing, Y.-F.: Block GMRES method with inexact breakdowns and deflated restating. SIAM J. Matrix Anal. Appl. 35(4), 1625–1651 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  4. Aliaga, J.I., Boley, D.L., Freund, R.W., Hernández, V.: A Lanczos-type method for multiple starting vectors. Math. Comput. 69, 1577–1601 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bristeau, M.O., Erhel, J.: Augmented conjugate gradient. Application in an iterative process for the solution of scattering problems. Numer. Algorithms 18, 71–90 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  6. Birk, S.: Deflated shifted block Krylov subspace methods for Hermitian positive definite matrices, Ph.D. thesis. Fakultät für Mathematik und Naturwissenschaften, Bergische Universität Wuppertal (2015)

  7. Birk, S., Frommer, A.: A deflated conjugate gradient method for multiple right hand sides and multiple shifts. Numer. Algorithms 67, 507–529 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  8. Baglama, J.: Dealing with linear dependence during the iterations of the restarted block Lanczos methods. Numer. Algorithms 25, 23–36 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  9. Björck, A., Palge, C.: Loss and recapture of orthogonality in the modified Gram–Schmidt algorithm. SIAM J. Matrix Anal. Appl. 13(1), 176–190 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  10. Chen, J.: A deflated version of the block conjugate gradient algorithm with an application to Gaussian process maximum likelihood estimation. Preprint ANL/MCS-P1927-0811, Argonne National Laboratory, Argonne (2011)

  11. Cockett, R.: The block conjugate gradient for multiple right hand sides in a direct current resistivity inversion. http://www.row1.ca/s/pdfs/courses/BlockCG.pdf. Accessed 28 Feb 2015

  12. Chan, T.-F., Wan, W.-L.: Analysis of projection methods for solving linear systems with multiple right-hand sides. SIAM J. Sci. Comput. 18(6), 1698–1721 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  13. Chapman, A., Saad, Y.: Deflated and augmented Krylov subspace techniques. Numer. Linear Algebra Appl. 4(1), 43–66 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  14. Chatfield, D.C., Reeves, M.S., Truhlar, D.G., Duneczky, C., Schwenke, D.W.: Complex generalized minimal residual algorithm for iterative solution of quantum mechanical reactive scattering equations. J. Chem. Phys. 97(11), 8322–8333 (1992)

    Article  Google Scholar 

  15. Davis, T.A., Hu, Y.F.: The University of Florida sparse matrix collection. ACM Trans. Math. Softw. 38(1), 249–260 (2011)

    MathSciNet  MATH  Google Scholar 

  16. Deb, K.: Multi-objective Optimization Using Evolutionary Algorithms. Wiley, Chichester, New York (2001)

    MATH  Google Scholar 

  17. Erhel, J., Guyomarch, F.: An augmented conjugate gradient method for solving consecutive symmetric positive definite linear systems. SIAM J. Matrix Anal. Appl. 21(4), 1279–1299 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  18. Ekström, S.E., Neytcheva, M.: Enabling the full potential of the deflation techniques? Report. Harbin. In: The 13th International Conference of China Matrix Theory and Its Application (2018)

  19. Gaul, A., Gutknecht, M.H., Liesen, J., Nabben, R.: A framework for deflated and augmented Krylov subspace methods. SIAM J. Matrix Anal. Appl. 34(2), 495–518 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  20. Gutknecht, M.H.: Block Krylov space methods for linear systems with multiple right-hand sides: an introduction. In: Siddiqi, A.H., Duff, I.S., Christensen, O. (eds.) Modern Mathematical Models, Methods and Algorithms for Real World Systems, pp. 420–447. Anamaya, New Delhi (2007)

    Google Scholar 

  21. Golub, G.H., Ruiz, D., Touhami, A.: A hybrid approach combining Chebyshev filter and conjugate gradient for solving linear systems with multiple right-hand sides. SIAM J. Matrix Anal. Appl. 29(3), 774–795 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  22. Giraud, L., Langou, J., Rozloznik, M.: The loss of orthogonality in the Gram–Schmidt orthogonalization process. Comput. Math. Appl. 50(7), 1069–1075 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  23. Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49(6), 409–436 (1952)

    Article  MathSciNet  MATH  Google Scholar 

  24. Ji, H., Li, Y.-H.: A breakdown-free block conjugate gradient method. BIT Numer. Math. 57, 379–403 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  25. Kahl, K., Rittich, H.: The deflated conjugate gradient method: convergence, perturbation and accuracy. Linear Algebra Appl. 515, 111–129 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  26. Morgan, R.B.: A restarted GMRES method augmented with eigenvectors. SIAM J. Matrix Anal. Appl. 16(4), 1154–1171 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  27. Morgan, R.B.: Restarted block-GMRES with deflation of eigenvalues. Appl. Numer. Math. 54, 222–236 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  28. Nikishin, A.A., Yeremin, A.Y.: Variable block CG algorithms for solving large sparse symmetric positive definite linear systems on parallel computers, I: general iterative scheme. SIAM J. Matrix Anal. Appl. 16(4), 1135–1153 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  29. Nicolaides, R.A.: Deflation of conjugate gradients with applications to bundary value problems. SIAM J. Numer. Appl. 24(2), 355–365 (1987)

    Article  MATH  Google Scholar 

  30. O’Leary, D.P.: The block conjugate gradient algorithm and related methods. Linear Algebra Appl. 29, 293–322 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  31. Parks, M.L., de Sturler, E., Mackey, G., Johnson, D.D., Maiti, S.: Recycling Krylov subspaces for sequences of linear systems. SIAM J. Sci. Comput. 28(5), 1651–1674 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  32. Peng, Z., Shao, Y., Lee, J.-F.: Advanced model order reduction technique in real-life IC/package design. In: IEEE Electrical Design of Advanced Package and Systems Symposium (2010)

  33. Simon, H.D.: The Lanczos algorithm with partial reorthogonalization. Math. Comput. 42(165), 115–142 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  34. Soodhalter, K.M.: Block Krylov subspace recycling for shifted systems with unrelated right-hand sides. SIAM J. Sci. Comput. 38(1), A302–A324 (2016)

    Article  MathSciNet  Google Scholar 

  35. Saad, Y., Yeung, M., Erhel, J., Guyomarch, F.: A deflated version of the conjugate gradient algorithm. SIAM J. Sci. Comput. 21(5), 1909–1926 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  36. Saad, Y.: Analysis of augmented Krylov subspace methods. SIAM J. Matrix Anal. Appl. 18(2), 435–449 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  37. Sun, D.-L., Huang, T.-Z., Carpentieri, B., Jing, Y.-F.: Flexible and deflated variants of the block shifted GMRES method. J. Comput. Appl. Math. 345, 168–183 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  38. Sun, D.-L., Huang, T.-Z., Jing, Y.-F., Carpentieri, B.: A block GMRES method with deflated restarting for solving linear systems with multiple shifts and multiple right-hand sides. Numer. Linear Algebra Appl. 25(5), e2148 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  39. Sun, D.-L., Carpentieri, B., Huang, T.-Z., Jing, Y.-F.: A spectrally preconditioned and initially deflated variant of the restarted block GMRES method for solving multiple right-hand sides linear systems. Int. J. Mech. Sci. 144, 775–787 (2018)

    Article  Google Scholar 

  40. Sun, D.-L., Huang, T.-Z., Carpentieri, B., Jing, Y.-F.: A new shifted block GMRES method with inexact breakdowns for solving multi-shifted and multiple right-hand sides linear systems. J. Sci. Comput. 78(2), 746–769 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  41. Xiang, Y.-F., Jing, Y.-F., Huang, T.-Z., Sun, D.-L.: On adaptive restart procedures for the breakdown-free block conjugate gradient method. Report. Harbin. In: The 13th International Conference of China Matrix Theory and Its Application (2018)

Download references

Acknowledgements

The authors would like to gratefully thank the anonymous referees for their insightful comments and valuable suggestions, which enabled us to substantially improve the quality of this paper.

Author information

Authors and Affiliations

  1. School of Mathematical Sciences/Institute of Computational Science, University of Electronic Science and Technology of China, Chengdu, 611731, Sichuan, People’s Republic of China

    Yan-Fei Xiang, Yan-Fei Jing & Ting-Zhu Huang

Authors
  1. Yan-Fei Xiang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Yan-Fei Jing
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Ting-Zhu Huang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Yan-Fei Jing.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This research is supported by NSFC (61772003) and Science Strength Promotion Programme of UESTC.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiang, YF., Jing, YF. & Huang, TZ. A New Projected Variant of the Deflated Block Conjugate Gradient Method. J Sci Comput 80, 1116–1138 (2019). https://doi.org/10.1007/s10915-019-00969-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10915-019-00969-4

Keywords