Skip to main content
SpringerLink
Log in

Synchronization-Reducing Variants of the Biconjugate Gradient and the Quasi-Minimal Residual Methods

  • Conference paper
Algorithms and Architectures for Parallel Processing (ICA3PP 2013)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 8285))

  • 1537 Accesses

Abstract

The Biconjugate Gradient (BiCG) and the Quasi-Minimal Residual (QMR) method are among the popular iterative methods for the solution of large, sparse, non-symmetric systems of linear equations. When these methods are implemented on large-scale parallel computers, their scalability is limited by the synchronization caused when carrying out inner product-like operations. Therefore, we propose two new synchronization-reducing variants of BiCG and QMR in an attempt to mitigate these negative performance effects. The idea behind these new s-step variants is to group several dot products for joint execution. Although these new algorithms still reveal numerical instabilities, they are shown to keep the cost of inner product-like operations almost independent of the number of processes, thus improving scalability significantly.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Fletcher, R.: Conjugate gradient methods for indefinite systems. In: Watson, G. (ed.) Numerical Analysis. LNM, vol. 506, pp. 73–89. Springer, Heidelberg (1976)

    Chapter  Google Scholar 

  2. Freund, R.W., Nachtigal, N.M.: An implementation of the QMR method based on coupled two-term recurrences. SIAM J. Sci. Comput. 15(2), 313–337 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  3. Saad, Y.: Krylov subspace methods on supercomputers. SIAM J. Sci. Stat. Comput. 10(6), 1200–1232 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  4. van der Vorst, H.: Iterative methods for the solution of large systems of equations on supercomputers. Advances in Water Resources 13(3), 137–146 (1990)

    Article  Google Scholar 

  5. Demmel, J., Heath, M., van der Vorst, H.: Parallel numerical linear algebra. Acta Numerica 2(1), 111–197 (1993)

    Article  Google Scholar 

  6. Duff, I.S., van der Vorst, H.A.: Developments and trends in the parallel solution of linear systems. Parallel Computing 25(13-14), 1931–1970 (1999)

    Article  MathSciNet  Google Scholar 

  7. Bücker, H.M.: Iteratively solving large sparse linear systems on parallel computers. NIC Serices, John Von Neumann Institute f. Computing. Jülich 10, 521–548 (2002)

    Google Scholar 

  8. Bücker, H.M., Sauren, M.: Reducing global synchronization in the biconjugate gradient method. In: Yang, T. (ed.) Parallel numerical computations with applications, pp. 63–76. Kluwer Academic Publishers, Norwell (1999)

    Chapter  Google Scholar 

  9. Fischer, B., Freund, R.: An inner product-free conjugate gradient-like algorithm for Hermitian positive definite systems. In: Brown, J., et al. (eds.) Proc. Cornelius Lanczos Intern. Centenary Conf., pp. 288–290. SIAM (1994)

    Google Scholar 

  10. Meurant, G.: The conjugate gradient method on supercomputers. Supercomputer 13, 9–17 (1986)

    Google Scholar 

  11. Van Rosendale, J.: Minimizing inner product data dependencies in conjugate gradient iteration. NASA Contractor Report NASA–CR–172178, NASA Langley Research Center, Center, Hampton, VA (1983)

    Google Scholar 

  12. Bücker, H.M., Sauren, M.: A Variant of the Biconjugate Gradient Method Suitable for Massively Parallel Computing. In: Bilardi, G., Ferreira, A., Lüling, R., Rolim, J. (eds.) IRREGULAR 1997. LNCS, vol. 1253, pp. 72–79. Springer, Heidelberg (1997)

    Chapter  Google Scholar 

  13. Bücker, H.M., Sauren, M.: A Parallel Version of the Quasi-Minimal Residual Method Based on Coupled Two-Term Recurrences. In: Waśniewski, J., Dongarra, J., Madsen, K., Olesen, D. (eds.) PARA 1996. LNCS, vol. 1184, pp. 157–165. Springer, Heidelberg (1996)

    Chapter  Google Scholar 

  14. Chronopoulos, A.T.: A Class of Parallel Iterative Methods Implemented on Multiprocessors. Technical report UIUCDCS–R–86–1267, Department of Computer Science, University of Illinois, Urbana, Illinois (1986)

    Google Scholar 

  15. Chronopoulos, A.T., Gear, C.W.: S-step iterative methods for symmetric linear systems. J. Comput. Appl. Math. 25(2), 153–168 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  16. Chronopoulos, A.T., Swanson, C.D.: Parallel iterative s-step methods for unsymmetric linear systems. Parallel Computing 22(5), 623–641 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  17. Kim, S.K., Chronopoulos, A.: A class of Lanczos-like algorithms implemented on parallel computers. Parallel Computing 17(6-7), 763–778 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  18. Kim, S.K., Chronopoulos, A.T.: An efficient nonsymmetric Lanczos method on parallel vector computers. J. Comput. Appl. Math. 42(3), 357–374 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  19. Alvarez-Dios, J.A., Cabaleiro, J.C., Casal, G.: A generalization of s-step variants of gradient methods. J. Comput. Appl. Math. 236(12), 2938–2953 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  20. Mohiyuddin, M., Hoemmen, M., Demmel, J., Yelick, K.: Minimizing communication in sparse matrix solvers. In: Proc. Conf. High Perf. Comput. Networking, Storage and Analysis, SC 2009, pp. 36:1–36:12. ACM, New York (2009)

    Google Scholar 

  21. Hoemmen, M.F.: Communication-avoiding Krylov subspace methods. PhD thesis, EECS Department, University of California, Berkeley (2010)

    Google Scholar 

  22. Carson, E., Knight, N., Demmel, J.: Avoiding communication in two-sided Krylov subspace methods. SIAM J. Sci. Comput. 35(5), S42–S61 (2013)

    Google Scholar 

  23. Ghysels, P., Ashby, T.J., Meerbergen, K., Vanroose, W.: Hiding global communication latency in the GMRES algorithm on massively parallel machines. SIAM J. Sci. Comput. 35(1), 48–71 (2013)

    Article  MathSciNet  Google Scholar 

  24. Ghysels, P., Vanroose, W.: Hiding global synchronization latency in the preconditioned Conjugate Gradient algorithm. Parallel Computing (in press, 2013)

    Google Scholar 

  25. Curfmann McInnes, L., Smith, B., Zhang, H., Mills, R.T.: Hierarchical and nested Krylov methods for extreme-scale computing. Parallel Computing (in press, 2013)

    Google Scholar 

  26. Lanczos, C.: An iteration method for the solution of the eigenvalue problem of linear differential and integral operators. J. Res. Nat. Bur. Stand. 45(4), 255–282 (1950)

    Article  MathSciNet  Google Scholar 

  27. Feuerriegel, S., Bücker, H.M.: A normalization scheme for the non-symmetric s-Step Lanczos algorithm. In: Kołodziej, J., Aversa, R., Zhang, J., Amato, F., Fortino, G. (eds.) ICA3PP 2013, Part II. LNCS, vol. 8286, pp. 30–39. Springer, Heidelberg (2013)

    Google Scholar 

  28. Freund, R., Nachtigal, N.: QMR: a quasi-minimal residual method for non-Hermitian linear systems. Num. Math. 60(1), 315–339 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  29. Sauren, M., Bücker, H.M.: On deriving the quasi-minimal residual method. SIAM Review 40(4), 922–926 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  30. van der Vorst, H.A., Ye, Q.: Residual replacement strategies for Krylov subspace iterative methods for the convergence of true residuals. SIAM J. Sci. Comput. 22(3), 835–852 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  31. Carson, E., Demmel, J.: A residual replacement strategy for improving the maximum attainable accuracy of s-step Krylov subspace methods. Technical Report UCB/EECS–2012–197, University of California, Berkeley (2012)

    Google Scholar 

  32. Gustafsson, M., Demmel, J., Holmgren, S.: Numerical evaluation of the communication-avoiding Lanczos algorithm. Technical Report 2012–001, Department of Information Technology, Uppsala University (January 2012)

    Google Scholar 

  33. Freund, R.W., Hochbruck, M.: A biconjugate gradient type algorithm on massively parallel architectures. In: Vichnevetsky, R., Miller, J.J.H. (eds.) IMACS 1991 Proc. 13th World Congress Comput. Appl. Math, pp. 720–721. Criterion Press, Dublin (1991)

    Google Scholar 

  34. Freund, R.W., Hochbruck, M.: A biconjugate gradient-type algorithm for the iterative solution of non-Hermitian linear systems on massively parallel architectures. In: Brezinski, C., Kulisch, U. (eds.) IMACS 1991, Proc. 13th World Congress Comput. Appl. Math. I, pp. 169–178. Elsevier Science Publishers (1992)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Freiburg, 79098, Freiburg, Germany

    Stefan Feuerriegel

  2. Friedrich Schiller University Jena, 07743, Jena, Germany

    H. Martin Bücker

Authors
  1. Stefan Feuerriegel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. H. Martin Bücker
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute of Computer Science, Cracow University of Technology, Warszawska 24, 31-155, Cracow, Poland

    Joanna Kołodziej

  2. Dipartimento di Ingegneria, Seconda Universita’ di Napoli, 81031, Aversa, CE, Italy

    Beniamino Di Martino

  3. DIMES and ICAR-CNR, c/o Università della Calabria, 87036, Rende, CS, Italy

    Domenico Talia

  4. College of Computing and Information Sciences, Rochester Institute of Technology, 14623, Rochester, NY, USA

    Kaiqi Xiong

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer International Publishing Switzerland

About this paper

Cite this paper

Feuerriegel, S., Bücker, H.M. (2013). Synchronization-Reducing Variants of the Biconjugate Gradient and the Quasi-Minimal Residual Methods. In: Kołodziej, J., Di Martino, B., Talia, D., Xiong, K. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2013. Lecture Notes in Computer Science, vol 8285. Springer, Cham. https://doi.org/10.1007/978-3-319-03859-9_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-03859-9_19

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-03858-2

  • Online ISBN: 978-3-319-03859-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation