Skip to main content
SpringerLink
Log in

Parallel MCGLS and ICGLS Methods for Least Squares Problems on Distributed Memory Architectures

  • Conference paper
  • First Online:
Parallel and Distributed Processing and Applications (ISPA 2003)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2745))

Abstract

In this paper we mainly study the parallelization of the CGLS method, a basic iterative method for large and sparse least squares problems in which the conjugate gradient method is applied to solve normal equations. On modern parallel architectures its parallel performance is always limited because of the global communication required for inner products, the main bottleneck of parallel performance. In this paper, we describe a modified CGLS (MCGLS) method which improve parallel performance by assembling the results of a number of inner products collectively and by creating situations where communication can be overlapped with computation. More importantly, we also propose an improved CGLS (ICGLS) method to reduce inner product’s global synchronization points to half, then significantly improve the parallel performance accordingly compared with the standard CGLS method and the MCGLS method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Z. Bai, D. Hu, and L. Reichel. A newton basis GMRES implementation. Technical Report 91-03, University of Kentucky, 1991.

    Google Scholar 

  2. A. Basermann, B. Reichel, and C. Schelthoff. Preconditioned CG methods for sparse matrices on massively parallel machines. Parallel Computing, 23(3):381–398, 1997.

    Article  MATH  MathSciNet  Google Scholar 

  3. Å. Björck. Numerical Methods for Least Squares Problems. SIAM, Philadelphia, 1995.

    Google Scholar 

  4. Å. Björck, T. Elfving, and Z. Strakos. Stability of conjugate gradient-type methods for linear least squares problems. Technical Report LiTH-MAT-R-1995-26, Department of Mathematics, Linköping University, 1994.

    Google Scholar 

  5. H. M. Bücker and M. Sauren. A parallel version of the quasi-minimal residual method based on coupled two-term recurrences. In J. Waśniewski, J. Dongarra, K. Madsen, and D. Olesen, editors, Proceedings of Workshop on Applied Parallel Computing in Industrial Problems and Optimization (Para96), LNCS184, Lecture Notes in Computer Science, pages 157–165. Technical University of Denmark, Lyngby, Denmark, Springer-Verlag, August 1996.

    Google Scholar 

  6. H. M. Bűcker and M. Sauren. Parallel biconjugate gradient methods for linear systems. In L. T. Yang, editor, Parallel Numerical Computations with Applications, pages 51–70. Kluwer Academic Publishers, 1999.

    Google Scholar 

  7. L. G. C. Crone and H. A. van der Vorst. Communication aspects of the conjugate gradient method on distributed memory machines. Supercomputer, X(6):4–9, 1993.

    Google Scholar 

  8. E. de Sturler. A parallel variant of the GMRES(m). In Proceedings of the 13th IMACS World Congress on Computational and Applied Mathematics. IMACS, Criterion Press, 1991.

    Google Scholar 

  9. E. de Sturler and H. A. van der Vorst. Reducing the effect of the global communication in GMRES(m) and CG on parallel distributed memory computers. Technical Report 832, Mathematical Institute, University of Utrecht, Utrecht, The Netherland, 1994.

    Google Scholar 

  10. J. W. Demmel, M. T. Heath, and H. A. van der Vorst. Parallel numerical algebra. Acta Numerica, 1993. Cambridge Press, New York.

    Google Scholar 

  11. J. J. Dongarra, I. S. Duff, D. C. Sorensen, and H. A. van der Vorst. Solving Linear Systems on Vector and Shared Memory Computers. SIAM, Philadelphia, PA, 1991.

    Google Scholar 

  12. I. S. Duff and J. K. Reid. A comparison of some methods for the solution of sparse over-determined system of linear equations. Journal of the Institute of Mathematics and Its Applications, 17(3):267–280, 1976.

    Article  MATH  MathSciNet  Google Scholar 

  13. T. Elfving. On the conjugate gradient method for solving linear least squares problems. Technical Report LiTH-MAT-R-78-3, Department of Mathematics, Linköping University, 1978.

    Google Scholar 

  14. G. H. Golub and W. Kahan. Calculating the singular values and pseudo-inverse of a matrix. SIAM Journal on Numerical Analysis, 2:205–224, 1965.

    Article  MathSciNet  Google Scholar 

  15. M. R. Hestenes and E. Stiefel. Methods of conjugate gradients for solving linear system. Journal of Research of National Bureau of Standards, B49:409–436, 1952.

    MathSciNet  Google Scholar 

  16. P.L. Montgomery. Parallel block Lanczos. Microsoft Research, Redmond, USA, Transparencies of a talk presented at RSA-2000, January 2000.

    Google Scholar 

  17. C. C. Paige and M. A. Saunders. LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Transactions on Mathematical Software, 8:43–71, 1982.

    Article  MATH  MathSciNet  Google Scholar 

  18. C. Pommerell. Solution of large unsymmetric systems of linear equations. PhD thesis, ETH, 1992.

    Google Scholar 

  19. E. Stiefel. Ausgleichung ohne aufstellung der gausschen normalgleichungen. Wiss. Z. Technische Hochschule Dresden, 2:441–442, 1952/1953.

    MathSciNet  Google Scholar 

  20. L. T. Yang and R. P. Brent. Quantitative performance analysis of the improved quasi-minimal residual method on massively distributed memory computers. Advances in Engineering Software, 33:169–177, 2002.

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, St. Francis Xavier University, P.O. Box 5000, Antigonish, B2G 2W5, NS, Canada

    Laurence Tianruo Yang

  2. Computing Laboratory, Oxford University, Wolfson Building, Park Road, Oxford, OX1 3QD, UK

    Laurence Tianruo Yang & Richard P. Brent

Authors
  1. Laurence Tianruo Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Richard P. Brent
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Software, The University of Aizu, Aizu-Wakamatsu City, Fukushima, 965-8580, Japan

    Minyi Guo

  2. Department of Computer Science, St. Francis Xavier University, Antigonish, NS, B2G 2W5, Canada

    Laurence Tianruo Yang

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Yang, L.T., Brent, R.P. (2003). Parallel MCGLS and ICGLS Methods for Least Squares Problems on Distributed Memory Architectures. In: Guo, M., Yang, L.T. (eds) Parallel and Distributed Processing and Applications. ISPA 2003. Lecture Notes in Computer Science, vol 2745. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-37619-4_21

Download citation

  • DOI: https://doi.org/10.1007/3-540-37619-4_21

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-40523-8

  • Online ISBN: 978-3-540-37619-4

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation