Abstract
In this paper we mainly study the parallelization of the CGLS method, a basic iterative method for large and sparse least squares problems in which the conjugate gradient method is applied to solve normal equations. On modern parallel architectures its parallel performance is always limited because of the global communication required for inner products, the main bottleneck of parallel performance. In this paper, we describe a modified CGLS (MCGLS) method which improve parallel performance by assembling the results of a number of inner products collectively and by creating situations where communication can be overlapped with computation. More importantly, we also propose an improved CGLS (ICGLS) method to reduce inner product’s global synchronization points to half, then significantly improve the parallel performance accordingly compared with the standard CGLS method and the MCGLS method.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Z. Bai, D. Hu, and L. Reichel. A newton basis GMRES implementation. Technical Report 91-03, University of Kentucky, 1991.
A. Basermann, B. Reichel, and C. Schelthoff. Preconditioned CG methods for sparse matrices on massively parallel machines. Parallel Computing, 23(3):381–398, 1997.
Å. Björck. Numerical Methods for Least Squares Problems. SIAM, Philadelphia, 1995.
Å. Björck, T. Elfving, and Z. Strakos. Stability of conjugate gradient-type methods for linear least squares problems. Technical Report LiTH-MAT-R-1995-26, Department of Mathematics, Linköping University, 1994.
H. M. Bücker and M. Sauren. A parallel version of the quasi-minimal residual method based on coupled two-term recurrences. In J. Waśniewski, J. Dongarra, K. Madsen, and D. Olesen, editors, Proceedings of Workshop on Applied Parallel Computing in Industrial Problems and Optimization (Para96), LNCS184, Lecture Notes in Computer Science, pages 157–165. Technical University of Denmark, Lyngby, Denmark, Springer-Verlag, August 1996.
H. M. Bűcker and M. Sauren. Parallel biconjugate gradient methods for linear systems. In L. T. Yang, editor, Parallel Numerical Computations with Applications, pages 51–70. Kluwer Academic Publishers, 1999.
L. G. C. Crone and H. A. van der Vorst. Communication aspects of the conjugate gradient method on distributed memory machines. Supercomputer, X(6):4–9, 1993.
E. de Sturler. A parallel variant of the GMRES(m). In Proceedings of the 13th IMACS World Congress on Computational and Applied Mathematics. IMACS, Criterion Press, 1991.
E. de Sturler and H. A. van der Vorst. Reducing the effect of the global communication in GMRES(m) and CG on parallel distributed memory computers. Technical Report 832, Mathematical Institute, University of Utrecht, Utrecht, The Netherland, 1994.
J. W. Demmel, M. T. Heath, and H. A. van der Vorst. Parallel numerical algebra. Acta Numerica, 1993. Cambridge Press, New York.
J. J. Dongarra, I. S. Duff, D. C. Sorensen, and H. A. van der Vorst. Solving Linear Systems on Vector and Shared Memory Computers. SIAM, Philadelphia, PA, 1991.
I. S. Duff and J. K. Reid. A comparison of some methods for the solution of sparse over-determined system of linear equations. Journal of the Institute of Mathematics and Its Applications, 17(3):267–280, 1976.
T. Elfving. On the conjugate gradient method for solving linear least squares problems. Technical Report LiTH-MAT-R-78-3, Department of Mathematics, Linköping University, 1978.
G. H. Golub and W. Kahan. Calculating the singular values and pseudo-inverse of a matrix. SIAM Journal on Numerical Analysis, 2:205–224, 1965.
M. R. Hestenes and E. Stiefel. Methods of conjugate gradients for solving linear system. Journal of Research of National Bureau of Standards, B49:409–436, 1952.
P.L. Montgomery. Parallel block Lanczos. Microsoft Research, Redmond, USA, Transparencies of a talk presented at RSA-2000, January 2000.
C. C. Paige and M. A. Saunders. LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Transactions on Mathematical Software, 8:43–71, 1982.
C. Pommerell. Solution of large unsymmetric systems of linear equations. PhD thesis, ETH, 1992.
E. Stiefel. Ausgleichung ohne aufstellung der gausschen normalgleichungen. Wiss. Z. Technische Hochschule Dresden, 2:441–442, 1952/1953.
L. T. Yang and R. P. Brent. Quantitative performance analysis of the improved quasi-minimal residual method on massively distributed memory computers. Advances in Engineering Software, 33:169–177, 2002.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yang, L.T., Brent, R.P. (2003). Parallel MCGLS and ICGLS Methods for Least Squares Problems on Distributed Memory Architectures. In: Guo, M., Yang, L.T. (eds) Parallel and Distributed Processing and Applications. ISPA 2003. Lecture Notes in Computer Science, vol 2745. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-37619-4_21
Download citation
DOI: https://doi.org/10.1007/3-540-37619-4_21
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40523-8
Online ISBN: 978-3-540-37619-4
eBook Packages: Springer Book Archive