Abstract
The performance of CGLS, a basic iterative method whose main idea is to organize the computation of conjugate gradient method applied to normal equations for solving least squares problems. On modern architecture is always limited because of the global communication required for inner products. Inner products often therefore present a bottleneck, and it is desirable to reduce or even eliminate all the inner products. Following a note of B. Fischer and R. Freund [11], an inner product-free conjugate gradient-like algorithm is presented that simulates the standard conjugate gradient by approximating the conjugate gradient orthogonal polynomial by suitable chosen orthogonal polynomial from Bernstein-Szegö class. We also apply this kind of algorithm into normal equations as CGLS to solve the least squares problems and compare the performance with the standard and modified approaches.
This is a preview of subscription content, log in via an institution.
Preview
Unable to display preview. Download preview PDF.
References
Z. Bai, D. Hu, and L. Reichel. A newton basis GMRES implementation. Technical Report 91-03, University of Kentucky, 1991.
M. W. Berry and R. J. Plemmons. Algorithms and experiments for structural mechanics on high performance architecture. Computer Methods in Applied Mechanics and Engineering, 64:1987, 487–507.
Å. Björck, T. Elfving, and Z. Strakos. Stability of conjugate gradient-type methods for linear least squares problems. Technical Report LiTH-MAT-R-1995-26, Department of Mathematics, Linköping University, 1994.
I. C. Ohio, C. L. Monma, and D. F. Shanno. Further development of a primal-dual interior point method. ORSA Journal on Computing, 2(4):304–311, 1990.
L. G. C. Crone and H. A. van der Vorst. Communication aspects of the conjugate gradient method on distributed memory machines. Supercomputer, X(6):4–9, 1993.
E. de Sturler. A parallel variant of GMRES(m). In The 13th IMACS World Congress on Computational and Applied Mathematics. IMACS, Criterion Press, 1991.
E. de Sturler and H. A. van der Vorst. Reducing the effect of the global communication in GMRES(m) and CG on parallel distributed memory computers. Technical Report 832, Mathematical Institute, University of Utrecht, Utrecht, The Netheland, 1994.
J. W. Demmel, M. T. Heath, and H. A. van der Vorst. Parallel numerical algebra. Acta Numerica, 1993. Cambridge Press, New York.
J. J. Dongarra, I. S. Duff, D. C. Sorensen, and H. A. van der Vorst. Solving Linear Systems on Vector and Shared Memory Computers. SIAM, Philadelphia, PA, 1991.
T. Elfving. On the conjugate gradient method for solving linear least squares problems. Technical Report LiTH-MAT-R-78-3, Department of Mathematics, Linköping University, 1978.
B. Fischer and R. W. Freund. An inner product-free conjugate gradient-like algorithm for hermitian positive definite systems. In Preceedings of the Lanczos Centenary Conference, 1994.
B. Fischer and R. W. Freund. On adaptive weighted polynomial preconditioning for hermitian positive definite matrices. SIAM Jounal on Scientific Computing, 15(2):408–426, 1994.
M. Fortin and R. Glowinski. Augmented Lagrangian Methods: Application to the Numerical Solution of Boundary-value Problems. NH, 1983.
R. W. Freund. On some approximation problems for complex polynomials. Constructive Approximation, 4:111–121, 1988.
G. H. Golub and W. Kahan. Calculating the singular values and pseudo-inverse of a matrix. SIAM Journal on Numerical Analysis, 2:205–224, 1965.
M. T. Heath, R. J. Plemmons, and R. C. Ward. Sparse orthogonal schemes for structure optimization using the force method. SIAM Journal on Scientific and Statistical Computing, 5(3):514–532, 1984.
C. C. Paige and M. A. Saunders. LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Transactions on Mathematical Software, 8:43–71, 1982.
C. Pommerell. Solution of large unsymmetric systems of linear equations. PhD thesis, ETH, 1992.
G. Szegö. Orthogonal Polynomials. American Mathematical Society, Providence, RI, USA, 4 edition, 1975.
T. Yang. Data distribution and communication schemes for least squares problems on massively distributed memory computers. In Proceedings of International Conference on Computational Modelling, September 1996. Dubna, Russia.
T. Yang. Iterative methods for least squares and total least squares problems. Licentiate Thesis LiU-TEK-LIC-1996:25, 1996. Linköping University, 581 83, Linköping, Sweden.
T. Yang. Parallel least squares problems on massively distributed memory computers. In Proceedings of The Eleventh International Conference on Computer and Information Science, November 1996. Middle East Technical Unievrsity, Antalya, Turkey.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1996 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yang, T. (1996). Parallel inner product-free algorithm for least squares problems. In: Waśniewski, J., Dongarra, J., Madsen, K., Olesen, D. (eds) Applied Parallel Computing Industrial Computation and Optimization. PARA 1996. Lecture Notes in Computer Science, vol 1184. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-62095-8_76
Download citation
DOI: https://doi.org/10.1007/3-540-62095-8_76
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-62095-2
Online ISBN: 978-3-540-49643-4
eBook Packages: Springer Book Archive