Abstract
We propose an automatic preconditioning scheme for large sparse numerical optimization. The strategy is based on an examination of the sparsity pattern of the Hessian matrix: using a graph-theoretic heuristic, a block-diagonal approximation to the Hessian matrix is induced. The blocks are submatrices of the Hessian matrix; furthermore, each block is chordal. That is, under a positive definiteness assumption, the Cholesky factorization can be applied to each block without creating any new nonzeros (fill). Therefore the preconditioner is space efficient. We conduct a number of numerical experiments to determine the effectiveness of the preconditioner in the context of a linear conjugate-gradient algorithm for optimization.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
A.V. Aho, J.E. Hopcroft and J.D. Ullman,The Design and Analysis of Computer Algorithms (Addison-Wesley, Menlo Par, CA, 1974).
T.F. Coleman,Large Sparse Numerical Optimization (Springer-Verlag, Berlin, 1984).
T.F. Coleman and J.J. Moré, “Estimation of sparse Jacobian matrices and graph coloring problems,”SIAM Journal on Numerical Analysis 20 (1983) 187–209.
T.F. Coleman and J.J. Moré, “Estimation of sparse Hessian matrices and graph coloring problems,”Mathematical Programming 28 (1984) 243–270.
T.F. Coleman, B. Garbow and J.J. Moré, “Software for estimating sparse Jacobian matrices,”ACM Transactions on Mathematical Software 10 (1984) 329–347.
T.F. Coleman, B. Garbow and J.J. Moré, “Software for estimating sparse Hessian matrices,”ACM Transactions on Mathematical Software 11 (1985) 363–378.
T.F. Coleman and Jin-yi Cai, “The cyclic coloring problem and estimation of sparse Hessian matrices,”SIAM Journal on Algebraic and Discrete Methods 7 (1986) 221–235.
P.M. Dearing, D.R. Shier and D.D. Warner, “Maximal chordal subgraphs,” Technical Report 406, Clemson University (Clemson, SC, 1983).
J.E. Dennis, Jr. and T. Steihaug, “On the successive projections approach to least-squares problems,”SIAM Journal on Numerical Analysis 23 (1986) 717–733.
J.J. Dongarra, J.R. Bunch, C.B. Moler and G.W. Stewart,LINPACK Users Guide (SIAM Publications, Philadelphia, 1978).
G.C. Everstine, “A comparison of three resequencing algorithms for the reduction of matrix profile and wave font,”International Journal on Numerical Methods in Engineering 14 (1979) 837–853.
A.J. George and J.W. Liu,Computer Solution of Large Sparse Positive Definite Systems (Prentice-Hall, Englewood Cliffs, 1981).
P.E. Gill, W. Murray and M.H. Wright,Practical Optimization (Academic Press, New York, 1981).
G.H. Golub and C.F. Van Loan,Matrix Computations (The Johns Hopkins University Press, Baltimore, MD, 1983).
M.C. Golumbic,Algorithmic Graph Theory and Perfect Graphs (Academic Press, New York, 1980).
I. Gustafsson, “A class of first-order factorization methods,”BIT 18 (1980) 142–156.
T.A. Manteuffel, “An incomplete factorization technique for positive definite linear systems,”Mathematics of Computation 34 (1980) 473–497.
N. Munksgaard, “Solving sparse symmetric sets of linear equations by preconditioned conjugate gradients,”ACM Transactions on Mathematical Software 6 (1980) 206–219.
M.J.D. Powell, “A new algorithm for unconstrained optimization,” in: J.B. Rosen, O.L. Mangasarian and K. Ritter, eds,Nonlinear Programming (Academic Press, New York, 1970) pp. 31–65.
M.J.D. Powell and Ph.L. Toint, “On the estimation of sparse Hessian matrices,”SIAM Journal on Numerical Analysis 16 (1979) 1060–1074.
D.J. Rose, R.E. Tarjan and G.S. Leuker, “Algorithmic aspects of vertex elimination on graphs,”SIAM Journal on Computing 5 (1976) 266–283.
T. Steihaug, “The conjugate gradient method and trust regions in large scale optimization,”SIAM Journal on Numerical Analysis 20 (1983) 626–637.
M.N. Thapa, “Optimization of unconstrained functions with sparse Hessian matrices—Newton-type methods,”Mathematical Programming 29 (1984) 156–186.
Ph.L. Toint, “Some numerical results using a sparse matrix updating formula in unconstrained optimization,”Mathematics of Computation 32 (1978) 839–851.
Ph.L. Toint, “Towards an efficient sparsity exploiting Newton method for minimization,” in: I.S. Duff, ed.,Sparse Matrices and their Uses (Academic Press, New York, 1981) pp. 57–87.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Coleman, T.F. A chordal preconditioner for large-scale optimization. Mathematical Programming 40, 265–287 (1988). https://doi.org/10.1007/BF01580736
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01580736