Abstract
We propose a generalization of the limited-memory Cholesky factorization of Lin and Moré (SIAM J. Sci. Comput. 21(1), 24–45, 1999) to the symmetric indefinite case with special interest in symmetric quasi-definite matrices. We use this incomplete factorization to precondition two formulations of linear systems arising from regularized interior-point methods for quadratic optimization. An advantage of the limited-memory approach is predictable memory requirements. We establish existence of incomplete factors when the input matrix is an H-matrix but our numerical results illustrate that the factorization succeeds more generally. An appropriate diagonal shift is applied whenever the input matrix is not quasi definite. As the memory parameter increases an efficiency measure of the preconditioner suggested by Scott and Tůma (2013) improves. The combination of the 3×3 block formulation analyzed by Greif, Moulding, and Orban (SIAM J. Optim. 24(1), 49–83, 2014), the SYMAMD ordering, and a moderate memory parameter results in encouraging performance.
Similar content being viewed by others
References
Amestoy, P., Davis, T., Duff, I.: An approximate minimum degree ordering algorithm. SIAM J. Matrix Anal. Appl. 17(4), 886–905 (1996). doi:10.1137/S0895479894278952
Anderson, B.D.O., Moore, J.B.: Optimal Filtering. Prentice-Hall, Englewood Cliffs (1979)
Aravkin, A.Y., Burke, J.V., Pillonetto, G.: Optimization viewpoint on kalman smoothing with applications to robust and sparse estimation. In: Compressed Sensing & Sparse Filtering, pp. 237–280. Springer (2014)
Arioli, M., Orban, D.: Iterative methods for symmetric quasi-definite linear systems—Part I: Theory. Cahier du GERAD G-2013-32, GERAD, Montréal, QC, Canada (2013)
Axelsson, O.: Iterative Solution Methods. Cambridge University Press, Cambridge (1994)
Bunch, J.R., Kaufman, L.: Some stable methods for calculating inertia and solving symmetric linear systems. Math. Comput. 31(137), 163–179 (1977)
Buoni, J.J.: A stable method for the incomplete factorization of H-matrices. Linear Algebra Appl. 129, 143–154 (1990). doi:10.1016/0024-3795(90)90302-S
Byrd, R.H., Nocedal, J., Waltz, R.A.: KNITRO: An integrated package for nonlinear optimization. In G. di Pillo and M. Roma, editors, Large-Scale Nonlinear Optimization, volume 83 of Nonconvex Optimization and Its Applications, pp. 35–59. Springer Verlag (2006)
Chan, W.M., George, A.: A linear time implementation of the reverse Cuthill-McKee algorithm. BIT Numer. Math. 20(1), 8–14 (1980). doi:10.1007/BF01933580
Courtier, P.: Dual formulation of four-dimensional variational assimilation. Q. J. R. Meteorol. Soc. 123, 2449–2461 (1997). doi:10.1002/qj.49712354414
Cuthill, E., McKee, J.: Reducing the bandwidth of sparse symmetric matrices. In: Proceedings of the 1969 24th national conference, ACM ’69, pp. 157–172 (1969). doi:10.1145/800195.805928
Dolan, E., Moré, J.: Benchmarking optimization software with performance profiles. Math. Program. Ser. B 91, 201–213 (2002). doi:10.1007/s101070100263
Dollar, H.S., Gould, N.I.M., Schilders, W.H.A., Wathen, A.J.: Implicit-factorization preconditioning and iterative solvers for regularized saddle-point systems. SIAM J. Matrix Anal. Appl. 28(1), 170–189 (2006). doi:10.1137/05063427X
Elman, H., Silvester, D., Wathen, A.: Finite Elements and Fast Iterative Solvers with Applications in Incompressible Fluid Dynamics. Numerical Mathematics and Scientific Computation. Oxford University Press, Oxford (2005)
Forsgren, A., Gill, Ph. E., Wright, M.H.: Interior methods for nonlinear optimization. SIAM Rev. 44(4), 525–597 (2002). doi:10.1137/S0036144502414942
Friedlander, M.P., Orban, D.: A primal-dual regularized interior-point method for convex quadratic programs. Math. Program. Comput. 4(1), 71–107 (2012). doi:10.1007/s12532-012-0035-2
Friedlander, M.P., van Den Berg, E.: Spot—a linear-operator toolbox (2013). www.cs.ubc.ca/labs/scl/spot
Gill, P.E., Saunders, M.A., Shinnerl, J.R.: On the stability of Cholesky factorization for symmetric quasidefinite systems. SIAM J. Optim. 17(1), 35–46 (1996). doi:10.1137/S0895479893252623
Golub, G.H., Van Loan, C.F., 3rd: Matrix Computations. Johns Hopkins Series in the Mathematical Sciences. Johns Hopkins, Baltimore (1996)
Gould, N.I.M., Orban, D., Toint, Ph. L.: CUTEr and SifDec, a Constrained and Unconstrained Testing Environment, revisited. ACM Trans. Math. Softw. 29(4), 373–394 (2003)
Gould, N.I.M., Orban, D., Toint, Ph. L.: Numerical methods for large-scale nonlinear optimization. Acta Numerica 14, 299–361 (2005). doi:10.1017/S0962492904000248
Greif, C., He, S., Liu, P.: sym-ildl: Incomplete L D L T factorizations of indefinite symmetric matrices (2013). URL www.cs.ubc.ca/inutard/html
Greif, C., Moulding, E., Orban, D.: Bounds on the eigenvalues of block matrices arising from interior-point methods. SIAM J. Optim. 24(1), 49–83 (2014). doi:10.1137/120890600
Gustafsson, I.: A class of first order factorization methods. BIT 18(2), 142–156 (1978). doi:10.1007/BF01931691
Higham, N.J., 2nd: Accuracy and Stability of Numerical Algorithms. Number OT80 in Other Titles in Applied Mathematics. SIAM, Philadelphia (2002)
HSL: A collection of Fortran codes for large-scale scientific computation. www.hsl.rl.ac.uk (2013)
Jones, M.T., Plassmann, P.E.: An improved incomplete Cholesky factorization. ACM Trans. Math. Softw. 21(1), 5–17 (1995). doi:10.1145/200979.200981
Kaporin, I.E.: High quality preconditioning of a general symmetric positive definite matrix based on its \(U^{T U + U^{T} R + R^{T} U}\) decomposition. Numer. Linear Algebra Appl. 5(6), 483–509 (1998). doi:10.1002/(SICI)1099-1506(199811/12)5:6<483::AID-NLA156>3.0.CO;2-7
Li, N., Saad, Y., Chow, E.: Crout versions of ILU for general sparse matrices. SIAM J. Sci. Comput. 25(2), 716–728 (2003). doi:10.1137/S1064827502405094
Li, N., Saad, Y.: Crout versions of ILU factorization with pivoting for sparse symmetric matrices. Electron. Trans. Numer. Anal. 20, 75–85 (2005)
Lin, C.-J., Moré, J.J.: Incomplete Cholesky factorizations with limited memory. SIAM J. Sci. Comput. 21(1), 24–45 (1999). doi:10.1137/S1064827597327334
Manteuffel, T.A.: Shifted incomplete Cholesky factorization. In: Duff, I. S., Stewart, G. W. (eds.) Sparse Matrix Proceedings, pp. 41–61. SIAM, Philadelphia (1979)
Manteuffel, T.A.: An incomplete factorization technique for positive definite linear systems. Math. Comput. 34(150), 473–497 (1980)
Maros, I., Mészáros, C.: A repository of convex quadratic programming problems. Optim. Methods Softw. 11 & 12, 671–681 (1999). Special Issue on Interior Point Methods.
Mehrotra, S.: On the implementation of a primal-dual interior point method. SIAM J. Optim. 2(4), 575–601 (1992). doi:10.1137/0802028
Meijerink, J.A., van der Vorst, H.A.: An iterative solution method for linear equations systems of which the coefficient matrix is a symmetric M-matrix. Math. Comput. 137, 148–162 (1977)
Messaoudi, A.: On the stability of the incomplete LU-factorizations and characterizations of H-matrices. Numer. Math. 69(3), 321–331 (1995). doi:10.1007/s002110050095
Munksgaard, N.: Solving sparse symmetric sets of linear equations by preconditioned conjugate gradients. ACM Trans. Math. Softw. 6(2), 206–219 (1980). doi:10.1145/355887.355893
Ortega, J.M.: Introduction to Parallel and Vector Solution of Linear Systems. Fontiers of Computer Science. Springer Science+Business Media, New York (1988). Originally published by Plenum Press
Paige, C.C., Saunders, M.A.: Solution of sparse indefinite systems of linear equations. SIAM J. Numer. Anal. 12(4), 617–629 (1975). doi:10.1137/0712047
Saad, Y.: ILUT: A dual threshdol incomplete LU factorization. Numer Linear Algebra Appl 1(4), 387–402 (1994). doi:10.1002/nla.1680010405
Saunders, M.A.: PDCO—primal-dual interior methods. Class notes #7 for Large-Scale Numerical Optimization, MS&E 318/CME 338, Stanford University, Spring 2013. http://web.stanford.edu/group/SOL/software/pdco
Scott, J., Tůma, M.: HSL_MI28: an efficient and robust limited-memory incomplete Cholesky factorization code. Preprint RAL-P-2013-004. Rutherford Appleton Laboratory, Didcot (2013)
Tismenetsky, M.: A new preconditioning technique for solving large sparse linear systems. Linear Algebra Appl. 154–156, 331–353 (1991). doi:10.1016/0024-3795(91)90383-8
Tůma, M.: A note on the L D L T decomposition of matrices from saddle-point problems. SIAM J. Matrix Anal. Appl. 23(4), 903–915 (2002). doi:10.1137/S0895479897321088
Vanderbei, R.J.: Symmetric quasi-definite matrices. SIAM J. Optim. 5(1), 100–113 (1995). doi:10.1137/0805005
Vanderbei, R.J., Shanno, D.F.: An interior point algorithm for nonconvex nonlinear programming. Comput. Optim. Appl. 13(1–3), 231–252 (1999). doi:10.1023/A:1008677427361
Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. (2006). doi:10.1007/s10107-004-0559-y
Author information
Authors and Affiliations
Corresponding author
Additional information
Research partially supported by an NSERC Discovery Grant.
Rights and permissions
About this article
Cite this article
Orban, D. Limited-memory LDL⊤ factorization of symmetric quasi-definite matrices with application to constrained optimization. Numer Algor 70, 9–41 (2015). https://doi.org/10.1007/s11075-014-9933-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11075-014-9933-x
Keywords
- Preconditioning
- Symmetric quasi definite
- Incomplete factorization
- Limited-memory factorization
- Interior-point methods