Skip to main content
Log in

Limited-memory LDL factorization of symmetric quasi-definite matrices with application to constrained optimization

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

We propose a generalization of the limited-memory Cholesky factorization of Lin and Moré (SIAM J. Sci. Comput. 21(1), 24–45, 1999) to the symmetric indefinite case with special interest in symmetric quasi-definite matrices. We use this incomplete factorization to precondition two formulations of linear systems arising from regularized interior-point methods for quadratic optimization. An advantage of the limited-memory approach is predictable memory requirements. We establish existence of incomplete factors when the input matrix is an H-matrix but our numerical results illustrate that the factorization succeeds more generally. An appropriate diagonal shift is applied whenever the input matrix is not quasi definite. As the memory parameter increases an efficiency measure of the preconditioner suggested by Scott and Tůma (2013) improves. The combination of the 3×3 block formulation analyzed by Greif, Moulding, and Orban (SIAM J. Optim. 24(1), 49–83, 2014), the SYMAMD ordering, and a moderate memory parameter results in encouraging performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Amestoy, P., Davis, T., Duff, I.: An approximate minimum degree ordering algorithm. SIAM J. Matrix Anal. Appl. 17(4), 886–905 (1996). doi:10.1137/S0895479894278952

    Article  MathSciNet  MATH  Google Scholar 

  2. Anderson, B.D.O., Moore, J.B.: Optimal Filtering. Prentice-Hall, Englewood Cliffs (1979)

    Google Scholar 

  3. Aravkin, A.Y., Burke, J.V., Pillonetto, G.: Optimization viewpoint on kalman smoothing with applications to robust and sparse estimation. In: Compressed Sensing & Sparse Filtering, pp. 237–280. Springer (2014)

  4. Arioli, M., Orban, D.: Iterative methods for symmetric quasi-definite linear systems—Part I: Theory. Cahier du GERAD G-2013-32, GERAD, Montréal, QC, Canada (2013)

  5. Axelsson, O.: Iterative Solution Methods. Cambridge University Press, Cambridge (1994)

    Book  Google Scholar 

  6. Bunch, J.R., Kaufman, L.: Some stable methods for calculating inertia and solving symmetric linear systems. Math. Comput. 31(137), 163–179 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  7. Buoni, J.J.: A stable method for the incomplete factorization of H-matrices. Linear Algebra Appl. 129, 143–154 (1990). doi:10.1016/0024-3795(90)90302-S

    Article  MathSciNet  Google Scholar 

  8. Byrd, R.H., Nocedal, J., Waltz, R.A.: KNITRO: An integrated package for nonlinear optimization. In G. di Pillo and M. Roma, editors, Large-Scale Nonlinear Optimization, volume 83 of Nonconvex Optimization and Its Applications, pp. 35–59. Springer Verlag (2006)

  9. Chan, W.M., George, A.: A linear time implementation of the reverse Cuthill-McKee algorithm. BIT Numer. Math. 20(1), 8–14 (1980). doi:10.1007/BF01933580

    Article  MathSciNet  MATH  Google Scholar 

  10. Courtier, P.: Dual formulation of four-dimensional variational assimilation. Q. J. R. Meteorol. Soc. 123, 2449–2461 (1997). doi:10.1002/qj.49712354414

    Article  MATH  Google Scholar 

  11. Cuthill, E., McKee, J.: Reducing the bandwidth of sparse symmetric matrices. In: Proceedings of the 1969 24th national conference, ACM ’69, pp. 157–172 (1969). doi:10.1145/800195.805928

  12. Dolan, E., Moré, J.: Benchmarking optimization software with performance profiles. Math. Program. Ser. B 91, 201–213 (2002). doi:10.1007/s101070100263

    Article  MATH  Google Scholar 

  13. Dollar, H.S., Gould, N.I.M., Schilders, W.H.A., Wathen, A.J.: Implicit-factorization preconditioning and iterative solvers for regularized saddle-point systems. SIAM J. Matrix Anal. Appl. 28(1), 170–189 (2006). doi:10.1137/05063427X

    Article  MathSciNet  MATH  Google Scholar 

  14. Elman, H., Silvester, D., Wathen, A.: Finite Elements and Fast Iterative Solvers with Applications in Incompressible Fluid Dynamics. Numerical Mathematics and Scientific Computation. Oxford University Press, Oxford (2005)

    Google Scholar 

  15. Forsgren, A., Gill, Ph. E., Wright, M.H.: Interior methods for nonlinear optimization. SIAM Rev. 44(4), 525–597 (2002). doi:10.1137/S0036144502414942

    Article  MathSciNet  MATH  Google Scholar 

  16. Friedlander, M.P., Orban, D.: A primal-dual regularized interior-point method for convex quadratic programs. Math. Program. Comput. 4(1), 71–107 (2012). doi:10.1007/s12532-012-0035-2

    Article  MathSciNet  MATH  Google Scholar 

  17. Friedlander, M.P., van Den Berg, E.: Spot—a linear-operator toolbox (2013). www.cs.ubc.ca/labs/scl/spot

  18. Gill, P.E., Saunders, M.A., Shinnerl, J.R.: On the stability of Cholesky factorization for symmetric quasidefinite systems. SIAM J. Optim. 17(1), 35–46 (1996). doi:10.1137/S0895479893252623

    MathSciNet  MATH  Google Scholar 

  19. Golub, G.H., Van Loan, C.F., 3rd: Matrix Computations. Johns Hopkins Series in the Mathematical Sciences. Johns Hopkins, Baltimore (1996)

    Google Scholar 

  20. Gould, N.I.M., Orban, D., Toint, Ph. L.: CUTEr and SifDec, a Constrained and Unconstrained Testing Environment, revisited. ACM Trans. Math. Softw. 29(4), 373–394 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  21. Gould, N.I.M., Orban, D., Toint, Ph. L.: Numerical methods for large-scale nonlinear optimization. Acta Numerica 14, 299–361 (2005). doi:10.1017/S0962492904000248

    Article  MathSciNet  MATH  Google Scholar 

  22. Greif, C., He, S., Liu, P.: sym-ildl: Incomplete L D L T factorizations of indefinite symmetric matrices (2013). URL www.cs.ubc.ca/inutard/html

  23. Greif, C., Moulding, E., Orban, D.: Bounds on the eigenvalues of block matrices arising from interior-point methods. SIAM J. Optim. 24(1), 49–83 (2014). doi:10.1137/120890600

    Article  MathSciNet  MATH  Google Scholar 

  24. Gustafsson, I.: A class of first order factorization methods. BIT 18(2), 142–156 (1978). doi:10.1007/BF01931691

    Article  MathSciNet  MATH  Google Scholar 

  25. Higham, N.J., 2nd: Accuracy and Stability of Numerical Algorithms. Number OT80 in Other Titles in Applied Mathematics. SIAM, Philadelphia (2002)

    Book  Google Scholar 

  26. HSL: A collection of Fortran codes for large-scale scientific computation. www.hsl.rl.ac.uk (2013)

  27. Jones, M.T., Plassmann, P.E.: An improved incomplete Cholesky factorization. ACM Trans. Math. Softw. 21(1), 5–17 (1995). doi:10.1145/200979.200981

    Article  MathSciNet  MATH  Google Scholar 

  28. Kaporin, I.E.: High quality preconditioning of a general symmetric positive definite matrix based on its \(U^{T U + U^{T} R + R^{T} U}\) decomposition. Numer. Linear Algebra Appl. 5(6), 483–509 (1998). doi:10.1002/(SICI)1099-1506(199811/12)5:6<483::AID-NLA156>3.0.CO;2-7

    Article  MathSciNet  MATH  Google Scholar 

  29. Li, N., Saad, Y., Chow, E.: Crout versions of ILU for general sparse matrices. SIAM J. Sci. Comput. 25(2), 716–728 (2003). doi:10.1137/S1064827502405094

    Article  MathSciNet  Google Scholar 

  30. Li, N., Saad, Y.: Crout versions of ILU factorization with pivoting for sparse symmetric matrices. Electron. Trans. Numer. Anal. 20, 75–85 (2005)

    MathSciNet  Google Scholar 

  31. Lin, C.-J., Moré, J.J.: Incomplete Cholesky factorizations with limited memory. SIAM J. Sci. Comput. 21(1), 24–45 (1999). doi:10.1137/S1064827597327334

    Article  MathSciNet  MATH  Google Scholar 

  32. Manteuffel, T.A.: Shifted incomplete Cholesky factorization. In: Duff, I. S., Stewart, G. W. (eds.) Sparse Matrix Proceedings, pp. 41–61. SIAM, Philadelphia (1979)

    MATH  Google Scholar 

  33. Manteuffel, T.A.: An incomplete factorization technique for positive definite linear systems. Math. Comput. 34(150), 473–497 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  34. Maros, I., Mészáros, C.: A repository of convex quadratic programming problems. Optim. Methods Softw. 11 & 12, 671–681 (1999). Special Issue on Interior Point Methods.

    Article  MATH  Google Scholar 

  35. Mehrotra, S.: On the implementation of a primal-dual interior point method. SIAM J. Optim. 2(4), 575–601 (1992). doi:10.1137/0802028

    Article  MathSciNet  MATH  Google Scholar 

  36. Meijerink, J.A., van der Vorst, H.A.: An iterative solution method for linear equations systems of which the coefficient matrix is a symmetric M-matrix. Math. Comput. 137, 148–162 (1977)

    MathSciNet  Google Scholar 

  37. Messaoudi, A.: On the stability of the incomplete LU-factorizations and characterizations of H-matrices. Numer. Math. 69(3), 321–331 (1995). doi:10.1007/s002110050095

    Article  MathSciNet  MATH  Google Scholar 

  38. Munksgaard, N.: Solving sparse symmetric sets of linear equations by preconditioned conjugate gradients. ACM Trans. Math. Softw. 6(2), 206–219 (1980). doi:10.1145/355887.355893

    Article  MATH  Google Scholar 

  39. Ortega, J.M.: Introduction to Parallel and Vector Solution of Linear Systems. Fontiers of Computer Science. Springer Science+Business Media, New York (1988). Originally published by Plenum Press

    Book  Google Scholar 

  40. Paige, C.C., Saunders, M.A.: Solution of sparse indefinite systems of linear equations. SIAM J. Numer. Anal. 12(4), 617–629 (1975). doi:10.1137/0712047

    Article  MathSciNet  Google Scholar 

  41. Saad, Y.: ILUT: A dual threshdol incomplete LU factorization. Numer Linear Algebra Appl 1(4), 387–402 (1994). doi:10.1002/nla.1680010405

    Article  MathSciNet  MATH  Google Scholar 

  42. Saunders, M.A.: PDCO—primal-dual interior methods. Class notes #7 for Large-Scale Numerical Optimization, MS&amp;E 318/CME 338, Stanford University, Spring 2013. http://web.stanford.edu/group/SOL/software/pdco

  43. Scott, J., Tůma, M.: HSL_MI28: an efficient and robust limited-memory incomplete Cholesky factorization code. Preprint RAL-P-2013-004. Rutherford Appleton Laboratory, Didcot (2013)

    Google Scholar 

  44. Tismenetsky, M.: A new preconditioning technique for solving large sparse linear systems. Linear Algebra Appl. 154–156, 331–353 (1991). doi:10.1016/0024-3795(91)90383-8

    Article  MathSciNet  Google Scholar 

  45. Tůma, M.: A note on the L D L T decomposition of matrices from saddle-point problems. SIAM J. Matrix Anal. Appl. 23(4), 903–915 (2002). doi:10.1137/S0895479897321088

    Article  MathSciNet  MATH  Google Scholar 

  46. Vanderbei, R.J.: Symmetric quasi-definite matrices. SIAM J. Optim. 5(1), 100–113 (1995). doi:10.1137/0805005

    Article  MathSciNet  MATH  Google Scholar 

  47. Vanderbei, R.J., Shanno, D.F.: An interior point algorithm for nonconvex nonlinear programming. Comput. Optim. Appl. 13(1–3), 231–252 (1999). doi:10.1023/A:1008677427361

    Article  MathSciNet  MATH  Google Scholar 

  48. Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. (2006). doi:10.1007/s10107-004-0559-y

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Dominique Orban.

Additional information

Research partially supported by an NSERC Discovery Grant.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Orban, D. Limited-memory LDL factorization of symmetric quasi-definite matrices with application to constrained optimization. Numer Algor 70, 9–41 (2015). https://doi.org/10.1007/s11075-014-9933-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-014-9933-x

Keywords

Mathematics Subject Classifications (2000)

Navigation