Skip to main content
SpringerLink
Log in

A comparison of reduced and unreduced KKT systems arising from interior point methods

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

We address the iterative solution of KKT systems arising in the solution of convex quadratic programming problems. Two strictly related and well established formulations for such systems are studied with particular emphasis on the effect of preconditioning strategies on their relation. Constraint and augmented preconditioners are considered, and the choice of the augmentation matrix is discussed. A theoretical and experimental analysis is conducted to assess which of the two formulations should be preferred for solving large-scale problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Notes

  1. Throughout we use the hat symbol, \(\,\widehat{\,}\,\), for matrices corresponding to the nonsymmetric \(3\times 3\) formulation.

References

  1. Altman, A., Gondzio, J.: Regularized symmetric indefinite systems in interior point methods for linear and quadratic optimization. Optim. Methods Softw. 11, 275–302 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  2. Axelsson, O.: Preconditioners for regularized saddle point matrices. J. Numer. Math. 19(2), 91–112 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  3. Baryamureeba, V., Steihaug, T.: On the convergence of an inexact primal-dual interior point method for linear programming. In: Lirkov, I., Margenov, S., Wasniewski, J. (eds.) Proceedings of the 5th International Conference on Large-Scale Scientific Computing. Lecture Notes in Computer Science, vol. 3743, pp. 629–637. Springer, Berlin (2006)

  4. Bellavia, S.: An inexact interior point method. J. Optim. Theory Appl. 96, 109–121 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  5. Benzi, M., Golub, G.H., Liesen, J.: Numerical solution of saddle point problems. Acta Numer. 14, 1–137 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  6. Benzi, M., Olshanskii, M.A.: An augmented Lagrangian-based approach to the Oseen problem. SIAM J. Sci. Comput. 28, 2095–2113 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  7. Bergamaschi, L., Gondzio, J., Zilli, G.: Preconditioning indefinite systems in interior point methods for optimization. Comput. Optim. Appl. 28, 149–171 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  8. Cafieri, S., D’Apuzzo, M., De Simone, V., di Serafino, D.: Stopping criteria for inner iterations in inexact potential reduction methods: a computational study. Comput. Optim. Appl. 36, 165–193 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  9. Cao, Zhi-Hao: Augmentation block preconditioners for saddle point-type matrices with singular (1, 1) blocks. Numer. Linear Algebra Appl. 15, 515–533 (2008)

    Article  MathSciNet  Google Scholar 

  10. Castro, J., Cuesta, J.: Quadratic regularizations in an interior-point method for primal block-angular problems. Math. Program. 130, 415–445 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  11. D’Apuzzo, M., De Simone, V., di Serafino, D.: On mutual impact of linear algebra and large-scale optimization with focus on interior point methods. Comput. Optim. Appl. 45, 283–310 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  12. Dembo, R.S., Eisenstat, S.C., Steihaug, T.: Inexact newton methods. SIAM J. Numer. Anal. 19, 400–408 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  13. Dollar, H.S.: Constraint-style preconditioners for regularized saddle-point systems. SIAM J. Matrix Anal. Appl. 29, 672–684 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  14. Dollar, H.S., Gould, N.I.M., Schilders, W.H.A., Wathen, A.J.: Implicit-factorization preconditioning and iterative solvers for regularized saddle-point systems. SIAM J. Matrix Anal. Appl. 28, 170–189 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  15. Dollar, H.S., Gould, N.I.M., Schilders, W.H.A., Wathen, A.J.: Using constraint preconditioners with regularized saddle-point problems. Comput. Optim. Appl. 36, 249–270 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  16. Durazzi, C., Ruggiero, V.: Indefinitely preconditioned conjugate gradient method for large sparse equality and inequality constrained quadratic problems. Numer. Linear Algebra Appl. 10, 673–688 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  17. Elman, H.C., Silvester, D.J., Wathen, A.J.: Finite Elements and Fast Iterative Solvers. Oxford University Press, Oxford (2005)

    MATH  Google Scholar 

  18. Forsgren, A.: Inertia-controlling factorizations for optimization algorithms. Appl. Numer. Math. 43, 91–107 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  19. Forsgren, A., Gill, P.E., Griffin, J.D.: Iterative solution of augmented systems arising in interior methods. SIAM J. Optim. 18, 666–690 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  20. Forsgren, A., Gill, P.E., Shinnerl, J.R.: Stability of symmetric ill-conditioned systems arising in interior methods for constrained optimization. SIAM J. Optim. 17, 187–211 (1996)

    MathSciNet  MATH  Google Scholar 

  21. Friedlander, M.P., Orban, D.: A primal-dual regularization interior-point method for convex quadratic programs. Math. Program. Comput. 4, 71–107 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  22. Gondzio, J.: Convergence analysis of an inexact feasible interior point method for convex quadratic programming. SIAM J. Optim. 23, 1510–1527 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  23. Gould, N.I.M., Orban, D., Toint, PhL: Cuter (and sifdec), a contrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29, 373–394 (2003)

    Article  MATH  Google Scholar 

  24. Gould, N.I.M., Simoncini, V.: Spectral analysis of saddle point matrices with indefinite leading blocks. SIAM J. Matrix Anal. Appl. 31, 1152–1171 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  25. Greif, C., Moulding, E., Orban, D.: Bounds on eigenvalues of matrices arising from interior-point methods. SIAM J. Optim. 24, 49–83 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  26. Greif, C., Schötzau, D.: Preconditioners for saddle point linear systems with highly singular (1, 1) blocks. Electron. Trans. Numer. Anal. 22, 114–121 (2006)

    MathSciNet  MATH  Google Scholar 

  27. Higham, N.J.: Functions of Matrices: Theory and Computation. Society for Industrial and Applied Mathematics, Philadelphia, PA (2008)

    Book  MATH  Google Scholar 

  28. Keller, C., Gould, N.I.M., Wathen, A.J.: Constraint preconditioning for indefinite linear systems. SIAM J. Matrix Anal. Appl. 21, 1300–1317 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  29. Lukšan, L., Vlček, J.: Indefinitely preconditioned inexact Newton method for large sparse equality constrained nonlinear programming problems. Numer. Linear Algebra Appl. 5, 219–247 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  30. Maros, I., Meszaros, C.: A repository of convex quadratic programming problems. Optim. Methods Softw. 11&12, 671–681 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  31. Morini, B., Simoncini, V., Tani, M.: Spectral estimates for unreduced symmetric KKT systems arising from interior point methods. Numer. Linear Algebra Appl. 23, 776–800 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  32. Paige, C.C., Saunders, M.A.: Solution of sparse indefinite systems of linear equations. SIAM J. Numer. Anal. 12, 617–629 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  33. PDCO: Primal-Dual interior method for Convex Objectives. http://stanford.edu/group/software/pdco

  34. Rees, T., Greif, C.: A preconditioner for linear systems arising from interior point optimization methods. SIAM J. Sci. Comput. 29, 1992–2007 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  35. Rozložník, M., Simoncini, V.: Krylov subspace methods for saddle point problem with indefinite preconditioning. SIAM J. Matrix Anal. Appl. 24(2), 368–391 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  36. Saad, Y., Schultz, M.H.: GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 7, 856–869 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  37. Saunders, M.A.: Cholesky-based methods for sparse least-squares: the benefit of regularization. In: Adams, L., Nazareth, J.L. (eds.) Linear and Nonlinear Conjugate Gradient-Related Methods, pp. 92–100. SIAM, Philadelphia (1994)

    Google Scholar 

  38. Shen, S.Q., Huang, T.Z., Zhang, J.S.: Augmentation block triangular preconditioners for regularized saddle point problems. SIAM J. Matrix Anal. Appl. 33, 721–741 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  39. Wright, S.J.: Stability of linear equations solvers in interior-point methods. SIAM J. Matrix Anal. Appl. 16, 1287–1307 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  40. Wright, S.J.: Primal-Dual Interior-Point Methods. SIAM, Philadelphia (1997)

    Book  MATH  Google Scholar 

Download references

Acknowledgements

We wish to thank Michael Saunders for his many comments and suggestions on an earlier draft of this manuscript.

Author information

Authors and Affiliations

  1. Dipartimento di Matematica, Università di Bologna, Piazza di Porta S. Donato 5, 40127, Bologna, Italy

    Valeria Simoncini

  2. Dipartimento di Ingegneria Industriale, Università degli Studi di Firenze, viale G.B. Morgagni 40, 50134, Florence, Italy

    Benedetta Morini

  3. Dipartimento di Matematica, Università di Pavia, via A. Ferrata 1, 27100, Pavia, Italy

    Mattia Tani

Authors
  1. Benedetta Morini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Valeria Simoncini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mattia Tani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Benedetta Morini.

Additional information

Work partially supported by INdAM-GNCS under the 2015 Project Metodi di regolarizzazione per problemi di ottimizzazione e applicazioni.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Morini, B., Simoncini, V. & Tani, M. A comparison of reduced and unreduced KKT systems arising from interior point methods. Comput Optim Appl 68, 1–27 (2017). https://doi.org/10.1007/s10589-017-9907-8

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-017-9907-8

Keywords

Search

Navigation