Skip to main content
SpringerLink
Log in

Using Improved Directions of Negative Curvature for the Solution of Bound-Constrained Nonconvex Problems

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

In this work, we describe the efficient use of improved directions of negative curvature for the solution of bound-constrained nonconvex problems. We follow an interior-point framework, in which the key point is the inclusion of computational low-cost procedures to improve directions of negative curvature obtained from a factorisation of the KKT matrix. From a theoretical point of view, it is well known that these directions ensure convergence to second-order KKT points. As a novelty, we consider the convergence rate of the algorithm with exploitation of negative curvature information. Finally, we test the performance of our proposal on both CUTEr/st and simulated problems, showing empirically that the enhanced directions affect positively the practical performance of the procedure.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. It is important to remark that, although MCF is a procedure to compute DNC, we always force \(d_n^k = 0\) for this choice of the \(\mathbf {IA}\) algorithm. This is equivalent to an MCF method, without including DNC explicitly.

  2. Note that our goal is not to compare results obtained from both methods. Rather, we aim to show empirical performance gains are achieved by including the enhanced order information within an optimisation algorithm.

  3. By “better minimum” we mean a significant relative reduction in the value of f(x).

  4. Note that good CT values are obtained for \(\mathbf {IA}_{\mathbf {MCF}}^{(\infty )}\) due to the use of built-in MATLAB function eig for eigenvalue decomposition, especially efficient for small problems.

References

  1. Bazaraa, M.S., Sherali, H.D., Shetty, C.M.: Nonlinear Programming: Theory and Algorithms, 3rd edn. Wiley, Hoboken, New Jersey (2006)

    Book  MATH  Google Scholar 

  2. Fiacco, A.V., McCormick, G.P.: Nonlinear Programming: Sequential Unconstrained Minimization Techniques. Research Analysis Corporation, McLean. Reprinted by SIAM in 1990, Mc Lean, Virginia (1968)

  3. Gill, P.E., Murray, W.: Newton-type methods for unconstrained and linearly constrained optimization. Math. Progr. 7(1), 311–350 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  4. Fletcher, R., Freeman, T.L.: A modified Newton method for minimization. J. Optim. Theory Appl. 23(3), 357–372 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  5. Mukai, H., Polak, E.: A second order algorithm for the general nonlinear programming problem. J. Optim. Theory Appl. 4(26), 515–532 (1978)

    Article  MATH  Google Scholar 

  6. McCormick, G.: A modification of Armijo’s step-size rule for negative curvature. Math. Progr. 13(1), 111–115 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  7. Moré, J.J., Sorensen, D.C.: On the use of directions of negative curvature in a modified Newton method. Math. Progr. 16(1), 1–20 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  8. Goldfarb, D.: Curvilinear path steplength algorithms for minimization which use directions of negative curvature. Math. Progr. 18(1), 31–40 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  9. Moguerza, J.M., Prieto, F.J.: An augmented Lagrangian interior-point method using directions of negative curvature. Math. Progr. 95(3), 573–616 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  10. Sanmatías, S., Vercher, E.: A generalized conjugate gradient algorithm. J. Optim. Theory Appl. 98(2), 489–502 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  11. Moguerza, J.M., Prieto, F.J.: Combining search directions using gradient flows. Math. Progr. 96(3), 529–559 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  12. Hager, W.W., Zhang, H.: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16(1), 170–192 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  13. Sun, J., Yang, X., Chen, X.: Quadratic cost flow and the conjugate gradient method. Eur. J. Oper. Res. 164(1), 104–114 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  14. Andreani, R., Birgin, E.G., Martínez, J.M., Schuverdt, M.L.: Second-order negative-curvature methods for box-constrained and general constrained optimization. Comput. Optim. Appl. 45(2), 209–236 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  15. Murray, W., Ng, K.M.: An algorithm for nonlinear optimization problems with binary variables. Comput. Optim. Appl. 47(2), 257–288 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  16. Cartis, C., Gould, N.I.M., Toint, P.L.: Complexity bounds for second-order optimality in unconstrained optimization. J. Complex 28(1), 93–108 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  17. Curtis, F.E., Robinson, D.P.: Exploiting negative curvature directions in stochastic optimization (2017). arXiv:1703.00412

  18. Gill, P.E., Kungurtsev, V., Robinson, D.P.: A stabilized SQP method: global convergence. IMA J. Numer. Anal. 37, 407–443 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  19. Gill, P.E., Kungurtsev, V., Robinson, D.P.: A stabilized SQP method: superlinear convergence. Math. Progr. 1–42 (2017)

  20. Bunch, J.R., Parlett, B.N.: Direct methods for solving symmetric indefinite systems of linear equations. SIAM J. Numer. Anal. 8(4), 639–655 (1971)

    Article  MathSciNet  MATH  Google Scholar 

  21. Gill, P.E., Murray, W., Wright, M.H.: Practical Optimization. Academic Press, Bingley, UK (1981)

    MATH  Google Scholar 

  22. Gould, N.I.M., Lucidi, S., Roma, M., Toint, P.L.: Exploiting negative curvature directions in linesearch methods for unconstrained optimization. Optim. Method Softw. 14(1), 75–98 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  23. Boman, E.G., Murray, W.: An iterative approach to computing directions of negative curvature. In: Proceedings of the Fifth Copper Mountain Conference on Iterative Methods (1998)

  24. Olivares, A., Moguerza, J.M.: Improving directions of negative curvature in an efficient manner. Ann. Oper. Res. 166(1), 183–201 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  25. Guo, C.H., Higham, N.J., Tisseur, F.: An improved arc algorithm for detecting definite Hermitian pairs. SIAM J. Matrix Anal. Appl. 31(3), 1131–1151 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  26. Mehrotra, S.: On the implementation of a primal-dual interior point method. SIAM J. Optim. 2(4), 575–601 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  27. Gay, D.M., Overton, M.L., Wright, M.H.: A primal-dual interior method for nonconvex nonlinear programming. In: Yuan, Y. (ed.) Advances in Nonlinear Programming, pp. 31–56. Kluwer Academic, Dordrecht (1998)

    Chapter  Google Scholar 

  28. Wright, M.: Some properties of the Hessian of the logarithmic barrier function. Math. Progr. 67(1), 265–295 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  29. Stewart, G.W., Sun, J.: Matrix Perturbation Theory. Academic Press, London, UK (1990)

    MATH  Google Scholar 

  30. Forsgren, A., Gill, P.E., Murray, W.: Computing modified Newton directions using a partial Cholesky factorization. SIAM J. Sci. Comput. 16(1), 139–150 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  31. Olivares, A., Moguerza, J.M., Prieto, F.J.: Nonconvex optimization using negative curvature within a modified linesearch. Eur. J. Oper. Res. 189(3), 706–722 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  32. Sanmatías, S., Roma, M.: Un método de búsqueda lineal con direcciones combinadas para la optimización irrestringida. In: Actas del XXVI Congreso Nacional de Estadística e Investigación Operativa (2001)

  33. Benchakroun, A., Dussault, J.P., Mansouri, A.: A two parameter mixed interior-exterior penalty algorithm. ZOR—Math. Methods Oper. Res. 41(1), 25–55 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  34. Dussault, J.P.: Numerical stability and efficiency of penalty algorithms. SIAM J. Numer. Anal. 32(1), 296–317 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  35. Forsgren, A., Gill, P.E.: Primal-dual interior methods for nonconvex nonlinear programming. SIAM J. Optim. 8(4), 1132–1152 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  36. Birgin, E.G., Haeser, G., Ramos, A.: Augmented Lagrangians with constrained subproblems and convergence to second-order stationary points. Optimization Online (December 2016). http://www.optimization-online.org/DB_HTML/2016/12/5758.html

  37. Gould, N.I.M., Orban, D., Toint, P.L.: CUTEr and SifDec: a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29(4), 373–394 (2003)

    Article  MATH  Google Scholar 

  38. Byrd, R.H., Schnabel, R.B., Shultz, G.A.: Approximate solution of the trust region problem by minimization over two-dimensional subspaces. Math. Progr. 40(1), 247–263 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  39. El-Bakry, A.S., Tapia, R.A., Tsuchiya, T., Zhang, Y.: On the formulation and theory of the Newton interior-point method for nonlinear programming. J. Optim. Theory Appl. 89(3), 507–541 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  40. Yamashita, H., Yabe, H.: Superlinear and quadratic convergence of some primal-dual interior point methods for constrained optimization. Math. Progr. 75(3), 377–397 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  41. Moguerza, J.M., Olivares, A., Prieto, F.J.: A note on the use of vector barrier parameters for interior-point methods. Eur. J. Oper. Res. 181(2), 571–585 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  42. The MathWorks: MATLAB Release 2016b. The MathWorks, Inc., Natick, Massachusetts, United States. (2016). http://www.mathworks.com/

  43. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Progr. 91(2), 201–213 (2002)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work has been partially supported by research projects GROMA (MTM2015-63710-P), PPI (RTC-2015-3580-7) and UNIKO (RTC-2015-3521-7), funded by the Spanish Ministry of Economy and Competitiveness, and the methaodos.org research group at URJC. This work was completed while the first author was visiting KTH, supported with grants from URJC’s postdoctoral programmes. We are also grateful to Alberto Olivares, from URJC, for his support with the code and helpful comments and discussions.

Author information

Authors and Affiliations

  1. Rey Juan Carlos University, Madrid, Spain

    Javier Cano & Javier M. Moguerza

  2. Carlos III University, Madrid, Spain

    Francisco J. Prieto

Authors
  1. Javier Cano
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Javier M. Moguerza
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Francisco J. Prieto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Javier Cano.

Additional information

Communicated by Alexey F. Izmailov.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cano, J., Moguerza, J.M. & Prieto, F.J. Using Improved Directions of Negative Curvature for the Solution of Bound-Constrained Nonconvex Problems. J Optim Theory Appl 174, 474–499 (2017). https://doi.org/10.1007/s10957-017-1137-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-017-1137-9

Keywords

Mathematics Subject Classification

Search

Navigation