Skip to main content

Advertisement

Log in

Interior-point methods for nonconvex nonlinear programming: cubic regularization

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

In this paper, we present a primal-dual interior-point method for solving nonlinear programming problems. It employs a Levenberg-Marquardt (LM) perturbation to the Karush-Kuhn-Tucker (KKT) matrix to handle indefinite Hessians and a line search to obtain sufficient descent at each iteration. We show that the LM perturbation is equivalent to replacing the Newton step by a cubic regularization step with an appropriately chosen regularization parameter. This equivalence allows us to use the favorable theoretical results of Griewank (The modification of Newton’s method for unconstrained optimization by bounding cubic terms, 1981), Nesterov and Polyak (Math. Program., Ser. A 108:177–205, 2006), Cartis et al. (Math. Program., Ser. A 127:245–295, 2011; Math. Program., Ser. A 130:295–319, 2011), but its application at every iteration of the algorithm, as proposed by these papers, is computationally expensive. We propose a hybrid method: use a Newton direction with a line search on iterations with positive definite Hessians and a cubic step, found using a sufficiently large LM perturbation to guarantee a steplength of 1, otherwise. Numerical results are provided on a large library of problems to illustrate the robustness and efficiency of the proposed approach on both unconstrained and constrained problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Algorithm 1
Algorithm 2
Algorithm 3
Algorithm 4
Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Benson, H.Y., Shanno, D.F., Vanderbei, R.J.: Interior-point methods for nonconvex nonlinear programming: filter methods and merit functions. Comput. Optim. Appl. 23(2), 257–272 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  2. Benson, H.Y., Sen, A., Shanno, D.F., Vanderbei, R.J.: Interior point algorithms, penalty methods and equilibrium problems. Comput. Optim. Appl. 34(2), 155–182 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  3. Bongartz, I., Conn, A.R., Gould, N.I.M., Toint, P.: CUTE: Constrained and unconstrained testing environment. ACM Trans. Math. Softw. 21(1), 123–160 (1995)

    Article  MATH  Google Scholar 

  4. Byrd, R.H., Nocedal, J., Waltz KNITRO, R.A.: An integrated package for nonlinear optimization. In: Di Pillo, G., Roma, M. (eds.) Large-Scale Nonlinear Optimization, pp. 35–59. Springer, New York (2006)

    Chapter  Google Scholar 

  5. Cartis, C., Gould, N.I.M., Toint, Ph.L.: Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results. Math. Program., Ser. A 127, 245–295 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  6. Cartis, C., Gould, N.I.M., Toint, Ph.L.: Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity. Math. Program., Ser. A 130, 295–319 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  7. Cartis, C., Gould, N.I.M., Toint, Ph.L.: Evaluation copmlexity of adaptive cubic regularization methods for convex unconstrained optimization. Optim. Methods Softw., iFirst:1–23 (2011)

    Google Scholar 

  8. Cartis, C., Gould, N.I.M., Toint, Ph.L.: An adaptive cubic regularisation algorithm for nonconvex optimization with convex constraints and its function-evaluation complexity. IMA J. Numer. Anal. 32(4), 1662–1695 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  9. Conn, A.R., Gould, N., Toint, Ph.L.: Constrained and unconstrained testing environment. http://www.dci.clrc.ac.uk/Activity.asp?CUTE

  10. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Technical report, Argonne National Laboratory (2001)

  11. Fletcher, R., Leyffer, S.: User manual for filterSQP. Technical Report NA-181, University of Dundee Report (1998)

  12. Fourer, R., Gay, D.M., Kernighan, B.W.: AMPL: a Modeling Language for Mathematical Programming. Scientific Press, San Francisco (1993)

    Google Scholar 

  13. Gill, P.E., Murray, W., Saunders SNOPT, M.A.: An SQP algorithm for large-scale constrained optimization. SIAM J. Optim. 12, 979–1006 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  14. Gould, N.I.M., Robinson, D.P., Thorne, H.S.: On solving trust-region and other regularised subproblems in optimization. Math. Program. Comput. 2(1), 21–57 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  15. Gould, N.I.M., Porcelli, M., Toint, Ph.L.: Updating the regularization parameter in the adaptive cubic regularization algorithm. Comput. Optim. Appl. 53(1), 1–22 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  16. Griewank, A.: The modification of Newton’s method for unconstrained optimization by bounding cubic terms. Technical Report NA/12, Department of Applied Mathematics and Theoretical Physics, University of Cambridge (1981)

  17. Hock, W., Schittkowski, K.: Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems, vol. 187. Springer, Heidelberg (1981)

    MATH  Google Scholar 

  18. Hogg, J.D., Scott, J.A.: A note on the solve phase of a multicore solver. Technical Report RAL-TR-2010-007, Rutherford Appleton Laboratory, Chilton, Oxfordshire, England (2010)

  19. Levenberg, K.: A method for the solution of certain problems in least squares. Q. Appl. Math. 2, 164–168 (1944)

    MATH  MathSciNet  Google Scholar 

  20. Marquardt, D.: An algorithm for least-squares estimation of nonlinear parameters. SIAM J. Appl. Math. 11, 431–441 (1963)

    Article  MATH  MathSciNet  Google Scholar 

  21. Morrison, D.D.: Methods for nonlinear least-squares problems and convergence proofs. In: Lorell, J., Yagi, F. (eds.) Proceedings of the Seminar on Tracking Programs and Orbit Determination, Pasadena, USA, pp. 1–9 (1960). Jet Propulsion Laboratory

    Google Scholar 

  22. Nesterov, Yu.: Cubic regularization of Newton’s method for convex problems with constraints. Technical Report 39, CORE (2006)

  23. Nesterov, Yu.: Accelerating the cubic regularization of Newton’s method on convex problems. Math. Program., Ser. B 112, 159–181 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  24. Nesterov, Yu., Polyak, B.T.: Cubic regularization of Newton method and its global performance. Math. Program., Ser. A 108, 177–205 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  25. Shanno, D.F., Vanderbei, R.J.: Interior-point methods for nonconvex nonlinear programming: orderings and higher-order methods. Math. Program. 87(2), 303–316 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  26. Vanderbei, R.J.: LOQO user’s manual. Technical Report SOR 92-5, Princeton University, (1992). Revised 1995

  27. Vanderbei, R.J.: AMPL models. http://orfe.princeton.edu/~rvdb/ampl/nlmodels

  28. Vanderbei, R.J., Shanno, D.F.: An interior-point algorithm for nonconvex nonlinear programming. Comput. Optim. Appl. 13, 231–252 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  29. Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program., Ser. A 106(1), 25–57 (2006)

    Article  MATH  Google Scholar 

  30. Weiser, M., Deuflhard, P., Erdmann, B.: Affine conjugate adaptive Newton methods for nonlinear elastomechanics. Optim. Methods Softw. 22(3), 413–431 (2007)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Hande Y. Benson.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Benson, H.Y., Shanno, D.F. Interior-point methods for nonconvex nonlinear programming: cubic regularization. Comput Optim Appl 58, 323–346 (2014). https://doi.org/10.1007/s10589-013-9626-8

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-013-9626-8

Keywords

Navigation