Abstract
In this paper, we present a primal-dual interior-point method for solving nonlinear programming problems. It employs a Levenberg-Marquardt (LM) perturbation to the Karush-Kuhn-Tucker (KKT) matrix to handle indefinite Hessians and a line search to obtain sufficient descent at each iteration. We show that the LM perturbation is equivalent to replacing the Newton step by a cubic regularization step with an appropriately chosen regularization parameter. This equivalence allows us to use the favorable theoretical results of Griewank (The modification of Newton’s method for unconstrained optimization by bounding cubic terms, 1981), Nesterov and Polyak (Math. Program., Ser. A 108:177–205, 2006), Cartis et al. (Math. Program., Ser. A 127:245–295, 2011; Math. Program., Ser. A 130:295–319, 2011), but its application at every iteration of the algorithm, as proposed by these papers, is computationally expensive. We propose a hybrid method: use a Newton direction with a line search on iterations with positive definite Hessians and a cubic step, found using a sufficiently large LM perturbation to guarantee a steplength of 1, otherwise. Numerical results are provided on a large library of problems to illustrate the robustness and efficiency of the proposed approach on both unconstrained and constrained problems.








Similar content being viewed by others
References
Benson, H.Y., Shanno, D.F., Vanderbei, R.J.: Interior-point methods for nonconvex nonlinear programming: filter methods and merit functions. Comput. Optim. Appl. 23(2), 257–272 (2002)
Benson, H.Y., Sen, A., Shanno, D.F., Vanderbei, R.J.: Interior point algorithms, penalty methods and equilibrium problems. Comput. Optim. Appl. 34(2), 155–182 (2006)
Bongartz, I., Conn, A.R., Gould, N.I.M., Toint, P.: CUTE: Constrained and unconstrained testing environment. ACM Trans. Math. Softw. 21(1), 123–160 (1995)
Byrd, R.H., Nocedal, J., Waltz KNITRO, R.A.: An integrated package for nonlinear optimization. In: Di Pillo, G., Roma, M. (eds.) Large-Scale Nonlinear Optimization, pp. 35–59. Springer, New York (2006)
Cartis, C., Gould, N.I.M., Toint, Ph.L.: Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results. Math. Program., Ser. A 127, 245–295 (2011)
Cartis, C., Gould, N.I.M., Toint, Ph.L.: Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity. Math. Program., Ser. A 130, 295–319 (2011)
Cartis, C., Gould, N.I.M., Toint, Ph.L.: Evaluation copmlexity of adaptive cubic regularization methods for convex unconstrained optimization. Optim. Methods Softw., iFirst:1–23 (2011)
Cartis, C., Gould, N.I.M., Toint, Ph.L.: An adaptive cubic regularisation algorithm for nonconvex optimization with convex constraints and its function-evaluation complexity. IMA J. Numer. Anal. 32(4), 1662–1695 (2012)
Conn, A.R., Gould, N., Toint, Ph.L.: Constrained and unconstrained testing environment. http://www.dci.clrc.ac.uk/Activity.asp?CUTE
Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Technical report, Argonne National Laboratory (2001)
Fletcher, R., Leyffer, S.: User manual for filterSQP. Technical Report NA-181, University of Dundee Report (1998)
Fourer, R., Gay, D.M., Kernighan, B.W.: AMPL: a Modeling Language for Mathematical Programming. Scientific Press, San Francisco (1993)
Gill, P.E., Murray, W., Saunders SNOPT, M.A.: An SQP algorithm for large-scale constrained optimization. SIAM J. Optim. 12, 979–1006 (2002)
Gould, N.I.M., Robinson, D.P., Thorne, H.S.: On solving trust-region and other regularised subproblems in optimization. Math. Program. Comput. 2(1), 21–57 (2010)
Gould, N.I.M., Porcelli, M., Toint, Ph.L.: Updating the regularization parameter in the adaptive cubic regularization algorithm. Comput. Optim. Appl. 53(1), 1–22 (2012)
Griewank, A.: The modification of Newton’s method for unconstrained optimization by bounding cubic terms. Technical Report NA/12, Department of Applied Mathematics and Theoretical Physics, University of Cambridge (1981)
Hock, W., Schittkowski, K.: Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems, vol. 187. Springer, Heidelberg (1981)
Hogg, J.D., Scott, J.A.: A note on the solve phase of a multicore solver. Technical Report RAL-TR-2010-007, Rutherford Appleton Laboratory, Chilton, Oxfordshire, England (2010)
Levenberg, K.: A method for the solution of certain problems in least squares. Q. Appl. Math. 2, 164–168 (1944)
Marquardt, D.: An algorithm for least-squares estimation of nonlinear parameters. SIAM J. Appl. Math. 11, 431–441 (1963)
Morrison, D.D.: Methods for nonlinear least-squares problems and convergence proofs. In: Lorell, J., Yagi, F. (eds.) Proceedings of the Seminar on Tracking Programs and Orbit Determination, Pasadena, USA, pp. 1–9 (1960). Jet Propulsion Laboratory
Nesterov, Yu.: Cubic regularization of Newton’s method for convex problems with constraints. Technical Report 39, CORE (2006)
Nesterov, Yu.: Accelerating the cubic regularization of Newton’s method on convex problems. Math. Program., Ser. B 112, 159–181 (2008)
Nesterov, Yu., Polyak, B.T.: Cubic regularization of Newton method and its global performance. Math. Program., Ser. A 108, 177–205 (2006)
Shanno, D.F., Vanderbei, R.J.: Interior-point methods for nonconvex nonlinear programming: orderings and higher-order methods. Math. Program. 87(2), 303–316 (2000)
Vanderbei, R.J.: LOQO user’s manual. Technical Report SOR 92-5, Princeton University, (1992). Revised 1995
Vanderbei, R.J.: AMPL models. http://orfe.princeton.edu/~rvdb/ampl/nlmodels
Vanderbei, R.J., Shanno, D.F.: An interior-point algorithm for nonconvex nonlinear programming. Comput. Optim. Appl. 13, 231–252 (1999)
Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program., Ser. A 106(1), 25–57 (2006)
Weiser, M., Deuflhard, P., Erdmann, B.: Affine conjugate adaptive Newton methods for nonlinear elastomechanics. Optim. Methods Softw. 22(3), 413–431 (2007)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Benson, H.Y., Shanno, D.F. Interior-point methods for nonconvex nonlinear programming: cubic regularization. Comput Optim Appl 58, 323–346 (2014). https://doi.org/10.1007/s10589-013-9626-8
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10589-013-9626-8