Skip to main content
SpringerLink
Log in

Cubic-regularization counterpart of a variable-norm trust-region method for unconstrained minimization

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

In a recent paper, we introduced a trust-region method with variable norms for unconstrained minimization, we proved standard asymptotic convergence results, and we discussed the impact of this method in global optimization. Here we will show that, with a simple modification with respect to the sufficient descent condition and replacing the trust-region approach with a suitable cubic regularization, the complexity of this method for finding approximate first-order stationary points is \(O(\varepsilon ^{-3/2})\). We also prove a complexity result with respect to second-order stationarity. Some numerical experiments are also presented to illustrate the effect of the modification on practical performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bianconcini, T., Liuzzi, G., Morini, B., Sciandrone, M.: On the use of iterative methods in cubic regularization for unconstrained optimization. Comput. Optim. Appl. 60(1), 35–57 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  2. Birgin, E.G., Gardenghi, J.L., Martínez, J.M., Santos, S.A., Toint, PhL: Worst-Case Evaluation Complexity for Unconstrained Nonlinear Optimization using high-order regularized models, Technical Report naXys-05-2015, Namur Center for Complex Systems (naXys). University of Namur, Namur (2015)

    Google Scholar 

  3. Birgin, E.G., Martínez, J.M.: Practical Augmented Lagrangian Methods for Constrained Optimization. SIAM, Philadelphia (2014)

    Book  MATH  Google Scholar 

  4. Birgin, E.G., Martínez, J.M., Raydan, M.: Spectral projected gradient methods: review and perspectives. J. Stat. Softw. 60(3) (2014)

  5. Cartis, C., Gould, N.I.M., Toint, PhL: On the complexity of steepest descent, Newton’s and regularized Newton’s methods for nonconvex unconstrained optimization. SIAM J. Optim. 20, 2833–2852 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  6. Cartis, C., Gould, N.I.M., Toint, PhL: Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results. Math. Program. Ser. A 127, 245–295 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  7. Cartis, C., Gould, N.I.M., Toint, PhL: Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity. Math. Program. Ser. A 130, 295–319 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  8. Celis, M.R., Dennis, J.E., Tapia, R.A.: A trust-region strategy for nonlinear equality constrained optimization. In: Boggs, P., Byrd, R., Schnabel, R., Publications, S.I.A.M. (eds.) Numerical Optimization, pp. 71–82. SIAM Publications, Philadelphia (1985)

    Google Scholar 

  9. Curtis, F.E., Robinson, D.P., Samadi, M.: A trust-region algorithm with a worst-case iteration complexity of \(O(\varepsilon ^{-3/2})\) for nonconvex optimization. Math. Program. (2016). doi:10.1007/s10107-016-1026-2

    Google Scholar 

  10. Dennis, J.E., El-Alem, M., Maciel, M.C.: A global convergence theory for general trust-region-based algorithms for equality constrained optimization. SIAM J. Optim. 7, 177–207 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  11. Dussault,J.,P.: Simple unified convergence proofs for the trust-region methods and a new ARC variant, Technical Report, University of Sherbrooke, Sherbrooke, Canada (2015)

  12. El-Alem, M.: A robust trust region algorithm with a nonmonotonic penalty parameter scheme for constrained optimization. SIAM J. Optim. 5, 348–378 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  13. Gomes, F.M., Maciel, M.C., Martínez, J.M.: Nonlinear programming algorithms using trust regions and augmented Lagrangians with nonmonotone penalty parameters. Math. Program. 84, 161–200 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  14. Grapiglia, G.N., Yuan, J., Yuan, Y.-X.: On the convergence and worst-case complexity of trust-region and regularization methods for unconstrained optimization. Math. Program. 152, 491–520 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  15. Griewank, A.: The modification of Newton’s method for unconstrained optimization by bounding cubic terms, Technical Report NA/12. University of Cambridge, Department of Applied Mathematics and Theoretical Physics (1981)

  16. Gould, N.I.M., Porcelli, M., Toint, PhL: Updating the regularization parameter in the adaptive cubic regularization algorithm. Comput. Optim. Appl. 53, 1–22 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  17. Karas, E.W., Santos, S.A., Svaiter, B.F.: Algebraic rules for quadratic regularization of Newton’s method. Comput. Optim. Appl. 60(2), 343–376 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  18. Lu, S., Wei, Z., Li, L.: A trust region algorithm with adaptive cubic regularization methods for nonsmooth convex minimization. Comput. Optim. Appl. 51, 551–573 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  19. Martínez, J.M.: Inexact restoration method with Lagrangian tangent decrease and new merit function for nonlinear programming. J. Optim. Theory Appl. 111, 39–58 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  20. Martínez, J.M., Raydan, M.: Separable cubic modeling and a trust-region strategy for unconstrained minimization with impact in global optimization. J. Glob. Optim. 63(2), 319–342 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  21. Nesterov, Y., Polyak, B.T.: Cubic regularization of Newton’s method and its global performance. Math. Program. 108(1), 177–205 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  22. Nesterov, Y.: Accelerating the cubic regularization of Newton’s method on convex problems. Math. Program. Ser. B 112, 159–181 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  23. Toint, P.L.: Private communication (2015)

Download references

Acknowledgments

We thank two referees for carefully reading the paper and for many constructive comments and suggestions.

Author information

Authors and Affiliations

  1. Department of Applied Mathematics, IMECC-UNICAMP, University of Campinas, Rua Sérgio Buarque de Holanda, 651 Cidade Universitária “Zeferino Vaz”, Distrito Barão Geraldo, Campinas, SP, 13083-859, Brazil

    J. M. Martínez

  2. Departamento de Cómputo Científico y Estadística, Universidad Simón Bolívar, Ap. 89000, Caracas, 1080, Venezuela

    M. Raydan

Authors
  1. J. M. Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Raydan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Raydan.

Additional information

This work was supported by PRONEX-CNPq/FAPERJ (E-26/111.449/2010-APQ1), CEPID–Industrial Mathematics/FAPESP (Grant 2011/51305-02), FAPESP (Projects 2013/05475-7 and 2013/07375-0), and CNPq (Project 400926/2013-0).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Martínez, J.M., Raydan, M. Cubic-regularization counterpart of a variable-norm trust-region method for unconstrained minimization. J Glob Optim 68, 367–385 (2017). https://doi.org/10.1007/s10898-016-0475-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-016-0475-8

Keywords

Search

Navigation