Skip to main content
SpringerLink
Log in

On the use of iterative methods in cubic regularization for unconstrained optimization

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

In this paper we consider the problem of minimizing a smooth function by using the adaptive cubic regularized (ARC) framework. We focus on the computation of the trial step as a suitable approximate minimizer of the cubic model and discuss the use of matrix-free iterative methods. Our approach is alternative to the implementation proposed in the original version of ARC, involving a linear algebra phase, but preserves the same worst-case complexity count. Further we introduce a new stopping criterion in order to properly manage the “over-solving” issue arising whenever the cubic model is not an adequate model of the true objective function. Numerical experiments conducted by using a nonmonotone gradient method as inexact solver are presented. The obtained results clearly show the effectiveness of the new variant of ARC algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. The collection is available at http://www.cs.cas.cz/luksan/test.html

References

  1. Bellavia, S., Morini, B.: Strong local convergence properties of adaptive regularized methods for nonlinear least-squares. IMA J. Numer. Anal. doi:10.1093/imanum/dru021

  2. Bellavia, S., Cartis, C., Gould, N.I.M., Morini, B., Toint, PhL: Convergence of a regularized euclidean residual algorithm for nonlinear least-squares. SIAM J. Numer. Anal. 48, 1–29 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  3. Benson, H.Y., Shanno, D.F.: Interior-Point methods for nonconvex nonlinear programming: cubic regularization. Comput. Optim. Appl. 58, 323–346 (2014)

    Article  MATH  MathSciNet  Google Scholar 

  4. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Berlin (2006)

    MATH  Google Scholar 

  5. Cartis, C., Gould, N.I.M., Toint, PhL: On the complexity of steepest descent, Newton’s and regularized Newton’s methods for nonconvex unconstrained optimization. SIAM J. Optim. 20, 2833–2852 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  6. Cartis, C., Gould, N.I.M., Toint, PhL: Adaptive cubic overestimation methods for unconstrained optimization. Part I: motivation, convergence and numerical results. Math. Program. A 127, 245–295 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  7. Cartis, C., Gould, N.I.M., Toint, PhL: Adaptive cubic overestimation methods for unconstrained optimization. Part II: worst-case function-evaluation complexity. Math. Program. A 130, 295–319 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  8. Cartis, C., Gould, N.I.M., Toint, PhL: An adaptive cubic regularization algorithm for nonconvex optimization with convex constraints and its function-evaluation complexity. IMA J. Numer. Anal. 32, 1662–1695 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  9. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. Ser. A 91, 201–213 (2002)

    Article  MATH  Google Scholar 

  10. Gould, N.I.M., Orban, D., Toint, PhL: GALAHAD–a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization. ACM Trans. Math. Softw. 29, 353–372 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  11. Gould, N.I.M., Orban, D., Toint, PhL: CUTEr (and SifDec), a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29, 373–394 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  12. Gould, N.I.M., Porcelli, M., Toint, PhL: Updating the regularization parameter in the adaptive cubic regularization algorithm. Comput. Optim. Appl. 53, 1–22 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  13. Gould, N.I.M., Lucidi, S., Roma, M., Toint, PhL: Solving the trust-region subproblem using the Lanczos method. SIAM J. Optim. 9, 504–525 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  14. Griewank, A.: The Modification of Newton’s Method for Unconstrained Optimization by Bounding Cubic Terms. Technical Report NA, 12 (1981). Department of Applied Mathematics and Theoretical Physics, University of Cambridge, UK (1981)

    Google Scholar 

  15. Grippo, L., Sciandrone, M.: Nonmonotone globalization techniques for the Barzilai–Borwein gradient method. Comput. Optim. Appl. 23, 143–169 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  16. Nesterov, Yu.: Modified Gauss–Newton scheme with worst-case guarantees for global performance. Optim. Methods Softw. 22, 469–483 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  17. Nesterov, Yu., Polyak, B.T.: Cubic regularization of Newton’s method and its global performance. Math. Program. 108, 177–205 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  18. Toint, L.: Nonlinear stepsize control, trust regions and regularizations for unconstrained optimization. Optim. Methods Softw. 28, 82–95 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  19. Weiser, M., Deuflhard, P., Erdmann, B.: Affine conjugate adaptive Newton methods for nonlinear elastomechanics. Optim. Methods Softw. 22, 413–431 (2007)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

The authors are indebted with Prof. Philippe L. Toint for his valuable insights and suggestions on the worst-case complexity of ARC algorithms that led to significant improvements of the paper. The authors are also very grateful to the referees for their constructive remarks that had a significant influence on the revised version of this work.Work partially supported by INdAM-GNCS under the 2014 Project Metodi di regolarizzazione per problemi di ottimizzazione vincolata.

Author information

Authors and Affiliations

  1. Dipartimento di Ingegneria dell’Informazione, Via di S. Marta 3, 50139 , Florence, Italia

    Tommaso Bianconcini & Marco Sciandrone

  2. KKT srl, Viale Mazzini 40, 50132 , Florence, Italia

    Tommaso Bianconcini

  3. Istituto di Analisi dei Sistemi ed Informatica “Antonio Ruberti” (IASI) Consiglio Nazionale delle Ricerche (CNR), Viale Manzoni 30, 00185 , Rome, Italia

    Giampaolo Liuzzi

  4. Dipartimento di Ingegneria Industriale, Università degli Studi di Firenze, viale Morgagni 40, 50134 , Florence, Italia

    Benedetta Morini

Authors
  1. Tommaso Bianconcini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giampaolo Liuzzi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Benedetta Morini
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marco Sciandrone
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Benedetta Morini.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (tex 46 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bianconcini, T., Liuzzi, G., Morini, B. et al. On the use of iterative methods in cubic regularization for unconstrained optimization. Comput Optim Appl 60, 35–57 (2015). https://doi.org/10.1007/s10589-014-9672-x

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-014-9672-x

Keywords

Search

Navigation