Skip to main content
SpringerLink
Log in

A filter sequential adaptive cubic regularization algorithm for nonlinear constrained optimization

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

In this paper, we propose a filter sequential adaptive regularization algorithm using cubics (ARC) for solving nonlinear equality constrained optimization. Similar to sequential quadratic programming methods, an ARC subproblem with linearized constraints is considered to obtain a trial step in each iteration. Composite step methods and reduced Hessian methods are employed to tackle the linearized constraints. As a result, a trial step is decomposed into the sum of a normal step and a tangential step which is computed by a standard ARC subproblem. Then, the new iteration is determined by filter methods and ARC framework. The global convergence of the algorithm is proved under some reasonable assumptions. Preliminary numerical experiments and comparison results are reported.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust Region Methods. Society for Industrial and Applied Mathematics, Philedalphia (2000)

    Book  MATH  Google Scholar 

  2. Fletcher, R., Leyffer, S.: Nonlinear programming without a penalty function. Math. Program. 91(2, Ser. A), 239–269 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  3. Cartis, C., Gould, N.I.M., Toint, P.L.: Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results. Math. Program. 127(2, Ser. A), 245–295 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  4. Griewank, A.: The modification of Newton’s method for unconstrained optimization by bounding cubic terms. Technical report Technical report NA/12 (1981). Department of Applied Mathematics and Theoretical Physics, University of Cambridge (1981)

  5. Nesterov, Y., Polyak, B.T.: Cubic regularization of Newton method and its global performance. Math. Program. 108(1, Ser. A), 177–205 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  6. Cartis, C., Gould, N.I.M., Toint, P.L.: Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity. Math. Program. 130(2, Ser. A), 295–319 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  7. Bellavia, S., Gurioli, G., Morini, B.: Adaptive cubic regularization methods with dynamic inexact Hessian information and applications to finite-sum minimization. IMA J. Numer. Anal. 41(1), 764–799 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  8. Benson, H.Y., Shanno, D.F.: Cubic regularization in symmetric rank-1 quasi-Newton methods. Math. Program. Comput. 10(4), 457–486 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  9. Bergou, E.H., Diouane, Y., Gratton, S.: A line-search algorithm inspired by the adaptive cubic regularization framework and complexity analysis. J. Optim. Theory Appl. 178(3), 885–913 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  10. Bianconcini, T., Sciandrone, M.: A cubic regularization algorithm for unconstrained optimization using line search and nonmonotone techniques. Optim. Methods Softw. 31(5), 1008–1035 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  11. Cartis, C., Gould, N.I.M., Toint, P.L.: A concise second-order complexity analysis for unconstrained optimization using high-order regularized models. Optim. Methods Softw. 35(2), 243–256 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  12. Dehghan Niri, T., Heydari, M., Hosseini, M.M.: An improvement of adaptive cubic regularization method for unconstrained optimization problems. Int. J. Comput. Math. 98(2), 271–287 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  13. Dussault, J.-P.: ARCq: a new adaptive regularization by cubics. Optim. Methods Softw. 33(2), 322–335 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  14. Martínez, J.M., Raydan, M.: Cubic-regularization counterpart of a variable-norm trust-region method for unconstrained minimization. J. Global Optim. 68(2), 367–385 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  15. Park, S., Jung, S.H., Pardalos, P.M.: Combining stochastic adaptive cubic regularization with negative curvature for nonconvex optimization. J. Optim. Theory Appl. 184(3), 953–971 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  16. Lu, S., Wei, Z., Li, L.: A trust region algorithm with adaptive cubic regularization methods for nonsmooth convex minimization. Comput. Optim. Appl. 51(2), 551–573 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  17. Zhao, T., Liu, H., Liu, Z.: New subspace minimization conjugate gradient methods based on regularization model for unconstrained optimization. Numer. Algoritm. 87(4), 1501–1534 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  18. Bergou, E., Diouane, Y., Gratton, S.: On the use of the energy norm in trust-region and adaptive cubic regularization subproblems. Comput. Optim. Appl. 68(3), 533–554 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  19. Gould, N.I.M., Porcelli, M., Toint, P.L.: Updating the regularization parameter in the adaptive cubic regularization algorithm. Comput. Optim. Appl. 53(1), 1–22 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  20. Cartis, C., Gould, N.I.M., Toint, P.L.: An adaptive cubic regularization algorithm for nonconvex optimization with convex constraints and its function-evaluation complexity. IMA J. Numer. Anal. 32(4), 1662–1695 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  21. Cartis, C., Gould, N.I., Toint, P.L.: Universal regularization methods: varying the power, the smoothness and the accuracy. SIAM J. Optim. 29(1), 595–615 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  22. Agarwal, N., Boumal, N., Bullins, B., Cartis, C.: Adaptive regularization with cubics on manifolds. Math. Program. 188(1, Ser. A), 85–134 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  23. Lubkoll, L., Schiela, A., Weiser, M.: An affine covariant composite step method for optimization with PDEs as equality constraints. Optim. Methods Softw. 32(5), 1132–1161 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  24. Ahmadzadeh, H., Mahdavi-Amiri, N.: A competitive inexact nonmonotone filter SQP method: convergence analysis and numerical results. Optim. Methods Softw., 1–34 (2021)

  25. Birgin, E.G., Martínez, J.M.: On regularization and active-set methods with complexity for constrained optimization. SIAM J. Optim. 28(2), 1367–1395 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  26. Jia, X., Liang, X., Shen, C., Zhang, L.-H.: Solving the cubic regularization model by a nested restarting Lanczos method. SIAM J. Matrix Anal. Appl. 43(2), 812–839 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  27. Hsia, Y., Sheu, R. -L., Yuan, Y.-x.: Theory and application of p-regularized subproblems for p > 2. Optim. Methods Softw. 32(5), 1059–1077 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  28. Jiang, R., Yue, M.-C., Zhou, Z.: An accelerated first-order method with complexity analysis for solving cubic regularization subproblems. Comput. Optim. Appl. 79(2), 471–506 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  29. Lieder, F.: Solving large-scale cubic regularization by a generalized eigenvalue problem. SIAM J. Optim. 30(4), 3345–3358 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  30. Fletcher, R., Gould, N.I.M., Leyffer, S., Toint, P.L., Wächter, A.: Global convergence of a trust-region SQP-filter algorithm for general nonlinear programming. SIAM J. Optim. 13(3), 635–6592003 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  31. Ulbrich, M., Ulbrich, S., Vicente, L.N.: A globally convergent primal-dual interior-point filter method for nonlinear programming. Math. Program. 100(2, Ser. A), 379–410 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  32. Walther, A., Biegler, L.: On an inexact trust-region SQP-filter method for constrained nonlinear optimization. Comput. Optim. Appl. 63(3), 613–638 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  33. Wächter, A., Biegler, L.T.: Line search filter methods for nonlinear programming: motivation and global convergence. SIAM J. Optim. 16 (1), 1–31 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  34. Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. 106(1, Ser. A), 25–57 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  35. Gould, N.I.M., Loh, Y., Robinson, D.P.: A nonmonotone filter SQP method: local convergence and numerical results. SIAM J. Optim. 25 (3), 1885–1911 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  36. Shen, C., Leyffer, S., Fletcher, R.: A nonmonotone filter method for nonlinear optimization. Comput. Optim. Appl. 52(3), 583–607 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  37. Pei, Y.G., Zhu, D.T.: On the global convergence of a projective trust region algorithm for nonlinear equality constrained optimization. Acta Math. Sin. (Engl. Ser.) 34(12), 1804–1828 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  38. Gould, N.I.M., Orban, D., Toint, P.L.: Cutest: a constrained and unconstrained testing environment with safe threads for mathematical optimization. Comput. Optim. Appl. 60(3), 545–557 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  39. Chen, Z., Dai, Y.-H., Liu, J.: A penalty-free method with superlinear convergence for equality constrained optimization. Comput. Optim. Appl. 76(3), 801–833 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  40. Liu, X., Yuan, Y.: A sequential quadratic programming method without a penalty function or a filter for nonlinear equality constrained optimization. SIAM J. Optim. 21(2), 545–571 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  41. Dolan, E.D., Moré, J. J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2, Ser. A), 201–213 (2002)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors are grateful to editor and reviewer for their valuable suggestions, insightful comments, and encouragement regarding this paper, which greatly help us to improve the presentation of this paper.

Funding

This study was financially supported by the National Natural Science Foundation of China (12071133) and Key Scientific Research Project for Colleges and Universities in Henan Province (21A110012).

Author information

Authors and Affiliations

  1. Engineering Laboratory for Big Data Statistical Analysis and Optimal Control, College of Mathematics and Information Science, Henan Normal University, Construction Road, Xinxiang, 453007, Henan Province, China

    Yonggang Pei & Shaofang Song

  2. Mathematics and Science College, Shanghai Normal University, Guilin Road, Shanghai, 200234, Shanghai, China

    Detong Zhu

Authors
  1. Yonggang Pei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shaofang Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Detong Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yonggang Pei.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Author contribution

Y. Pei and S. Song wrote the main manuscript text and did numerical experiments. D. Zhu presented some idea of the proofs of convergence. Y. Pei and S. Song revised the manuscript according to the reviewer’s comments. All authors reviewed the manuscript.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Shaofang Song and Detong Zhu contributed equally to this work.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pei, Y., Song, S. & Zhu, D. A filter sequential adaptive cubic regularization algorithm for nonlinear constrained optimization. Numer Algor 93, 1481–1507 (2023). https://doi.org/10.1007/s11075-022-01475-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-022-01475-9

Keywords

Search

Navigation