Skip to main content

Advertisement

SpringerLink
Log in

A sequential adaptive regularisation using cubics algorithm for solving nonlinear equality constrained optimization

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

The adaptive regularisation algorithm using cubics (ARC) is initially proposed for unconstrained optimization. ARC has excellent convergence properties and complexity. In this paper, we extend ARC to solve nonlinear equality constrained optimization and propose a sequential adaptive regularisation using cubics algorithm inspired by sequential quadratic programming (SQP) methods. In each iteration of our method, the trial step is computed via composite-step approach, i.e., it is decomposed into the sum of normal step and tangential step. By means of reduced-Hessian approach, a new ARC subproblem for nonlinear equality constrained optimization is constructed to compute the tangential step, which can supply sufficient decrease required in the proposed algorithm. Once the trial step is obtained, the ratio of the penalty function reduction to the model function reduction is calculated to determine whether the trial point is accepted. The global convergence of the algorithm is investigated under some mild assumptions. Preliminary numerical experiments are reported to show the performance of the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Data availability

There is no data generated or analysed in this paper.

References

  1. Agarwal, N., Boumal, N., Bullins, B., Cartis, C.: Adaptive regularization with cubics on manifolds. Math. Program. 188, 85–134 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  2. Bellavia, S., Gurioli, G., Morini, B.: Adaptive cubic regularization methods with dynamic inexact Hessian information and applications to finite-sum minimization. IMA J. Numer. Anal. 41(1), 764–799 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  3. Benson, H., Shanno, D.: Cubic regularization in symmetric rank-1 quasi-Newton methods. Math. Program. Comput. 10(4), 457–486 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  4. Benson, H.Y., Shanno, D.F.: Interior-point methods for nonconvex nonlinear programming: cubic regularization. Comput. Optim. Appl. 58(2), 323–346 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bergou, E., Diouane, Y., Gratton, S.: On the use of the energy norm in trust-region and adaptive cubic regularization subproblems. Comput. Optim. Appl. 68(3), 533–554 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  6. Bergou, E., Youssef, D., Serge, G.: A line-search algorithm inspired by the adaptive cubic regularization framework and complexity analysis. J. Optim. Theory Appl. 178(3), 885–913 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  7. Bianconcini, T., Sciandrone, M.: A cubic regularization algorithm for unconstrained optimization using line search and nonmonotone techniques. Optim. Methods Softw. 31(5), 1008–1035 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  8. Birgin, E., Martínez, J.: On regularization and active-set methods with complexity for constrained optimization. SIAM J. Optim. 28(2), 1367–1395 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  9. Byrd, R.H., Schnabel, R.B., Shultz, G.A.: A trust region algorithm for nonlinearly constrained optimization. SIAM J. Numer. Anal. 24(5), 1152–1170 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  10. Cartis, C., Gould, N., Toint, P.: Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results. Math. Program. 127(2), 245–295 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  11. Cartis, C., Gould, N., Toint, P.: Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity. Math. Program. 130(2), 295–319 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  12. Cartis, C., Gould, N., Toint, P.: A concise second-order complexity analysis for unconstrained optimization using high-order regularized models. Optim. Methods Softw. 35(2), 243–256 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  13. Cartis, C., Gould, N.I., Toint, P.L.: Universal regularization methods: varying the power, the smoothness and the accuracy. SIAM J. Optim. 29(1), 595–615 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  14. Cartis, C., Gould, N.I.M., Toint, P.L.: An adaptive cubic regularization algorithm for nonconvex optimization with convex constraints and its function-evaluation complexity. IMA J. Numer. Anal. 32(4), 1662–1695 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  15. Cartis, C., Gould, N.I.M., Toint, P.L.: On the evaluation complexity of constrained nonlinear least-squares and general constrained nonlinear optimization using second-order methods. SIAM J. Numer. Anal. 53(2), 836–851 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  16. Cartis, C., Gould, N.I.M., Toint, P.L.: Sharp worst-case evaluation complexity bounds for arbitrary-order nonconvex optimization with inexpensive constraints. SIAM J. Optim. 30(1), 513–541 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  17. Chen, Z., Dai, Y.H., Liu, J.: A penalty-free method with superlinear convergence for equality constrained optimization. Comput. Optim. Appl. 76(3), 801–833 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  18. Chen, Z., Qiu, S., Jiao, Y.: A penalty-free method for equality constrained optimization. J. Ind. Manag. Optim. 9(2), 391–409 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  19. Dehghan Niri, T., Heydari, M., Hosseini, M.M.: An improvement of adaptive cubic regularization method for unconstrained optimization problems. Int. J. Comput. Math. 98(2), 271–287 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  20. Dennis, J.E., Jr., El-Alem, M., Maciel, M.C.: A global convergence theory for general trust-region-based algorithms for equality constrained optimization. SIAM J. Optim. 7(1), 177–207 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  21. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  22. Dussault, J.P.: \({\rm ARC_q}\): a new adaptive regularization by cubics. Optim. Methods Softw. 33(2), 322–335 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  23. Gould, N.I.M., Orban, D., Toint, P.L.: Cutest: a constrained and unconstrained testing environment with safe threads for mathematical optimization. Comput. Optim. Appl. 60(3), 545–557 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  24. Gould, N.I.M., Porcelli, M., Toint, P.L.: Updating the regularization parameter in the adaptive cubic regularization algorithm. Comput. Optim. Appl. 53(1), 1–22 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  25. Griewank, A.: The modification of newton’s method for unconstrained optimization by bounding cubic terms. Tech. rep., Technical report NA/12 (1981). Department of Applied Mathematics and Theoretical Physics, University of Cambridge (1981)

  26. Hsia, Y., Sheu, R.L., Yuan, Y.X.: Theory and application of \(p\)-regularized subproblems for \(p>2\). Optim. Methods Softw. 32(5), 1059–1077 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  27. Jia, X., Liang, X., Shen, C., Zhang, L.H.: Solving the cubic regularization model by a nested restarting Lanczos method. SIAM J. Matrix Anal. Appl. 43(2), 812–839 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  28. Jiang, R., Yue, M.C., Zhou, Z.: An accelerated first-order method with complexity analysis for solving cubic regularization subproblems. Comput. Optim. Appl. 79(2), 471–506 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  29. Lieder, F.: Solving large-scale cubic regularization by a generalized eigenvalue problem. SIAM J. Optim. 30(4), 3345–3358 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  30. Lubkoll, L., Schiela, A., Weiser, M.: An affine covariant composite step method for optimization with PDEs as equality constraints. Optim. Methods Softw. 32(5), 1132–1161 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  31. Martínez, J.M.: On high-order model regularization for constrained optimization. SIAM J. Optim. 27(4), 2447–2458 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  32. Martínez, J.M., Raydan, M.: Cubic-regularization counterpart of a variable-norm trust-region method for unconstrained minimization. J. Glob. Optim. 68(2), 367–385 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  33. Nesterov, Y., Polyak, B.T.: Cubic regularization of Newton method and its global performance. Math. Program. 108, 177–205 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  34. Park, S., Jung, S.H., Pardalos, P.M.: Combining stochastic adaptive cubic regularization with negative curvature for nonconvex optimization. J. Optim. Theory Appl. 184(3), 953–971 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  35. Powell, M.J.D.: Convergence properties of algorithms for nonlinear optimization. SIAM Rev. 28(4), 487–500 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  36. Sha, L., Wei, Z., Li, L.: A trust region algorithm with adaptive cubic regularization methods for nonsmooth convex minimization. Comput. Optim. Appl. Int. J. 51(2), 551–573 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  37. Vardi, A.: A trust region algorithm for equality constrained minimization: convergence properties and implementation. SIAM J. Numer. Anal. 22(3), 575–591 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  38. Zhao, T., Liu, H., Liu, Z.: New subspace minimization conjugate gradient methods based on regularization model for unconstrained optimization. Numer. Algorithms 87(4), 1501–1534 (2021)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors are very grateful to the editor and the referees, whose valuable suggestions and insightful comments helped to improve significantly the paper. The authors gratefully acknowledge the supports of the National Natural Science Foundation of China (12071133), Key Scientific Research Project for Colleges and Universities in Henan Province (21A110012).

Author information

Authors and Affiliations

  1. Engineering Laboratory for Big Data Statistical Analysis and Optimal Control, College of Mathematics and Information Science, Henan Normal University, Xinxiang, China

    Yonggang Pei & Shaofang Song

  2. Mathematics and Science College, Shanghai Normal University, Shanghai, China

    Detong Zhu

Authors
  1. Yonggang Pei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shaofang Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Detong Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yonggang Pei.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pei, Y., Song, S. & Zhu, D. A sequential adaptive regularisation using cubics algorithm for solving nonlinear equality constrained optimization. Comput Optim Appl 84, 1005–1033 (2023). https://doi.org/10.1007/s10589-022-00449-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-022-00449-w

Keywords

Mathematics Subject Classification

Search

Navigation