Abstract
A nonlinear stepsize control framework for unconstrained optimization was recently proposed by Toint (Optim Methods Softw 28:82–95, 2013), providing a unified setting in which the global convergence can be proved for trust-region algorithms and regularization schemes. The original analysis assumes that the Hessians of the models are uniformly bounded. In this paper, the global convergence of the nonlinear stepsize control algorithm is proved under the assumption that the norm of the Hessians can grow by a constant amount at each iteration. The worst-case complexity is also investigated. The results obtained for unconstrained smooth optimization are extended to some algorithms for composite nonsmooth optimization and unconstrained multiobjective optimization as well.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
See Theorem 1.2.22 in Sun and Yuan [18].
References
Bellavia, S., Cartis, C., Gould, N.I.M., Morini, B., Toint, Ph.L.: Convergence of a regularized euclidean residual algorithm for nonlinear least-squares. SIAM J. Numer. Anal. 48, 1–29 (2010)
Cartis, C., Gould, N.I.M., Toint, Ph.L.: Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results. Math. Program. 127, 245–295 (2011)
Cartis, C., Gould, N.I.M., Toint, Ph.L.: Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function—and derivative—evaluation complexity. Math. Program. 130, 295–319 (2011)
Cartis, C., Gould, N.I.M., Toint, Ph.L.: On the evaluation complexity of composite function minimization with applications to nonconvex nonlinear programming. SIAM J. Optim. 21, 1721–1739 (2011)
Conn, A.R., Gould, N.I.M., Toint, Ph.L.: Trust-Region Methods. SIAM, Philadelphia (2000)
Dai, Y.H., Xu, D.C.: A new family of trust region algorithms for unconstrained optimization. J. Comp. Math. 21, 221–228 (2003)
Dennis, J.E., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, Englewood Cliffs (1983)
Fan, J.: Convergence rate of the trust region method for nonlinear equations under local error bound condition. Comput. Optim. Appl. 34, 215–227 (2006)
Fan, J., Yuan, Y.: A new trust region algorithm with trust region radius converging to zero. In: Li, D. (ed.) Proceeding of the 5th International Conference on Optimization: Techiniques and Applications, pp. 786–794. Hong Kong (2001)
Fletcher, R.: A model algorithm for composite nondifferentiable optimization problems. Math. Program. 17, 67–76 (1982)
Fliege, J., Svaiter, B.F.: Steepest descent methods for multicriteria optimization. Math. Meth. Oper. Res. 51, 479–494 (2000)
Guerraggio, A., Luc, D.T.: Optimality conditions for \(C^{1,1}\) vector optimization problems. J. Optim. Theory Appl. 109, 615–629 (2001)
Nesterov, Yu.: Modified Gauss–Newton scheme with worst-case guarantees for global performance. Optim. Methodos Softw. 22, 469–483 (2007)
Powell, M.J.D.: Convergence properties of a class of minimization algorithms. In: Mangasarian, O.L., Meyer, R.R., Robinson, S.M. (eds.) Nonlinear Programming, vol. 2. Academic Press, New York (1975)
Powell, M.J.D.: General Algorithms for Discrete Nonlinear Approximation Calculations. Report DAMTP 1983/NA2, Department of Applied Mathematics and Theoretical Physics. University of Cambridge, Cambridge, England (1983)
Powell, M.J.D.: On the global convergence of trust region algorithms for unconstrained minimization. Math. Program. 29, 297–303 (1984)
Shultz, G.A., Schnabel, R.B., Byrd, R.H.: A family of trust-region-based algorithms for unconstrained minimization with strong global convergence properties. SIAM J. Numer. Anal. 22, 47–67 (1985)
Sun, W., Yuan, Y.: Optimization Theory and Methods: Nonlinear Programming. Springer, Berlin (2006)
Toint, Ph.L.: Nonlinear stepsize control, trust regions and regularizations for unconstrained optimization. Optim. Methods Softw. 28, 82–95 (2013)
Toint, Ph.L.: Global convergence of a class of trust region methods for nonconvex minimization in Hilbert space. IMA J. Numer. Anal. 8, 231–252 (1988)
Villacorta, K.D.V., Oliveira, P.R., Soubeyran, A.: A trust-region method for unconstrained multiobjective problems with applications in satisficing processes. J. Optim. Theory Appl. 160, 865–889 (2014)
Yuan, Y.: Conditions for convergence of trust region algorithm for nonsmooth optimization. Math. Program. 31, 220–228 (1985)
Yuan, Y.: On the superlinear convergence of a trust region algorithm for nonsmooth optimization. Math. Program. 31, 269–285 (1985)
Zhang, J., Wang, Y.: A new trust region method for nonlinear equations. Math. Meth. Oper. Res. 58, 283–298 (2003)
Acknowledgments
This work was done while the first author was visiting the Institute of Computational Mathematics and Scientific/Engineering Computing of the Chinese Academy of Sciences. He would like to express his deep gratitude to Professor Ya-xiang Yuan, Professor Yu-hong Dai, Dr. Xin Liu and Dr. Ya-feng Liu for their warm hospitality. The authors also are grateful to two anonymous referees, whose comments helped a lot to improve the first version of this paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
G. N. Grapiglia was supported by CAPES, Brazil (Grant PGCI 12347/12-4). J. Yuan was partially supported by CAPES and CNPq, Brazil. Y. Yuan was partially supported by NSFC, China (Grant 11331012).
Rights and permissions
About this article
Cite this article
Grapiglia, G.N., Yuan, J. & Yuan, Yx. On the convergence and worst-case complexity of trust-region and regularization methods for unconstrained optimization. Math. Program. 152, 491–520 (2015). https://doi.org/10.1007/s10107-014-0794-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-014-0794-9
Keywords
- Global convergence
- Worst-case complexity
- Trust-region methods
- Regularization methods
- Unconstrained Optimization
- Composite nonsmooth optimization
- Multiobjective optimization