Abstract
In this paper, we propose a new regularized quasi-Newton method for unconstrained optimization. At each iteration, a regularized quasi-Newton equation is solved to obtain the search direction. The step size is determined by a non-monotone Armijo backtracking line search. An adaptive regularized parameter, which is updated according to the step size of the line search, is employed to compute the next search direction. The presented method is proved to be globally convergent. Numerical experiments show that the proposed method is effective for unconstrained optimizations and outperforms the existing regularized Newton method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Byrd, R.H., Nocedal, J., Schnabel, R.B.: Representations of quasi-Newton matrices and their use in limited memory methods. Math. Program. 63(1), 129–156 (1994)
Burdakov, O., Gong, L., Zikrin, S., et al.: On efficiently combining limited-memory and trust-region techniques. Math. Program. Comput. 9(1), 101–134 (2017)
Deng, N.Y., Xiao, Y., Zhou, F.J.: Nonmonotone trust region algorithm. J. Optim. Theory Appl. 26, 259–285 (1993)
Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2), 201–213 (2002)
Erway, J.B., Marcia, R.F.: On efficiently computing the eigenvalues of limited-memory quasi-newton matrices. SIAM J. Matrix Anal. A. 36(3), 1338–1359 (2015)
Gould, N.I.M., Orban, D., Toint, P.L.: CUTEst: a constrained and unconstrained testing environment with safe threads for mathematical optimization. Comput. Optim. Appl. 60(3), 545–557 (2015)
Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone line search technique for Newton’s method. SIAM J. Numer. Anal. 23(4), 707–716 (1986)
Gu, N.Z., Mo, J.T.: Incorporating nonmonotone strategies into the trust region method for unconstrained optimization. Comput. Math. Appl. 55(9), 2158–2172 (2008)
Li, D.H., Fukushima, M., Qi, L., et al.: Regularized newton methods for convex minimization problems with singular solutions. Comput. Optim. Appl. 28(2), 131–147 (2004)
Li, Y.J., Li, D.H.: Truncated regularized Newton method for convex minimizations. Comput. Optim. Appl. 43(1), 119–131 (2009)
Mor\(\acute{e}\), J.J., Sorensen, D. C.: Computing a Trust Region Step. SIAM J. Sci. Stat. Comput. 4(3), 553–572 (1983)
Polyak, R.A.: Regularized Newton method for unconstrained convex optimization. Math. Program. 120(1), 125–145 (2009)
Ueda, K., Yamashita, N.: Convergence properties of the regularized Newton method for the unconstrained nonconvex optimization. Appl. Math. Optim. 62(1), 27–46 (2010)
Ueda, K., Yamashita, N.: A regularized Newton method without line search for unconstrained optimization. Comput. Optim. Appl. 59(1), 321–351 (2014)
Zhang, H.C., Hager, W.W.: A nonmonotone line search technique and its application to unconstrained optimization. SIAM J. Optim. 12, 1043–1056 (2004)
Zhang, H., Ni, Q.: A new regularized quasi-Newton algorithm for unconstrained optimization. Appl. Math. Comput. 259, 460–469 (2015)
Acknowledgements
This work is supported by NSFC (11771210, 11701283, 11471159, 11571169, and 61661136001) and the Natural Science Foundation of Jiangsu Province (BK20141409).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhang, H., Ni, Q. A new regularized quasi-Newton method for unconstrained optimization. Optim Lett 12, 1639–1658 (2018). https://doi.org/10.1007/s11590-018-1236-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11590-018-1236-z