Abstract
In order to get a higher order accuracy of approximating the Hessian matrix of the objective function, we use the chain rule and propose two modified secant equations. An interesting property of the proposed methods is that these utilize information from two most recent steps where the usual secant equation uses only the latest step information. The other point of interest to one of the proposed methods is that it makes use of both gradient and function value information. We show that the modified BFGS methods based on the new secant equations are globally convergent. The presented experimental results illustrate that the proposed methods are efficient.



Similar content being viewed by others
References
Broyden, C. G.: A class of methods for solving nonlinear simultaneous equations. Math. Comp. 19, 577–593 (1965)
Broyden, C. G.: The convergence of a class of double-rank minimization algorithms: 2. new algorithm. IMA J. Appl. Math. 6(3), 222–231 (1970)
Byrd, R., Nocedal, J.: A tool for the analysis of quasi-Newton methods with application to unconstrained minimization. SIAM J. Numer. Anal. 26, 727–739 (1989)
Byrd, R., Nocedal, J., Yuan, Y.: Global convergence of a class of quasi-Newton methods on convex problems. SIAM J. Numer. Anal. 24, 1171–1189 (1987)
Dai, Y.: Convergence properties of the BFGS algorithm. SIAM J. Optim. 13, 693–701 (2003)
Dolan, E., Moré, J. J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)
Ford, J. A., Moghrabi, I. A.: Multi-step quasi-Newton methods for optimization. J. Comput. Appl. Math. 50, 305–323 (1994)
Ford, J. A., Moghrabi, I. A.: Alternating multi-step quasi-Newton methods for unconstrained optimization. J. Comput. Appl. Math. 82, 105–116 (1997)
Ford, J. A., Saadallah, A. F.: On the construction of minimisation methods of quasi-Newton type. J. Comput. Appl. Math. 20, 239–246 (1987)
Gould, N. I. M., Orban, D., Toint, Ph.L.: CUTEr, A constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29, 373–394 (2003)
Griewank, A.: The global convergence of partioned BFGS on problems with convex decompositons and Lipschitzian gradients. Math. Program. 50, 141–175 (1991)
Li, D. H., Fukushima, M.: On the global convergence of BFGS method for nonconvex unconstrained optimization problems. SIAM J. Optim. 11, 1054–1064 (2001)
Li, D. H., Fukushima, M.: A modified BFGS method and its global convergence in nonconvex minimization. J. Comput. Appl. Math. 129, 15–35 (2001)
Mascarenhas, W. F.: The BFGS method with exact line searches fails for non-convex objective functions. Math. Program. 99, 49–61 (2004)
Moghrabi, I. A.: Multi-Step quasi-Newton Method for Unconstrained Optimization. Ph.D. thesis, Depart. Comput. Sci. Univ. Essex (1993)
Moghrabi, I. A.: Exploiting function values in multi-step method. Int. J. Pure Appl. Math. 28(2), 187–196 (2006)
Moghrabi, I. A., Ford, J. A.: A nonlinear model for function-value multistep methods. Comput. Math. Appl. 42, 1157–1164 (2001)
Nocedal, J., Wright, S. J.: Numerical Optimization. Springer, New York (2006)
Pearson, J. D.: Variable metric methods of minimization. Comp. J. 12, 171–178 (1969)
Powell, M. J. D.: Some global convergence properties of a variable metric algorithm for minimization without exact line searches. In: Cottle, R.W., Lemke, C.E. (eds.) Nonlinear Programming, SIAM-AMS Proceedings (SIAM publications), vol. 9, pp 53–72 (1976)
Toint, Ph.L.: Global convergence of the partioned BFGS algorithm for convex partially separable opertimization. Math. Program. 36, 290–306 (1986)
Wei, Z., Li, G., Qi, L.: New quasi-Newton methods for unconstrained optimization problems. Appl. Math. Comput. 175, 1156–1188 (2006)
Yuan, G., Wei, Z.: Convergence analysis of a modified BFGS method on convex minimizations. Comput. Optim. Appl. 47, 237–255 (2010)
Zhang, J. Z., Deng, N. Y., Chen, L. H.: New quasi-Newton equation and related methods for unconstrained optimization. J. Optim. Theory Appl. 102, 147–167 (1999)
Zhang, J. Z., Xu, C. X.: Properties and numerical performance of quasi-Newton methods with modified quasi-Newton equation. J. Comput. Appl. Math. 137, 269–278 (2001)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
Here, in Table 1, the column headings have the following meanings:
- n::
-
number of variables of the problem
- No.::
-
problem number
Note 1
Assume that Lj, j = 0, 1, 2 and \( \frac {dX(t)}{dt} \) defined by (9) and (18), respectively, then we have
and
Note 2
Assume that Lj, j = 0, 1, 2 and \( \frac {dx(t,\theta )}{dt} \) defined by (9) and (11), respectively, then we have
Similarly
and
Proof of Lemma 2:
Proof Let ξk denote the angle between \(s^{\ast }_{k}\) and \(B_{k} s^{\ast }_{k}\), that is,
and
By taking the trace of both sides of (3), we obtain
where Tr(B) denotes the trace of B. On the other hand we have (see [19]):
where Det(B) is the determinant of B. Let
where B is a positive definite matrix. Using (59), (60) and (61), we obtain
From the definitions (57) and (58), the relation (62) can be written as
On the other hand, from definition of ψ(B), we have
where 0 < λ1 ≤ λ2 ≤ ... ≤ λn are the eigenvalues of B. Therefore we have
This, together with (64), result in
where
Note that the function
is nonpositive for all t > 0, therefore we have ηj ≥ 0, ∀j. Let us now define Jk to be a set consisting of the ⌈pk⌉ indices corresponding to the ⌈pk⌉ smallest values of ηj, for j ≤ k, and let \(\eta _{m_{k}}\) denote the largest of the ηj for j ∈ Jk. Then
This together with (65) yields
From the fact that the term inside the brackets in (66) is less than or equal to zero, using (66) and (67) we conclude that,
Therefore,
In addition, using (66) and (68), we get
Since, the function u(t) achieves its maximum value at t = 1, and satisfies \(u(t)\rightarrow -\infty \) both as \(t\rightarrow 0\) and as \(t\rightarrow \infty \). Therefore, it follows that
where \(\tilde {\beta }_{2}\) and β3 are positive constants.
Now, using (69) and (71), we obtain
Since,
The relations (72) and (73) imply (50) with \(a_{1}= \frac {\beta _{3}}{\beta _{1}},\) \(a_{2}={\beta ^{2}_{1}}\tilde {\beta }_{2}\) and a3 = β3. □
Proof of Lemma 3:
Proof The relation (44) implies
Therefore
equivalently,
On the other hand, for wk defined by (33), we have
From this fact |θ| > μ1 we can deduce that there exist positive constants κ1, κ2,..., κ6 such that
Hence,
where γ and τ are the same as that given in (41) and (43), respectively. Thus from the definition of \( y_{k}^{\ast } \) and relation (74) we have
Since αk is bounded, this completes the proof. □
Rights and permissions
About this article
Cite this article
Dehghani, R., Bidabadi, N. & Hosseini, M.M. Using nonlinear functions to approximate a new quasi-Newton method for unconstrained optimization problems. Numer Algor 87, 755–777 (2021). https://doi.org/10.1007/s11075-020-00986-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11075-020-00986-7