A nonmonotone trust region method for unconstrained optimization

Abstract

In this paper, we propose a nonmonotone trust region method for unconstrained optimization. Our method can be regarded as a combination of nonmonotone technique, fixed steplength and trust region method. When a trial step is not accepted, the method does not resolve the subproblem but generates a iterative point whose steplength is defined by a formula. We only allow increase in function value when trial steps are not accepted in close succession of iterations. Under mild conditions, we prove that the algorithm is global convergence and superlinear convergence. Primary numerical results are reported.

Introduction

In this paper, we consider the following unconstrained optimization problem:

$$\underset{x\in R^n}{\min }f(x)\text{,}$$

where f:Rⁿ → R is a twice continuously differentiable.

It is well known that trust region method is a kind of important and efficient methods for nonlinear optimization. Since it was proposed by Levenberg [7] and Marquardt [8] for nonlinear least-squares problems and by Goldfeld et al. [5] for unconstrained optimization, trust region method has been studied by many researchers, see [1], [2], [4], [10], [14], [15], [17] and papers cited therein.

Trust region methods are iterative methods. At each iterative point x_k, a trial step d_k is generated by solving the following subproblem:

$$\min g_k^{T}d+\frac{1}{2}{d}^T{B}_{k}d\equiv {\varphi }_k(d)$$

$$\mathrm{s}\text{.}\mathrm{t}\text{.}\parallel d\parallel ⩽{\Delta }_k\text{,}$$

where g_k = ∇f(x_k), B_k ∈ R^n×n is an approximate Hessian matrix of f at x_k, and Δ_k > 0 is a trust region radius. Some criterion is used to decided whether trial step d_k is accepted or not. If trial step is not accepted, the subproblem (1.2), (1.3) with a reduced trust region radius should be resolved until an acceptable step is found. Hence, the subproblem may be solved several times at an iteration and the total cost of computation for one iteration might be expensive for large scale problem.

In recent years, a variety of trust region methods have been proposed in the literature. For example, Nocedal and Yuan [12], and Gertz [9] presented methods which combine line search technique and trust region method. When the trial step is not successful, their methods performance a line search to find a iterative point instead of resolving the subproblem. Therefore, their methods require little computation than classic trust region methods. Deng et al. [3], Zhang et al. [17] and Sun [13] proposed various nonmonotone trust region methods for unconstrained optimization. These papers indicated that the nonmonotone algorithm is efficient, especially for ill-conditioned problems.

On the other hand, Sun and Zhang [6] and Chen and Sun [15] proposed a fixed steplength method for unconstrained optimization. In their approaches, without using line search, they computed the steplength by a formula at each iteration. Thus their methods might be practical in the cases that the line search is expensive or hard and allow a considerable saving in the number of function evaluations.

In this paper, we consider a method which combines nonmonotone technique, fixed steplength and trust region method. Our aim is improve the algorithm proposed in [12] and make it more effective in practical implementation. The main difference between the method in [12] and our method is that in the former one a steplength is computed by a line search when the trial step is not successful, whereas in our method a steplength is defined by a formula. We use the formula suggested by Sun and Zhang [6], Chen and Sun [15] to obtain an steplength. On the one hand, most of the nonmonotone method allows an increase in function value at each iteration. But our method only allows an increase in function value when trial steps are not accepted in close succession of iterations.

The paper is organized as follows. In Section 2, we describe our algorithm for unconstrained optimization problems which combines the techniques of fixed steplength, nonmonotonicity and trust region method. In Section 3, under suitable conditions, global convergence and superlinear convergence of the proposed algorithm are established. Primary numerical results are presented in Section 4.