A penalty-free-type nonmonotone trust-region method for nonlinear constrained optimization
Introduction
Consider the following nonlinear programming problem:where x ∈ Rn, c(x) = (c1(x), c2(x), … , cm(x))T, m < n, f(x) and ci(x) (i = 1, 2, … , m) are real functions defined in D = {x ∈ Rn∣l ⩽ x ⩽ u}. We assume that li < ui (i = 1, 2, … , n) and they may be infinite.
Let g(x) denote the gradient of f(x) and A(x) = (∇c1(x), ∇c2(x), … , ∇cm(x)). Throughout this paper, we let ∥ · ∥ and ∥ · ∥∞ be the l2-norm and the l∞-norm on Rn, respectively. We also use fk for f(xk), ck for c(xk), etc.
Now, we define the substationary point and ϕ-stationary point of (1.1) as follows. Definition 1.1 [17] For some x* ∈ D, if there exist μl ⩾ 0, μu ⩾ 0 such thatthen x* is called a ϕ-stationary point of (1.1). Definition 1.2 [5] For some x* ∈ D, if there exist μl ⩾ 0, μu ⩾ 0 and λ ∈ Rm such thatthen x* is called a substationary point of (1.1). Moreover, if c(x*) = 0, then x* is called a stationary point of (1.1).
Given xk ∈ D, an estimate of the solution, the problem (1.1) is often solved by the sequential quadratic programming (SQP) method and it is always assumed that a search direction dk can be computed by solving the following quadratic programming subproblem:where Bk ∈ Rn×n is a symmetric matrix. And then the new iterate point is xk+1 = xk + αk dk, where αk > 0 is a step length and αk depends on some line search technique [23]. Here it is usually assumed that Bk is positive definite and the linearized constraints are consistent.
There are many trust region methods for equality constrained nonlinear programming. For example, Byrd, Schnabel and Shultz [3], Celis, Dennis and Tapia [4], El Alem [14], Powell and Yuan [22], Vardi [26], Dennis and Vicente [13]. Several extensions of trust region method to the problems involving inequality constraints have been proposed. Some of them are based on trust region methods for box-constrained problems, see, e.g., Byrd [1], Omojokun [20], Dennis et al. [11], Conn et al. [8], Dennis and Vicente [13], Chen et al. [6]. There are other related approaches which combine trust region method with interior point method, such as Dennis et al. [12], Plantenga [21],Vicente [27], Byrd et al. [2], Cleman and Li [7]. For SQP filter methods, the global convergence has been established in Fletcher et al. [15], [16]. Filter methods and its predecessor do not use any penalty function, either. There are still other methods for solving (1.1), e.g., the generalized reduced gradient in Lasdon [19], the augmented Lagrangian approach in Chen et al. [5], Conn et al. [9], Gomes et al. [17] and so on.
In this paper, we will use [5], [17] and [25] as our main references. However, there are several related approaches and recent extensions that should be mentioned. We first refer to a class of trust region algorithms introduced by Byrd [1], Omojokun [20], Dennis et al. [11], the filtering idea introduced by Fletcher and Leyffer [15], [16]. Ulbrich et al. [25] presents a nonmonotone trust region methods for nonlinear equality constrained optimization without a penalty function. Chen et al. [5] and Gomes et al. [17] use trust-region method with the augmented Lagrangian function. The main contributions of our paper is to generalize the idea and the methods above from with nonlinear equality constraints or only with box-constraints to with general nonlinear constrained optimization problem. The method does not require a restoration procedure and does not use any penalty function. Under no linear independence constraint qualification, using the concept of substationary point [5] and ϕ-stationary point [17], we give the analysis of the global convergence. We also prove that there exists at least a limit point of the sequence generated by the new algorithm, which is a stationary point of the original problem, under the linear independence constraint qualification.
This paper is organized as follows. In Section 2, the formal algorithm is described. In Section 3, we prove that, under mild conditions, this algorithm is well defined and some global convergence results with or without linear independence constraint qualification are given. Some numerical experiments for problems from [18] and [24] collection are reported in Section 4. Finally, we give some final remarks on this approach.
Section snippets
Algorithm
Given an approximate estimate of the solution xk ∈ D at the kth iteration, then following Byrd [1], Omojokun [20], Dennis et al. [11] and Ulbrich et al. [25], we can get the trial step by computing a quasi-normal step and a tangential step . These two steps have the similar effect as Fletcher’s filtering idea. In other words, the purpose of the quasi-normal step is to improve the feasibility and one of the tangential step is to improve the value of the objective
Global convergence
At first, we give the following assumptions for the global convergence analysis.
- (A1)
f(x) and ci(x) (i = 1, 2, … , m) are twice continuously differentiable at all x ∈ D.
- (A2)
There exists a bounded convex closed set Ω such that xk ∈ Ω ⋂ D for all k.
- (A3)
The matrix sequence {Bk} is uniformly bounded.
Lemma 3.1
xk is a ϕ-stationary point of (1.1) if and only if for all Δ > 0.
Lemma 3.2
If , then xk is a substationary point of (1.1).
Lemma 3.3
Under Assumptions A1–A3, Algorithm 2.1 is well defined.
Proof
Suppose that Algorithm 2.1 does
Preliminary numerical experiments
In this section, a preliminary implementation of Algorithm 2.1 is given. A Matlab code (Version 6.1) was written corresponding to this implementation. The trust region subproblems (2.1), (2.2) have the general formThe approximate Hessian matrix Bk in the subproblem (2.2) is updated by means of the Powell’s safeguarded BFGS update formula.
For the numerical test, we have used the following parameter settings in Algorithm 2.1:
Final remarks
The method introduced in this paper has the following characteristics:
- (a)
the method can be used to solve the general nonlinear programming;
- (b)
it is a globalized sequential quadratic programming algorithm;
- (c)
it uses the trust region as a globalization strategy;
- (d)
the merit function is just the objective function itself;
- (e)
it does not use the penalty parameter;
- (f)
it is globally convergent under the very mild conditions.
References (27)
- R.H. Byrd, Robust trust region methods for constrained optimization, in: Third SIAM Conference on Optimization,...
- et al.
A trust region method based on interior point techniques for nonlinear programming
Math. Program.
(2000) - et al.
A trust region algorithm for nonlinearly constrained optimization
SIAM J. Numer. Anal.
(1987) - et al.
A trust region strategy for nonlinear equality constrained optimization
- et al.
A globally convergent trust region algorithm for optimization with general constrains and simple bounds
Acta Math. Appl. Sin.
(1999) - et al.
A nonmonotone trust region algorithm for optimization with simple bounds
Appl. Math. Optim.
(2001) - et al.
An interior trust region approach for nonlinear minimization subject to bounds
SIAM J. Optim.
(1996) - et al.
Global convergence of a class of trust region algorithms for optimization with simple bounds
SIAM J. Numer. Anal.
(1988) - et al.
A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds
SIAM J. Numer. Anal.
(1991) On perturbations in systems of linear inequalities
SIAM J. Numer. Anal.
(1973)
A global convergence theory for general trust-region-based algorithms for equality constrained optimization
SIAM J. Optim.
Trust-region interior-point SQP algorithms for a class of nonlinear programming problems
SIAM J. Control Optim.
On the convergence theory of trust-region-based algorithms for equality constrained optimization
SIAM J. Optim.
Cited by (14)
A penalty-free method with line search for nonlinear equality constrained optimization
2013, Applied Mathematical ModellingCitation Excerpt :The effectiveness of these so-called penalty-type methods hinges on how well the initial penalty parameter is chosen and how “intelligently” it is updated during the course of minimization. To avoid the selection of the penalty parameter, some authors research the technique without the penalty function, for example, see [4,7,8,14–16,20,26–28,30,31]. The methods which do not use any penalty function are called the penalty-free-type ones.
Global and local convergence of a penalty-free method for nonlinear programming
2013, Computers and Mathematics with ApplicationsA new penalty-free-type algorithm based on trust region techniques
2012, Applied Mathematics and ComputationCitation Excerpt :Global and local convergence analysis are established and numerical results is satisfactory. Its generalization in equality and bound constrained problems is proposed by Chen [6]. Except for filter methods and Ulbrich–Ulbrich’s nonmonotone penalty-free-type trust region methods [16], there are some other penalty-free-type algorithms, such as the “tolerance tube method” by Zoppke-Donaldson [22] and the “trust funnel method” by Gould and Toint [12].
Global and local convergence of a class of penalty-free-type methods for nonlinear programming
2012, Applied Mathematical ModellingGlobal convergence of a nonmonotone filter method for equality constrained optimization
2012, Applied Mathematics and ComputationCitation Excerpt :Similar to the Byrd-Omojokun class of algorithms, each step is composed into a quai-normal step and a tangential step. Regarding related work, we refer to [16–22]. This paper is organized as follows: Section 2 introduces the nonmonotone filter idea and the fraction of Cauchy decrease.
A nonmonotone filter trust region method for nonlinear constrained optimization
2009, Journal of Computational and Applied MathematicsCitation Excerpt :And the Lagrange function as one of the criteria to decide whether a trial point is accepted or not. Similar work can be seen in [4,5,13]. Motivated by the idea and methods above, we proposed a class of nonmonotone filter trust region methods for solving problem (P).