Inexact overlapped block Broyden methods for solving nonlinear equations☆
Introduction
In this paper we are concerned with the problem of solving the large system of nonlinear equationswhere F(x)=(f1,…,fN)T is a nonlinear operator from RN to RN. Such systems often arise when solving initial or boundary value problems in ordinary or partial differential equations. As well known, Newton methods and its variations [1], [2], etc. coupled with some direct solution technique such as Gaussian elimination are powerful solvers for these nonlinear systems in case one has a sufficiently good initial guess x0 and N is not too large. When the Jacobian is large and sparse, inexact Newton methods [3], [4], [5], [6], [7], etc. or some kind of nonlinear block-iterative methods [8], [9], [10], etc. may be used.
Inexact Newton methods is actually a two stage iterative method which has the following general form [3]:
For k=0 step 1 until convergence do
Find some sk which satisfies
F′(xk)sk=−F(xk)+rk, where .
Set xk+1=xk+sk.
Recently with the development of Krylov subspace projection methods, this class methods such as Arnoldi’s method [13] and the generalized minimum residual method (GMRES) [14] is widely used as the inner iteration for inexact Newton methods [4], [5], etc. This combined method is called inexact Newton–Krylov methods or nonlinear Krylov subspace projection methods. The Krylov methods have the virtue of requiring almost no matrix storage, resulting in a distinct advantage over direct methods for solving the large Newton equations. In particular, the product of Jacobian and some fixed vector (F′(xk)v) is only utilized in a Krylov method for solving F′(xk)sk=F(xk), and the product can be approximated by the difference quotientwhere σ is a scalar. So the Jacobian need not be computed explicitly. In [4] Brown gave the local convergence results for inexact Newton–Krylov methods with the difference approximation of Jacobian.
Nonlinear block-iterative methods in parallel is another class for large and sparse nonlinear equations, which chiefly consists of block Newton-type and block quasi-Newton methods. The classical nonlinear block-Jacobi method and nonlinear block-Gauss–Seidel method [1] are two original versions. A block-parallel Newton method via overlapping epsilon decompositions ([15]) was presented by Zecevic and Siljak [8]. In [9] the authors described a parallelizable Jacobi-type block Broyden method and more recently a partially asynchronous block Broyden method has been proposed by Xu [10].
In this paper, we consider an inexact block Broyden method with partially overlap which is a generalization of the parallelizable Jacobi-type block Broyden method. The basic idea is to directly perform the block Broyden iteration for the overlapped blocks and assemble the overlapping parts of the results in an average weighted manner at each iteration. The goal is to accelerate the block Broyden method by the overlapping parts. In Section 2, the new methods are presented. Section 3 describes the sufficient conditions under which the new methods are convergent. In Section 4, some useful techniques are considered. The numerical results for solving a function used as a test problem in [11] are also given. In Section 5, we draw conclusions and discuss the future work on this subject.
Section snippets
The new algorithm
In the following discussion, y*∈RN is an exact solution of system (1), i.e., F(y*)=0. Let y0 be an initial guess of y*, and suppose that it is possible to generate a new approximation yk of y* for k=0,1,… Suppose the components of y and F are conformally partitioned as follows:whereLet Si={i1,…,ini}, the partition is chosen such that ⋃i=1MSi={1,2,…,N} and . This partition may be
Local convergence
In this section, we give the conditions under which the new methods presented in the previous section are locally convergent.
Let , where denotes a diagonal matrix of order ni for i=1,…,MA nonlinear function Φ:RN×D′→RN could be defined bywhere D′={B=Bα(B1,…,BM)|Bi∈Rni×ni is nonsingular, i=1,…,M}. Obviously, if there exist y*∈RN and E*∈D′
Implementation and examples
In this section, several strategies for partitioning the Jacobian into weakly coupled, partially overlapped blocks are briefly described first, follow by the restarted technique and finally, a test function is described and some numerical results for solving it are presented.
As well known, the zero–nonzero structure of an N×N symmetric matrix A=(Aij) could be associated with a undirected graph G(A)=〈V,E〉, where the set V has N vertices {i,…,N} and E is a collection of unordered pairs of
Conclusions and discussion
In this paper a partially overlapped block Broyden method (Algorithm 1) is presented for solving large nonlinear systems. The basic idea is to directly perform the block Broyden iteration for the overlapped blocks and assemble the overlapping parts of the iterative results in an appropriate manner at each iteration. Combining it with some iterative or direct linear solvers, it is possible to obtain a family of nonlinear solvers which can be easily parallelized. In particular, an inexact version
References (22)
Convergence rates for inexact Newton-like methods at singular points and applications
Appl. Math. Comput.
(1999)Convergence of partially asynchronous block quasi-Newton methods for nonlinear systems of equations
J. Appl. Math. Comput.
(1999)- et al.
Iterative Solution of Nonlinear Equations in Several Variables
(1970) Methods for Solving Systems of Nonlinear Equations
(1998)- et al.
Inexact Newton methods
SIAM J. Numer. Anal.
(1982) A local convergence theory for combined Inexact-Newton/Finite-Difference projection methods
SIAM J. Numer. Anal.
(1987)- et al.
Hybrid Krylov methods for nonlinear systems of equations
SIAM J. Sci. Statist. Comput.
(1990) Convergence behaviour of inexact Newton methods
Math. Comp.
(1999)- et al.
A block-parallel Newton method via overlapping epsilon decompositions
SIAM J. Matrix Anal. Appl.
(1994) - et al.
Inexact block Jacobi–Broyden methods for solving nonlinear systems of equations
SIAM J. Sci. Comput.
(1997)
Comparing algorithms for solving sparse nonlinear systems of equations
SIAM J. Sci. Statist. Comput.
Cited by (13)
Weak convergence conditions for Inexact Newton-type methods
2011, Applied Mathematics and ComputationCitation Excerpt :For approximating a locally unique solution of a nonlinear equation in a Banach spaces setting, new semilocal convergence results for (INTM) are provided. Using a combination of Lipschitz/center–Lipschitz conditions, instead of only Lipschitz conditions, we provided an analysis with the following advantages over the works in [15–31]: weaker sufficient convergence conditions and larger convergence domain. Numerical examples further validating the results and special case of (INTM) are also provided in this study.
Convergence behaviour of inexact Newton methods under weak Lipschitz condition
2006, Journal of Computational and Applied MathematicsInexact block Newton methods for solving nonlinear equations
2005, Applied Mathematics and ComputationResearch in mathematics at Cameron University
2021, Research in Mathematics at Cameron UniversityDerivative-free Broyden’s method for inverse partially known Sturm-Liouville potential functions
2015, Thai Journal of MathematicsAdvances on iterative procedures
2011, Advances on Iterative Procedures
- ☆
The work has been partly Supported by National Key Basic Research Special Fund (No. 1998020306), and by CNSF (No. 19871047).