Global exponential stability in Lagrange sense for neutral type recurrent neural networks

doi:10.1016/j.neucom.2010.10.001

Neurocomputing

Volume 74, Issue 4, January 2011, Pages 638-645

https://doi.org/10.1016/j.neucom.2010.10.001 Get rights and content

Abstract

In this paper, the global exponential stability in Lagrange sense for continuous neutral type recurrent neural networks (NRNNs) with multiple time delays is studied. Three different types of activation functions are considered, including general bounded and two types of sigmoid activation functions. By constructing appropriate Lyapunov functions, some easily verifiable criteria for the ultimate boundedness and global exponential attractivity of NRNNs are obtained. These results can be applied to monostable and multistable neural networks as well as chaos control and chaos synchronization.

Introduction

Recurrent neural networks (RNNs) have found many applications since the pioneering work of Hopfield [11]. In employing RNNs to solve optimization, control, or signal processing problems, one of the most desirable properties of RNNs is the Lyapunov global stability. From a dynamical system point of view, globally stable networks in Lyapunov sense are monostable systems, which have a unique equilibrium attracting all trajectories asymptotically. A large body of research now exists on the study of global asymptotic stability for RNNs. We refer to [1], [5], [6], [13], [14], [15], [16], [19], [20], [21], [22], [23], [24], [26], [29], [36], [38], [39] and the references therein for detailed mathematical analysis on global convergence of various neural network models. In many other applications, however, monostable neural networks have been found computationally restrictive and multistable dynamics are essential to deal with important neural computations desired. For example, in a winner-take-all network [31], [34], where, depending on the external input (or the initial value), only the neuron with the strongest input (or highest initial value) should remain active. This is possible only if there are multiple equilibria with some being unstable. When a neural network is used as an associative memory storage or for pattern recognition, the existence of many equilibria is also necessary [4], [7], [11], [27]. In these applications, the neural networks are no longer globally stable, and more appropriate notions of stability are needed to deal with multistable systems.

Motivated by Yi and Tan [36], who take the view that boundedness, attractivity and complete convergence are three basic properties of a multistable network, we study the first two properties in this paper for NRNNs. More specifically, we generalize the conventional notion of stability in Lyapunov sense and study the global exponential stability (GES) in Lagrange sense for NRNNs with time delay. Our work extends the previous results obtained by Liao and Wang [17], [18], [20]. By constructing appropriate Lyapunov-like functions, we provide easily verifiable criteria for the boundedness of the networks and the existence of globally exponentially attractive (GEA) sets. Three different types of neuron activation functions are considered. Because we do not make any assumptions of the number of equilibria, our results can be used in analyzing both monostable and multistable networks. Once a network is proved to be globally exponentially stable in Lagrange sense, one needs only to focus the study of its dynamics inside the (compact) attractive set, where stable and unstable equilibria, periodic orbits, or even chaotic attractors may coexist. These more complex and richer dynamics are essential for the networks to possess multiple stable patterns that are often important in practical applications [2], [3], [8], [9], [25], [40], [41], [42]. In the special case where the compact attractive set is a single point, the network is then globally stable in Lyapunov sense and the attractive set is the unique equilibrium point.

It is worth to mention that unlike Lyapunov stability, Lagrange stability refers to the stability of the total system, rather than the stability of equilibrium points. The boundedness of solutions and the existence of global attractive sets lead to a total system concept of stability: (asymptotic) Lagrange stability. Our paper extends this concept of stability to neural networks and considers it as a compatible method for assessing the stability of multistable neural networks and their mathematical models.

We note that Lagrange stability has long been studied in theory and applications of dynamical systems. In [12], [37], LaSalle and Yoshizawa apply Lyapunov functions to study Lagrange stability. In [30], Rekasius considers asymptotic stability in Lagrange sense for nonlinear feedback control systems. In [32], Lagrange stability is discussed by Thornton and Mulholland as a useful concept for determining the stability of ecological systems. More recently, Passino and Burgess [28] adapt the concept of Lagrange stability to investigate discrete event systems, and Hassibi et al. [10] study the Lagrange stability of hybrid dynamical systems. See also [33], [35] for recent results on Lagrange stability for pendulum-like systems.

In this paper, we will further extend and generalize the results of our works [17], [18], [20] from RNNs to neutral NRNNs. The paper is organized as follows. In Section 2, we define the notion of GES in Lagrange sense and give a lemma, which will be used in proofs of all results of the paper. Section 3 provides main results for GES of NRNNs with three different types of activation functions. Then an application is obtained in Section 4. Finally,we give an example to illustrate the applications of our results in Section 5.

Section snippets

Preliminaries

Consider the following NRNNs with multiple time delays ${\begin{array}{l} c_{i} \frac{d x_{i} (t)}{d t} = - d_{i} x_{i} (t) + \sum_{j = 1}^{n} {\tilde{a}}_{i j} f_{j} (x_{j} (t)) + \sum_{j = 1}^{n} b_{i j} g_{j} (x_{j} (t - τ_{i j} (t))) + \sum_{j = 1}^{n} c_{i j} h_{j} ({\dot{x}}_{j} (t - τ_{i j} (t))) + I_{i} (t) \\ x_{i} (t) = φ_{i} (t) t \in [- τ, 0] \\ {\dot{x}}_{i} (t) = ψ_{i} (t) t \in [- τ, 0] \end{array}$ where $i = 1, 2, \dots, n$ , c_i>0 and d_i>0 are constants, x_i(t) is the state variable of the i-th neuron, $I_{i} (t) \in C (R, R)$ is variable input (bias), ${\tilde{a}}_{i j}, b_{i j}$ and c_ij are connection weights from neuron i to neuron j, τ_ij(t) are variable time delays, $f_{i} \in C (R, R)$ is the i-th neuron activation function, g_i, h_i are bounded

$f (\cdot), g (\cdot), h (\cdot) \in B$

Theorem 1

If the activation functions f(⋅), g(⋅), h(⋅) are all bounded; i.e., $| f_{i} (\cdot) | \leq k_{i}, | g_{i} (\cdot) | \leq l_{i}, | h_{i} (\cdot) | \leq {\bar{l}}_{i}$ , where $k_{i}, l_{i}, {\bar{l}}_{i}$ are all positive constants and x_if_i(x_i)>0, for $x_{i} \neq 0, f_{i} (0) = 0, i = 1, 2, \dots, n$ , then NRNN(1) is globally exponentially stable in Lagrange sense and Ω₁, Ω₂, Ω₃ are all GES sets, the set Ω=Ω₁∩Ω₂∩Ω₃ is a better GES for NRNN(1), where

Ω_{i} = {\frac{1}{2} \sum_{j = 1}^{n} (| a_{i j} | k_{j} + | b_{i j} | l_{j} + | c_{i j} | {\bar{l}}_{i}) + | I_{i} |} Ω_{1} = {x | \sum_{i = 1}^{n} \frac{c_{i} x_{i}^{2} (t)}{2} \leq (\sum_{i = 1}^{n} \frac{M_{i}^{2}}{ε_{i}}) / \frac{2 \min_{1 \leq i \leq n} (d_{i} - ε_{i})}{c_{i}}, 0 < ε_{i} < d_{i}} Ω_{2} = {x | | x_{i} | \leq \frac{2 M_{i}}{d_{i}}, i = 1, 2, \dots, n} Ω_{3} = {x | \sum_{i = 1}^{n} c_{i} | x_{i} | \leq \frac{\sum_{i = 1}^{n} 2 M_{i}}{\min}

An application

The equilibrium points, periodic solution and almost periodic solution of Eq. (1) and (2) or Eq. (3) are all positive invariant sets. Therefore, as a direct application of the results obtained in the previous, we have the following theorem.

Theorem 5

Let Ω is a GES and positive invariant set of Eq. (1) or Eq. (2) or Eq. (3), then outside the Ω, there are no bounded positive invariant sets that do not intersect Ω.

Proof

By contraries, suppose Q is the positive invariant set outside the set Ω. Thus, $\inf_{\bar{X} \in Ω X \in Q} ‖ X - \bar{X}$

Example

Example 1

Consider a 3-dimensional NRNN

(\begin{matrix} c_{1} {\dot{x}}_{1} (t) \\ c_{2} {\dot{x}}_{2} (t) \\ c_{3} {\dot{x}}_{3} (t) \end{matrix}) = (\begin{matrix} - d_{1} x_{1} (t) \\ - d_{2} x_{2} (t) \\ - d_{3} x_{3} (t) \end{matrix}) + (\begin{array}{l} - a_{11} & - a_{12} & a_{13} \\ a_{12} & - a_{22} & a_{23} \\ - a_{13} & - a_{23} & a_{33} \end{array}) (\begin{matrix} f (x_{1} (t)) \\ f (x_{2} (t)) \\ f (x_{3} (t)) \end{matrix}) + (\begin{matrix} b_{11} & b_{12} & b_{13} \\ b_{21} & b_{22} & b_{23} \\ b_{31} & b_{32} & b_{33} \end{matrix}) (\begin{matrix} y (x_{1} (t - τ_{1} (t))) \\ y (x_{2} (t - τ_{2} (t))) \\ y (x_{3} (t - τ_{3} (t))) \end{matrix}) + (\begin{array}{l} c_{11} & c_{12} & c_{13} \\ c_{21} & c_{22} & c_{23} \\ c_{31} & c_{32} & c_{33} \end{array}) (\begin{matrix} h ({\dot{x}}_{1} (t - {\tilde{τ}}_{1} (t))) \\ h ({\dot{x}}_{2} (t - {\tilde{τ}}_{2} (t))) \\ h ({\dot{x}}_{3} (t - {\tilde{τ}}_{3} (t))) \end{matrix}) + (\begin{matrix} I_{1} (t) \\ I_{2} (t) \\ I_{3} (t) \end{matrix}) f (x) = \frac{e^{x} - e^{- x}}{e^{x} + e^{- x}}, g (x) = \frac{2 x}{1 + x^{2}}, h (x) = arctg x,

a_{i j}

b_{i j}

c_{i j}

are all constants,

0 < τ_{i} \leq T

a_{i i} > 0

Take P = (\begin{array}{l} 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & - a_{33} \end{array}) then \bar{A} = (\begin{array}{l} - {\bar{a}}_{11} & - {\bar{a}}_{12} & {\bar{a}}_{13} \\ {\bar{a}}_{12} & - {\bar{a}}_{22} & {\bar{a}}_{23} \\ - {\bar{a}}_{13} & - {\bar{a}}_{23} & 0 \end{array}) \bar{A} + {\bar{A}}^{T} \leq 0

According to Theorem

Acknowledgement

This work was supported by the Natural Science Foundations of China under Grants 60874110 and 60974021.

Qi Luo received his Ph.D. degree in systems engineering from South China University of Technology, Guangzhou, China, in 2004. His current research interests include stability of stochastic systems and automatic control.

He is a Professor and Ph.D. advisor in the College of Information and Control, Nanjing University of Information Science and Technology, China.

References (42)

C.Y. Cheng et al.
Multistability and convergence in delayed neural networks.
Physica
(2007)
M. Forti
On global asymptotic stability of a class nonlinear systems arising in neural network theory
J. Differential Equations
(1994)
X.X. Liao et al.
Global exponential stability in Lagrange sense for recurrent neural networks with time delays
Nonlinear Anal. Real World Appl.
(2008)
X.X. Liao et al.
Positive invariant and global exponential attractive sets of neural networks with time delays
Neurocomputing
(2008)
H.H. Mai et al.
A semi-free weighting matrices approach for neutral-type delayed neural networks
J. Comput. Appl. Math.
(2009)
B. Mensour et al.
Controlling chaos to store information in delay differential equations
Phys. Lett. A
(1995)
M. Morita
Associative memory with nonmonotone dynamics
Neural Networks
(1993)
K.W. Thornton et al.
Lagrange stability and ecological systems
J. Theor. Biol.
(1974)
J. Wang et al.
Control of a class of pendulum-like systems with Lagrange stability
Automatica
(2006)
Y.Y. Chen et al.
Global exponential stability in Lagrange sense for a class of neural networks
J. Nanjing Univ. Inform. Sci. Technol.: Nat. Sci. Ed.
(2009)

C.Y. Cheng et al.

Multistability in recurrent neural networks

SIAM J. Appl. Math.

(2006)

L.O. Chua et al.

Cellular neural networks: theory

IEEE Trans. Circuits Syst.

(1988)

M. Forti et al.

New conditions for global stability of neural networks with application to linear and quadratic programming problems

IEEE Trans. Circuits Syst. I

(1995)

J. Foss et al.

Multistability and delayed recurrent loops

Phys. Rev. Lett.

(1996)

R. Hahnloser

On the piecewise analysis of liner threshold neurons

Neural Networks

(1998)

R. Hahnloser et al.

Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit

Nature

(2000)

A. Hassibi,, S.P. Boyd, J.P. How., A class of Lyapunov functionals for analyzing hybrid dynamical systems, in:...

J.J. Hopfield

Neurons with graded response have collective computational properties like those of two-state neurons

Proc. Natl. Acad. Sci.

(1984)

J. LaSalle

Some extensions of Lyapunov's second method

IRE Trans. Circuit Theory

(1960)

X.X. Liao

Stability of Hopfield-type neural networks

Sci. China Ser. A

(1993)

X.X. Liao

Mathematical theory of cellular networks(1)

Sci. China Ser. A

(1994)

Cited by (61)

Lagrange stability criteria for hypercomplex neural networks with time varying delays
2024, Communications in Nonlinear Science and Numerical Simulation
This article deals with the Lagrange stability (LS) of hypercomplex neural networks (HCNNs) with time-varying delays. To overcome the non-commutativity and non-associativity of HCNNs, the HCNNs are separated into equivalent $n + 1$ real-valued neural networks (RVNNs). Then, by Lyapunov theory, LS and the globally attractive exponential set are obtained. Since there are no assumptions about the number of equilibria, these results can also be used to evaluate monostable, multistable, and higher neural networks. Since, complex-valued neural networks (CVNNs), quaternion-valued neural networks (QVNNs), and octonion-valued neural networks (OVNNs), etc., are the particular cases of the HCNNs, therefore our results are more general. Finally, three numerical examples for CVNNs, QVNNs, and OVNNs are provided and analyzed to validate the obtained results.
New results on anti-synchronization in predefined-time for a class of fuzzy inertial neural networks with mixed time delays
2022, Neurocomputing
This paper investigates the anti-synchronization in predefined-time for fuzzy inertial neural networks (FINNs) with mixed delays. In the light of predefined-time stability theorems, the FINNs can reach anti-synchronization by employing two distinctive bilayer predefined-time control inputs which the setting-time is a explicit value without considering on initial values that has a great improvement over other settlement times in previous researches. Based on Lyapunov stability theory, sufficient conditions are converted to a type of algebraic inequalities which are very concise and avoid complicated calculations. In the end, some numerical examples and applications are illustrated to validate reliability of proposed results here.
Global Lagrange stability analysis of retarded SICNNs
2021, Chaos, Solitons and Fractals
The stability in the Lagrange sense for cellular neural networks (CNNs) has proven to be one of the effective tools to study multi-stable dynamics of the neural networks. In this article, rather than studying the existence and Lyapunov stability of an equilibrium point we investigate multi-stable dynamics of shunting inhibitory cellular neural networks (SICNNs) with time-varying delays and coefficients. This is the first paper that addresses the Lagrange stability for SICNNs. By constructing proper Lyapunov functions and using inequality techniques, we analyze three different types of activation functions, namely, bounded, sigmoid and Lipschitz-like type activation functions. New delay-dependent sufficient criteria are derived to ensure the global Lagrange stability for SICNNs. Furthermore, globally exponentially attractive sets are given for the different activation functions. Finally, an illustrating example with numerical simulations is given to support the theoretical results.
Lagrange stability of memristive quaternion-valued neural networks with neutral items
2020, Neurocomputing
This paper concentrates on the global exponential Lagrange stability of memristive quaternion-valued neural networks (MQVNNs) with neutral items. Some classical restrictions on the activation function of neutral item are removed. A new Lyapunov-Krasovskii functional (LKF) including information of neutral items is designed to overcome the difficulty induced by the coexist of quaternions, memristor, and neutral item. Furthermore, in order to reduce the conservativeness, both reciprocally convex inequality and Wirtinger-based inequality are extended to the quaternion domain. Based on the extended inequalities, Lyapunov theory, and novel analytical techniques, concise criteria in the form of linear matrix inequalities (LMIs) are proposed to ascertain the Lagrange stability of the interested MQVNNs. Finally, the correctness and effectiveness of theoretical results are checked by two numerical examples.
Global Lagrange stability for neutral-type inertial neural networks with discrete and distributed time delays
2020, Chinese Journal of Physics
This paper focus on the problem of global Lagrange stability for neutral-type inertial neural networks with discrete and distributed time delays. By choosing a proper variable substitution, an inertial neural network consisting of second-order differential equations can be converted into a first-order differential model. The sufficient conditions of the inertial neural network with neutral delay are derived by constructing suitable Lyapunov-Krasovskii functional candidates, introducing new free weighting matrices, utilizing inequality techniques and analytical method. Through the LMI condition, we analyze the global exponential stability of the delayed inertial neural networks in Lagrange sense. Meanwhile, the global exponential attractive set is also given. Finally, some example is given to illustrate our theoretical results.
Global exponential dissipativity of neutral-type BAM inertial neural networks with mixed time-varying delays
2020, Neurocomputing
This paper considers the global exponential dissipativity of neutral-type BAM inertial neural networks with mixed time-varying delays. Firstly, we transform the proposed BAM inertial neural networks to usual one. Secondly, by establishing a new neutral-type differential inequality and employing Lyapunov method and analytical techniques, some novel sufficient conditions in accordance with algebraic and linear matrix inequalities are obtained for the global exponential dissipativity of the addressed neural networks. Moreover, the globally exponentially attractive sets and the exponential convergence rate index are also assessed. Finally, the effectiveness of the obtained results is illustrated by some examples with numerical simulations.

View all citing articles on Scopus

He is a Professor and Ph.D. advisor in the College of Information and Control, Nanjing University of Information Science and Technology, China.

Zhigang Zeng received his B.S. degree in mathematics from Hubei Normal University, Huangshi, China, and his M.S. degree in ecological mathematics from the Hubei University, Hubei, China, in 1993 and 1996, respectively, and his Ph.D. degree in systems analysis and integration from the Huazhong University of Science and Technology, Wuhan, China, in 2003.

He is a Professor at the Department of Control Science and Engineering, the Huazhong University of Science and Technology, China. His current research interests include neural networks and stability analysis of dynamic systems.

Xiaoxin Liao received his B.S. degree in mathematics from the Wuhan University, China in 1963. His current research interests include stability of dynamical systems, neural networks, chaotic control, synchronization and automatic control.

He is a Professor and Ph.D advisor in the Department of Control Science and Engineering, the Huazhong University of Science and Technology, China. He is an editorial committee of the Applied Mathematics.

View full text

Global exponential stability in Lagrange sense for neutral type recurrent neural networks

Abstract

Introduction

Section snippets

Preliminaries

f(⋅),g(⋅),h(⋅)∈B

An application

Example

Acknowledgement

Physica

J. Differential Equations

Nonlinear Anal. Real World Appl.

Neurocomputing

J. Comput. Appl. Math.

Phys. Lett. A

Neural Networks

J. Theor. Biol.

Automatica

Global exponential stability in Lagrange sense for a class of neural networks

J. Nanjing Univ. Inform. Sci. Technol.: Nat. Sci. Ed.

Multistability in recurrent neural networks

SIAM J. Appl. Math.

Cellular neural networks: theory

IEEE Trans. Circuits Syst.

New conditions for global stability of neural networks with application to linear and quadratic programming problems

IEEE Trans. Circuits Syst. I

Multistability and delayed recurrent loops

Phys. Rev. Lett.

On the piecewise analysis of liner threshold neurons

Neural Networks

Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit

Nature

Neurons with graded response have collective computational properties like those of two-state neurons

Proc. Natl. Acad. Sci.

Some extensions of Lyapunov's second method

IRE Trans. Circuit Theory

Stability of Hopfield-type neural networks

Sci. China Ser. A

Mathematical theory of cellular networks(1)

Sci. China Ser. A

$f (\cdot), g (\cdot), h (\cdot) \in B$