Multistability and multiperiodicity of high-order competitive neural networks with a general class of activation functions

doi:10.1016/j.neucom.2011.09.032

Neurocomputing

Volume 82, 1 April 2012, Pages 1-13

https://doi.org/10.1016/j.neucom.2011.09.032 Get rights and content

Abstract

In this paper, high-order synaptic connectivity is introduced into competitive neural networks and the multistability and multiperiodicity issues are discussed for high-order competitive neural networks with a general class of activation functions. Based on decomposition of state space, Halanay inequality, Cauchy convergence principle and inequality technique, some sufficient conditions are derived for ascertaining equilibrium points to be located in any designated region and to be locally exponentially stable. As an extension of multistability, some similar results are presented for ensuring multiple periodic solutions when external inputs and time delay are periodic. The obtained results are different from and less restrictive than those given by Nie and Cao (2009 [25]), and the assumption $(H_{1}^{A})$ by Nie and Cao (2009 [25]) is relaxed. It is shown that high-order synaptic connectivity plays an important role on the number of equilibrium points and their dynamics. As a consequence, our results refute traditional viewpoint: high-order synaptic connectivity has faster convergence rate and greater storage capacity than first-order one. Finally, three examples with their simulations are given to show the effectiveness of the obtained results.

Introduction

It is well known that neural networks play important roles in many applications, such as image processing and optimization problem, etc. [1], [2], [3], [4]. In [5], the authors pointed out that the traditional neural networks with first-order synaptic connections are shown to have some limitations such as limited capacity when used in pattern recognition and optimization problems. By incorporating high-order synaptic connectivity into neural networks, the networks would improve dramatically their storage capacity [6] and increase the class of optimization problem [7]. In recent years, the stability analysis of high-order neural networks has attracted the attention of many researchers due to the advantage of high-order synaptic connectivity [8], [9], [10], [11], [12], [13], [14], [15], [16]. On the other hand, in the applications of associative memory storage, pattern recognition and decision making, existence of many equilibrium points or periodic solutions is a necessary feature [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35]. We use the notion of “multistability” or “multiperiodicity” to describe coexistence of multiple stable equilibrium points or periodic orbits. However, it is worth noting that: (1) Most of the existing works of high-order neural networks have only studied mono-stability and mono-periodicity, seldom have considered multistability and multiperiodicity of neural networks with high-order synaptic connectivity [31]. (2) In many previous papers [8], [9], [12], [13], [14], authors claim that one of their main motivations of research relies on the following basic facts: high-order neural networks have stronger approximation property, faster convergence rate, greater storage capacity and higher fault tolerance than first-order ones. Thus, an interesting question arises: does high-order synaptic connectivity always imply faster convergence rate and greater storage capacity?

Based on above motivations and inspired by [31], in this paper, we shall introduce high-order synaptic connectivity into competitive neural networks and study the multistability and multiperiodicity of high-order competitive neural networks (HOCNNs) with a general class of activation functions, where the general class of activation functions consist of general bounded smooth sigmoidal functions including $1 / (1 + e^{- ξ})$ , $\tanh ξ$ and $\arctan ξ$ as its special case. More precisely, we shall mainly focus on the following aspects:

•
For HOCNNs, we give some conditions to guarantee that the equilibrium point exists in any designated region, and is locally exponentially stable, by means of decomposition of state space, Halanay inequality, Cauchy convergence principle and inequality technique.
•
As an extension of multistability, we present the similar results for multiple periodic solutions when external inputs and time delay are periodic.
•
The obtained results are different from and less restrictive than those given in [25], and the assumption $(H_{1}^{A})$ in [25] is relaxed.
•
We reveal the effects of high-order synaptic connectivity on the existence and stability of equilibrium point or periodic solutions for HOCNNs and obtain a new insight: high-order synaptic connectivity may slow down convergence rate and reduce storage capacity of HOCNNs.

Section snippets

Model formulation and preliminaries

Competitive neural networks (CNNs) model the dynamics of cortical cognitive maps with unsupervised synaptic modifications. In this model, there are two types of state variable: that of the short-term memory (STM) describing the fast neural activity and that of long-term memory (LTM) describing the slow unsupervised synaptic modifications. The network can be arranged in a multilayer architecture, or the competition among the neurons can be implemented in just one competitive layer. The

Main results

In this section, we shall consider local exponential stability of equilibrium points for model (3) in each division region. Similar results can be also derived for HOCNNs with periodic delays and external inputs.

Theorem 1

Assume that Assumption 1, Assumption 2 hold. Then for each division $\tilde{N} = (N_{1}, N_{2}, N_{3})$ of N, each $Ω_{\tilde{N}}$ is positively invariant with respect to the solution flow generalized by system (3) with activation function (4).

Proof

For each division region $Ω_{\tilde{N}}$ and any initial condition $(ϕ^{T}, ψ^{T})^{T}$ $= (ϕ_{1}, \dots, ϕ_{N}, ψ_{1},$

Three illustrative examples

For convenience, we consider the following two-dimensional competitive neural networks: $\{\begin{matrix} \frac{d x_{i} (t)}{d t} = - a_{i} x_{i} (t) + \sum_{j = 1}^{2} D_{ij} f_{j} (x_{j} (t)) \\ + \sum_{j = 1}^{2} \sum_{l = 1}^{2} D_{ijl}^{τ} f_{j} (x_{j} (t - τ_{j} (t))) f_{l} (x_{l} (t - τ_{l} (t))) \\ + \sum_{j = 1}^{2} {\bar{D}}_{ij} \int_{t - σ}^{t} k_{ij} (t - s) f_{j} (x_{j} (s)) d s + B_{i} S_{i} (t) + I_{i}, \\ \frac{d S_{i} (t)}{d t} = - S_{i} (t) + f_{i} (x_{i} (t)), i = 1,2 . \end{matrix}$

Example 1

For system (28), take $a_{1} = B_{1} = 1$ , $a_{2} = B_{2} = 2$ , $D_{11} = 3$ , $D_{22} = 5$ , $D_{12} = D_{21} = {\bar{D}}_{11} = {\bar{D}}_{22} = 0$ , ${\bar{D}}_{12} = 0.5$ , ${\bar{D}}_{21} = - 0.5$ , $I_{1} = 0.5$ , $I_{2} = - 1$ , $σ = 5$ , $D_{1 jl}^{τ} = D_{2 jl}^{τ} = 0 (j, l = 1,2)$ , $k_{11} (s) = k_{22} (s) = 0$ , $k_{12} (s) = 2 e^{- 2 s} / (1 - e^{- 10})$ , $k_{21} (s) = 3 e^{- 3 s} / (1 - e^{- 15})$ , $f_{i} (x) = \arctan x (i = 1,2)$ .

By simple computations, we

Conclusions

In this paper, the multistability and multiperiodicity issues have been studied for HOCNNs with mixed delays and a general class of activation functions.

(1)
In view of the fact that neural information is often desired to be stored in a designated region, under some conditions, one obtained that HOCNNs have a unique locally exponentially stable equilibrium point in any designated region. Similar results are presented for multiple periodic solutions when external input and delays are periodic.
(2)
We

Acknowledgments

The authors would like to thank the reviewers and the editor for their valuable suggestions and comments which have led to a much improved paper. This work was jointly supported by the National Natural Science Foundation of China under Grant 11072059 and 111087, the Foundation for Young Professors of Jimei University and the Foundation of Fujian Higher Education (JA10184, JA11154, JA11144)

Xiaobing Nie received the B.S. degree in mathematics from Yantai Normal University, Yantai, China, in 1999, the M.S. degree in mathematics/applied mathematics from East China Normal University, Shanghai, China, in 2002, and the Ph.D. degree in mathematics/applied mathematics from Southeast University, Nanjing, China, in 2010.

Since July 2002, he has been with the Department of Mathematics, Southeast University, Nanjing, China. He is a very active reviewer for many international journals and has

References (40)

M. Morita
Associative memory with non-monotone dynamics
Neural Networks
(1993)
F.L. Ren et al.
LMI-based criteria for stability of high-order neural networks with time-varying delay
Nonlinear Anal.: Real World Appl.
(2006)
J.D. Cao et al.
Exponential stability of high-order bidirectional associative memory neural networks with time delays
Physica D
(2004)
J. Zhang et al.
Existence and stability of periodic solutions of high-order Hopfield neural networks with impulses and delays
J. Comput. Appl. Math.
(2009)
H.B. Gu et al.
Stability and periodicity in high-order neural networks with impulsive effects
Nonlinear Analy.: Real World Appl.
(2008)
D.W.C. Ho et al.
Global exponential stability of impulsive high-order BAM neural networks with time-varying delays
Neural Networks
(2006)
C.X. Ou
Anti-periodic solutions for high-order Hopfield neural networks
Comput. Math. Appl.
(2008)
B. Xiao et al.
Existence and exponential stability of positive almost periodic solutions for high-order Hopfield neural networks
Appl. Math. Model.
(2009)
W.N. Zhou et al.
Mixed time-delays dependent exponential stability for uncertain stochastic high-order neural networks
Appl. Math. Comput.
(2009)
C.Y. Cheng et al.
Multistability and convergence in delayed neural networks
Physica D
(2007)

J.D. Cao et al.

Multistability and multiperiodicity of delayed Cohen-Grossberg neural networks with a general class of activation functions

Physica D

(2008)

X.B. Nie et al.

Multistability of competitive neural networks with time-varying and distributed delays

Nonlinear Analy.: Real World Appl.

(2009)

G. Huang et al.

Multistability in bidirectional associative memory neural networks

Phys. Lett. A

(2008)

L.L. Wang et al.

Coexistence and local stability of multiple equilibria in neural networks with piecewise linear nondecreasing activation functions

Neural Networks

(2010)

W. Pan et al.

Monostability and multistability of genetic regulatory networks with different types of regulation functions

Nonlinear Analy.: Real World Appl.

(2010)

W. Pan et al.

On multistability of delayed genetic regulatory networks with multivariable regulation functions

Math. Biosci.

(2010)

Q.K. Song

Exponential stability of recurrent neural networks with both time-varying delays and general activation functions via LMI approach

Neurocomputing

(2008)

Q.K. Song et al.

Dynamical behaviors of fuzzy reaction-diffusion periodic cellular neural networks with variable coefficients and delays

Appl. Math. Model.

(2009)

L.O. Chua et al.

Cellular neural networks: theory

IEEE Trans. Circuits Syst.

(1988)

J. Hopfield

Neurons with graded response have collective computational properties like those of two state neurons

Proc. Natl. Acad. Sci. USA

(1984)

Cited by (53)

Multistability analysis of complex-valued recurrent neural networks with sine and cosine activation functions
2024, Neurocomputing
This paper is devoted to the multistability problem for complex-valued recurrent neural networks (CVRNNs) with a specific class of piecewise nonlinear activation functions. Firstly, a general class of piecewise nonlinear activation functions is presented to facilitate further analysis. Then, by resorting to the fixed point theorem and Lagrange’s mean value theorem, sufficient criteria are established which ascertain that the existence of ${(2 k + 1)}^{2 n}$ equilibrium points. Meanwhile, it also verifies that ${(k + 1)}^{2 n}$ equilibrium points of the considered CVRNNs with a piecewise nonlinear activation function are stable, where $k$ stands for a positive real number, which is associated with the frequency of sine and cosine functions. Moreover, the utilization of the activation functions provides a larger storage capacity in associative memory application. In the end of paper, three numerical examples are presented to demonstrate the feasibility and validity of the achieved theoretical results.
Multistability of switched complex-valued neural networks with state-dependent switching rules
2023, Neurocomputing
The paper focuses on the multistability problems of the switched complex-valued neural networks with state-dependent switching rules. Based on the differential inclusions theory and fixed point theorem, several sufficient conditions are derived to ascertain that there exist $25^{n}$ equilibria, $9^{n}$ of which are locally ex for n-neuron switched complex-valued neural networks. The number of stable equilibria of an n-neuron switched complex-valued neural network increases significantly from $4^{n}$ to $9^{n}$ compared with the conventional complex-valued neural networks. Finally, four numerical examples are presented to substantiate the theoretical results.
Coexistence and local stability of multiple equilibrium points for fractional-order state-dependent switched competitive neural networks with time-varying delays
2023, Neural Networks
This paper investigates the coexistence and local stability of multiple equilibrium points for a class of competitive neural networks with sigmoidal activation functions and time-varying delays, in which fractional-order derivative and state-dependent switching are involved at the same time. Some novel criteria are established to ensure that such $n$ -neuron neural networks can have $5^{m_{1}} \cdot 3^{m_{2}}$ total equilibrium points and $3^{m_{1}} \cdot 2^{m_{2}}$ locally stable equilibrium points with $m_{1} + m_{2} = n$ , based on the fixed-point theorem, the definition of equilibrium point in the sense of Filippov, the theory of fractional-order differential equation and Lyapunov function method. The investigation implies that the competitive neural networks with switching can possess greater storage capacity than the ones without switching. Moreover, the obtained results include the multistability results of both fractional-order switched Hopfield neural networks and integer-order switched Hopfield neural networks as special cases, thus generalizing and improving some existing works. Finally, four numerical examples are presented to substantiate the effectiveness of the theoretical analysis.
Multi-periodicity of switched neural networks with time delays and periodic external inputs under stochastic disturbances
2021, Neural Networks
This paper presents new theoretical results on the multi-periodicity of recurrent neural networks with time delays evoked by periodic inputs under stochastic disturbances and state-dependent switching. Based on the geometric properties of activation function and switching threshold, the neuronal state space is partitioned into $5^{n}$ regions in which $3^{n}$ ones are shown to be positively invariant with probability one. Furthermore, by using Itô’s formula, Lyapunov functional method, and the contraction mapping theorem, two criteria are proposed to ascertain the existence and mean-square exponential stability of a periodic orbit in every positive invariant set. As a result, the number of mean-square exponentially stable periodic orbits increases to $3^{n}$ from $2^{n}$ in a neural network without switching. Two illustrative examples are elaborated to substantiate the efficacy and characteristics of the theoretical results.
Multistability of delayed fractional-order competitive neural networks
2021, Neural Networks
This paper is concerned with the multistability of fractional-order competitive neural networks (FCNNs) with time-varying delays. Based on the division of state space, the equilibrium points (EPs) of FCNNs are given. Several sufficient conditions and criteria are proposed to ascertain the multiple $O (t^{- α})$ -stability of delayed FCNNs. The $O (t^{- α})$ -stability is the extension of Mittag-Leffler stability of fractional-order neural networks, which contains monostability and multistability. Moreover, the attraction basins of the stable EPs of FCNNs are estimated, which shows the attraction basins of the stable EPs can be larger than the divided subsets. These conditions and criteria supplement and improve the previous results. Finally, the results are illustrated by the simulation examples.
Synchronization control for memristive high-order competitive neural networks with time-varying delay
2019, Neurocomputing
This paper concerns the synchronization problem of memristive high-order competitive neural networks with time-varying delay. First, a novel control scheme with a linear term and a discontinuous term is proposed. Then, based on the Lyapunov stability theory, several criteria with algebraic form or matrix form are derived to ensure global exponential synchronization of the networks by adopting some inequality techniques. Finally, two numerical examples are presented to substantiate the effectiveness of the results.

View all citing articles on Scopus

Since July 2002, he has been with the Department of Mathematics, Southeast University, Nanjing, China. He is a very active reviewer for many international journals and has published more than 10 journal papers. His research interests include neural networks, multistability theory, nonsmooth system.

Zhenkun Huang received the M.S. degree from Fuzhou University, Fuzhou, China, in 2004, and the Ph.D. degree from Zhejiang University, Hangzhou, China, in 2008, both in applied mathematics. From September 2001, he joined School of Science, Jimei University, Xiamen, China, where he is currently an Associated Professor. He is selected into Training Program for Distinguished Young Scholars and Research Talents of Fujian Higher Education. He is the author or coauthor of more than 30 journal papers in such journals as IEEE Transactions on Neural Networks , IEEE Transactions on Circuits and Systems I: Regular Papers, Physics Letters A, Nonlinear Dynamics, Journal of Mathematical Analysis and Applications and so on. He is also a Reviewer of Mathematical Reviews. His research interests include nonlinear systems, neural networks, and stability analysis in dynamic systems (including continuous, discrete, and impulsive systems).

View full text

Multistability and multiperiodicity of high-order competitive neural networks with a general class of activation functions

Abstract

Introduction

Section snippets

Model formulation and preliminaries

Main results

Three illustrative examples

Conclusions

Acknowledgments

Neural Networks

Nonlinear Anal.: Real World Appl.

Physica D

J. Comput. Appl. Math.

Nonlinear Analy.: Real World Appl.

Neural Networks

Comput. Math. Appl.

Appl. Math. Model.

Appl. Math. Comput.

Physica D

Physica D

Nonlinear Analy.: Real World Appl.

Phys. Lett. A

Neural Networks

Nonlinear Analy.: Real World Appl.

Math. Biosci.

Neurocomputing

Appl. Math. Model.

Cellular neural networks: theory

IEEE Trans. Circuits Syst.

Neurons with graded response have collective computational properties like those of two state neurons

Proc. Natl. Acad. Sci. USA