Multistability of switched neural networks with sigmoidal activation functions under state-dependent switching

Abstract

This paper presents theoretical results on the multistability of switched neural networks with commonly used sigmoidal activation functions under state-dependent switching. The multistability analysis with such an activation function is difficult because state–space partition is not as straightforward as that with piecewise-linear activations. Sufficient conditions are derived for ascertaining the existence and stability of multiple equilibria. It is shown that the number of stable equilibria of an $n$-neuron switched neural networks is up to $3^n$ under given conditions. In contrast to existing multistability results with piecewise-linear activation functions, the results herein are also applicable to the equilibria at switching points. Four examples are discussed to substantiate the theoretical results.

Introduction

As known widely nowadays, neural networks play important roles in many technical areas, such as pattern recognition (Kwan and Cai, 1994, Suganthan et al., 1995, Zeng et al., 2005), associative memory (Isokawa et al., 2008, Zeng and Wang, 2008, Zeng and Wang, 2009), and other areas (Bao et al., 2018a, Bao et al., 2016). In recent decades, various neural network models, including Hopfield-type neural network model (Huang and Cao, 2010, Zeng et al., 2016), cellular neural networks with memristors (Bao and Zeng, 2013, Di et al., 2017, Duan et al., 2015), Cohen–Grossberg neural network model (Gang et al., 2007, Wang et al., 2006, Wang and Zou, 2002, Zhang and Wang, 2008), switched neural network model (Bao et al., 2018b, Li and Cao, 2007, Li et al., 2009, Liu et al., 2013, Zhao et al., 2016, Zhao and Zhao, 2017), have been developed and analyzed.

Because of the complexity of actual environment, the connections of neurons in a network may change frequently. Due to the link failure or a new creation of the connection topology in the network, the switching between some different connection topologies are inevitable (Daafouz et al., 2002, Zhao et al., 2009). To describe this switching phenomenon in the network, switched system is introduced. A switched system, consisting of a number of subsystems or configurations modeled by using differential or difference equations, operates in a mode switching among these subsystems. Generally speaking, two categories of switching modes are usually considered (Liberzon and Morse, 2015, Persis et al., 2003). One is time-dependent switching, i.e. the switching rule is controlled only by time. For example, a continuous switched system is characterized by the following differential equation:

$$\frac{dx\left(t\right)}{dt}=f_{\sigma (t,x)}\left(x\left(t\right)\right).$$

Let $F=\{f_p\left(x\right),p\in P\}$, the parameter $p$ takes the value in the index set $P$, each map $f_p\left(x\right):R^n\to R^n$ in $F$ is assumed to be locally Lipschitz, and $\sigma (t,x)=\sigma \left(t\right):[0,+\infty )\to P$ is the switching signal. The other type is state-dependent switching. In this case, the switching signal $\sigma (t,x)=\sigma \left(x\right):R^n\to P$ depends only on state $x$. Compared with the switched systems under time-dependent switching, switched systems under state-dependent switching with different initial values may take on different equations at the same instant. The dynamical analysis of switched systems is much more complicated and challenging than that of conventional ones. Due to the significant values of switched systems in both theory and practice, stability analysis of switched systems is an attractive topic; e.g., Li et al., 2005, Wu et al., 2011, Yuan et al., 2006 and Zhang, Tang, Miao, and Du (2013).

Multistability is a notion to characterize the coexistence of multiple stable equilibrium points or periodic solutions (Cheng et al., 2006, Liu et al., 2016a, Liu et al., 2016b, Liu et al., 2017a, Liu et al., 2017b, Liu et al., 2017c, Liu et al., 2018, Nie and Zheng, 2015, Wang, 2014, Wang and Chen, 2012, Wang et al., 2009, Wang et al., 2010). It is well recognized that multistability analysis of neural networks depends critically upon the type of activation functions. With different activation functions, different multistability stability criteria can be derived. For example, in an $n$-neuron Hopfield neural network with Mexican-hat-type activation functions, there exist at most $3^n$ equilibrium points and at most $2^n$ of them are locally stable and others are unstable (Wang, 2014, Wang and Chen, 2012). In a neural system with Gaussian activation functions, there exist $3^k$ equilibrium points, in which $2^k$ are exponentially stable, where $1\le k\le n$ (Liu et al., 2017a). In a neural system with piecewise-linear nondecreasing activation functions, there exist ${(2r+1)}^n$ equilibria, in which ${(r+1)}^n$ are locally exponentially stable, where $r$ is the number of pieces in the piecewise-linear activation functions (Wang et al., 2010). In a neural network with discontinuous Mexican-hat-type activation functions under reasonable conditions, there exist $5^n$ equilibrium points in $R^n$ and $4^n$ of them are located at the continuous part of the activation functions (Nie & Zheng, 2015).

Numerous results are available for the stability analysis of the switched systems, e.g., Branicky, 1998, Chen et al., 2010, Guo et al., 2018, Guo et al., 2019, Guo et al., 2014, Hu et al., 2015, Kahloul and Sakly, 2018, Li and Cao, 2007, Lian and Wang, 2015, Long and Wei, 2007, Long et al., 2013, Nie and Cao, 2015, Niu et al., 2017, Singh and Sukavanam, 2012, Song et al., 2018, Song et al., 2006, Tang and Zhao, 2017 and Yu, Fei, Yu, and Fei (2011). The global stability of switched neural networks and the control of switched systems are addressed in Branicky, 1998, Chen et al., 2010, Hu et al., 2015, Kahloul and Sakly, 2018, Li and Cao, 2007, Long and Wei, 2007, Long et al., 2013, Niu et al., 2017, Singh and Sukavanam, 2012, Song et al., 2018, Song et al., 2006, Tang and Zhao, 2017 and Yu et al. (2011).

In contrast to global stability, few results are available on the multistability of switched neural networks, especially under state-dependent switching, e.g., Guo et al., 2018, Guo et al., 2019, Guo et al., 2014 and Nie and Cao (2015). The multistability results of memristive neural networks with non-monotonic piecewise-linear activation functions are presented in Nie and Cao (2015). The existence and attractivity of multiple equilibria of memristor-based cellular neural networks with piecewise-linear activation functions are addressed in Guo et al. (2014). The multistability results of switched neural networks with piecewise-linear radial basis functions and monotonic piecewise-linear functions under state-dependent switching are presented in Guo et al., 2018, Guo et al., 2019.

As is well known, state–space partition is an important step for multistability analysis. In all existing results on multistability of switched neural networks, the activation functions considered are piecewise linear. The state–space is partitioned at the breakpoints of piecewise linear activation functions. However, state space partition is not straightforward for switched neural networks with smooth activation functions. The difficulties of multistability analysis are summarized as follows:

• State-space partition is an important step for multistability analysis. For switched neural networks with smooth activation functions, state–space partition is nontrivial.

• Due to the discontinuity of switched neural networks, the upper and lower bound functions are also discontinuous, which leads to the difficulty on extreme point analysis.

• Due to the discontinuity of switched neural networks, the existing analysis methods on the multistability are invalid. The existence and stability of equilibria with at least one component at a switching point need to be considered.

This paper addresses the multistability of switched neural networks with sigmoidal activation functions under state-dependent switching. By combing Brouwer fixed theorem, local linearized method, state space partition and discussion of switching threshold, sufficient conditions are derived for ascertaining the existence and stability of multiple equilibria. Compared with related existing works, the main contributions of this paper are listed as follows:

• There is no multistability results for switched neural networks with sigmoidal activation functions.

• The number of stable equilibria of an $n$-neuron switched neural networks is up to $3^n$ under given conditions.

• Sufficient conditions are derived for ascertaining the existence and stability of the maximal number and some numbers of equilibria on and off switching points.

• A switching rule is given for resulting in desirable number of stable equilibria.

The rest of paper is organized as follows. In Section 2, a switched neural network model with sigmoidal activation function under state-dependent switching is proposed. Then, some definitions and lemmas are presented. In Section 3, the multistability results of switched neural systems are addressed. In Section 4, four examples are given to substantiate the effectiveness of the obtained criteria. In Section 5, concluding remarks are given.