Multistability of state-dependent switching neural networks with discontinuous nonmonotonic piecewise linear activation functions
Introduction
In recent years, neural networks are of significant importance due to their wide applications throughout science and engineering [1], [2]. Existing classical neural network models, including cellular neural network model, Hopfield-type neural network model, switching neural network model, have been introduced and applied in practice, such as associative memory [3], [4], [5], [6] and pattern classification [7], [8], [9]. The dynamic analysis plays a vital part in applications and designs of neural networks.
When applying neural networks in associative memory and pattern recognition, multiple stable equilibria is indispensable. Multistability has a strong correlation with storage capacity of neural networks. There are many differences between multistability analysis and monostability analysis. For the monostability analysis, we should pay attention to the global stability and uniqueness of equilibrium point [10], [11]. By comparison, for multistability analysis, the number of equilibria, multistability and domains of those equilibria should be investigated. On this account, many related results have been proposed [12], [13], [14], [15], [16], [17], [18], [19].
What is noteworthy is that multistability analysis mostly relies on the types of activation functions. In multistability analysis, the activation functions are mainly concentrated on nondecreasing saturated activation functions, sigmoid activation functions, Maxican–Hat-type activation functions and so on. In the existing works, it should be remarkable that the research on multistability basically concentrates on neural networks with continuous activation functions [20], [21], [22], [23], [24], [25], [26]. When handling dynamical systems with very high-slope nonlinear elements, we often to model them using discontinuous right sides, rather than using high slopes with limited values [27], [28], [29]. Thus, how to extend these results to the discontinuous activation functions has naturally become an important research topic.
When the system receives an external signal or the system operation meets certain conditions, the connection weight may vanish or change [30], [31]. To describe this switching phenomenon in the networks, switching system is introduced. According to the switching modes, we can divide switching systems into two categories [32], [33]. One is state-dependent switching system, namely the switching rule is determined by state. The other is time-dependent switching system, the corresponding switching rule is controlled by time. Compared with time-dependent switching system, state-dependent switching system may take on different initial values according to different initial status. The dynamical analysis of state-dependent switching is more complicated than that of conventional ones. Due to the great value of switching systems in both theory and practice, stability analysis of switching systems has been widely studied [34], [35], [36], [37], [38], [39], [40], [41].
Despite the stability of switching neural networks has been extensively studied, for all we know, the multistability of switching neural networks, especially state-dependent switching neural networks has been rarely investigated. The previous research mostly concentrated on continuous activation functions, so it cannot be directly applied to discontinuous activation functions. As a consequence, it is essential for us to study the multistability of discontinuous nonmonotonic piecewise linear activations.
Based on the above considerations, this paper discusses the multistability of state-dependent switching neural networks with discontinuous nonmonotonic piecewise linear activation functions. Compared with related existing works, the main contributions of this paper are as follows:
(1) For n-neurons state-dependent switching neural networks with discontinuous activation functions, this paper presents it have equilibrium points, of which are located at the continuous points of activation functions and of which are located at the discontinuous points of activation functions. Among these equilibrium points, or are stable and others are unstable, which depend on the relationship between the switching threshold and the discontinuous points of activation functions.
(2) Compared with multistability of general discontinuous nonmonotonic activation functions [37], the total number of equilibrium points increased to from , and the number of stable equilibrium points increased to or from . Compared with multistability of state-dependent switching neural networks with piecewise-linear radial basis functions [35], we get the similar conclusions as above. It reveals that switching threshold and discontinuity can increase the number of equilibrium points.
The remaining parts of this paper are organized as follows: In Section 2, a state-dependent switching neural network model with discontinuous nonmonotonic activation functions is proposed. Then, some definitions and lemmas are presented. In Section 3, the multistability results of switching neural systems are addressed. In Section 4, two examples are given to substantiate the theoretical results. In Section 5, conclusion is given.
Section snippets
Preliminaries
Consider the state-dependent switching neural network model as follows:where is the state vector; is the neurons self-feedback connection weights matrix with , is the inner-neuron connection weight matrix; is the input or bias vector, and is the activation function.
In this paper, a class of discontinuous nonmonotonic
Main results
In this section, the multistability of state-dependent switching neural networks with discontinuous nonmonotonic piecewise linear activation functions is analysed. Since is the discontinuous point of the activation function, for switching threshold , there are two main situations for us to discuss: Situation , Situation . ( can be seen in Remark 3).
For Situation , in the continuous points of activation function (3), we have the following conclusions. Theorem 1 If the following
Numerical simulations
In this section, two illustrative examples are given to verified the theoretical results. Neural network model (1) with two neurons is described by:where the activation function is defined as (3) that is .
First, Let us consider the Situation A, that is the switching threshold . Example 1 Neural networks (44) with the input term .
Conclusion
This paper study the multistability of a state-dependent switching neural network model with discontinuous nonmonotonic piecewise linear activation function. By analyzing the discontinuous points and switching threshold, sufficient conditions are derived for ascertaining the coexistence and stability of equilibrium points. We can conclude that switching threshold and discontinuity can increase the number of equilibrium points. In addition, the relationship between the discontinuous points of
CRediT authorship contribution statement
Jiahui Zhang: Conceptualization, Methodology, Software, Writing - original draft, Visualization, Validation, Writing - review & editing. Song Zhu: Conceptualization, Methodology, Writing - review & editing. Nannan Lu: Software. Shiping Wen: Conceptualization, Methodology, Writing - review & editing.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This work was supported by the National Natural Science Foundation of China under Grant 61873271, 62006233, the Fundamental Research Funds for the Central Universities 2018XKQYMS15 and the Double-First-Rate Special Fund for Construction of China University of Mining and Technology No. 2018ZZCX14.
Jiahui Zhang was born in 1993. She received the B.Admin. degree in information management and information system from Chang’an University, Xi’an, China, in 2015. She is currently pursuing the M.S. degree in operational research and cybernetics with the China University of Mining and Technology, Xuzhou, China. She current research interests include neural networks and stochastic control.
References (41)
- et al.
Neural networks for control systems—a survey
Automatica
(1992) - et al.
Associative memories based on continuous-time cellular neural networks designed using space-invariant cloning templates
Neural Netw.
(2009) - et al.
Pattern recognition by homomorphic graph matching using Hopfield neural networks
Image Vision Comput.
(1995) - et al.
Memory pattern analysis of cellular neural networks
Phys. Lett. A
(2005) - et al.
Global exponential stability of a class of memristor-based recurrent neural networks with time-varying delays
Neurocomputing
(2012) - et al.
Multiple μ)stability analysis of complex-valued neural networks with unbounded time-varying delays
Neurocomputing
(2015) - et al.
Multistability and instability analysis of recurrent neural networks with time-varying delays
Neural Netw.
(2018) - et al.
Multistability and attraction basins of discrete-time neural networks with nonmonotonic piecewise linear activation functions
Neural Netw.
(2020) - et al.
Coexistence and local stability of multiple equilibria in neural networks with piecewise linear nondecreasing activation functions
Neural Netw.
(2010) - et al.
Multistability of switched neural networks with sigmoidal activation functions under state-dependent switching
Neural Netw.
(2020)
Analysis and design of associative memories based on recurrent neural network with discontinuous activation functions
Neurocomputing
Multistability of complex-valued neural networks with discontinuous activation functions
Neural Netw.
Synchronization of complex dynamical networks with switching topology: a switched system point of view
Automatica
Novel results on finite-time stabilization of state-based switched chaotic inertial neural networks with distributed delays
Neural Netw.
Decentralized event-triggered control for interconnected systems with unknown disturbances
J. Franklin Inst.
Cellular neural networks: applications
IEEE Trans. Circuits Syst.
design and analysis of high-capacity associative memories based on a class of discrete-time recurrent neural networks
IEEE Trans. Syst. Man Cybern.
Memory analysis for memristors and memristive recurrent neural networks
IEEE/CAA J. Automatica Sin.
Multistability of delayed hybrid impulsive neural networks with application to associative memories
IEEE Trans. Neural Netw. Learn Syst.
A new recurrent neural-network architecture for visual pattern recognition
IEEE Trans. Neural Netw.
Cited by (10)
Finite-time dissipative control for bidirectional associative memory neural networks with state-dependent switching and time-varying delays
2022, Knowledge-Based SystemsCitation Excerpt :Recently there have been a number of researchers working on the dynamic behavior of BAMNNs, and significant findings have been obtained in stability analysis, synchronization control, and other dynamical behaviors (see [9–11] and references therein). Switching systems, as a special type of hybrid system, have been an important subject of engineering and physical research [12–15], widely used in robot control, automotive industry, aircraft control and other applications. It is well known that switching systems can be divided into two categories under different switching modes.
Almost periodic solutions of memristive multidirectional associative memory neural networks with mixed time delays
2024, International Journal of Biomathematics
Jiahui Zhang was born in 1993. She received the B.Admin. degree in information management and information system from Chang’an University, Xi’an, China, in 2015. She is currently pursuing the M.S. degree in operational research and cybernetics with the China University of Mining and Technology, Xuzhou, China. She current research interests include neural networks and stochastic control.
Song Zhu was born in 1982. He received the B.S. degree in mathematics from Jiangsu Normal University, Xuzhou, China, in 2004, and the M.S. degree in probability and mathematical statistics and the Ph.D. degree in system engineering from the Huazhong University of Science and Technology, Wuhan, China, in 2007 and 2010, respectively. He is currently a Professor with the School of Mathematics, China University of Mining and Technology, Xuzhou. He has published over 60 international journal papers. His current research interests include neural networks and stochastic control.
Nannan Lu received the B.E. degree in electronic science and technology from PLA Information Engineering University, China, in 2006, the M.E. degree in communication and information system from China University of Mining and Technology (CUMT) in 2009, and the Ph. D. degree from Waseda University in 2013. Currently, she is a Lecturer with the School of Information and Control Engineering, CUMT. Her research interests include transfer learning, deep learning and sensor data fusion.
Shiping Wen received the M.Eng. degree in control science and engineering from the School of Automation, Wuhan University of Technology, Wuhan, China, in 2010, and the Ph.D. degree in control science and engineering from the School of Automation, Huazhong University of Science and Technology, Wuhan, in 2013. He is currently a Professor with the Australian AI Institute, Faculty of Engineering and Information Technology, University of Technology Sydney, Ultimo, NSW, Australia. His current research interests include memristor-based circuits and systems, neural networks, and deep learning. Dr. Wen serves as an Associate Editor for the IEEE Access and Neural Processing Letters.