Abstract
In this paper, a global asymptotic stability condition for Long Short-Term Memory neural networks is presented. Since this condition is formulated in terms of the networks’ weight matrices and biases that are essentially control variables, the same condition can be viewed as a way to globally asymptotically stabilize these networks. The condition and how to compute numerical values for the weight matrices and biases are illustrated by a number of numerical examples.
This is a preview of subscription content, log in via an institution to check access.
References
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Greff, K., Srivastava, R.K., Koutnk, J., Steunebrink, B.R., Schmidhuber, J.: LSTM: a search space odyssey. IEEE Trans. Neural Netw. Learn. Syst. 28(10), 2222–2232 (2017)
Guckenheimer, J., Holmes, P.: Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields. Springer, New York (2002)
Hirsch, M.W., Smale, S., Devaney, R.L.: Differential Equations, Dynamical Systems, and an Introduction to Chaos, 3rd edn. Elsevier Inc., Waltham (2013)
Bertschinger, N., Natschläger, T.: Real-time computation at the edge of chaos in recurrent neural networks. Neural Comput. 16(7), 1413–1436 (2004)
Stipanović, D.M., Murmann, B., Causo, M., Lekić, A., Rubies Royo, V., Tomlin, C.J., Beigne, E., Thuries, S., Zarudniev, M., Lesecq, S.: Some local stability properties of an autonomous long short-term memory neural network model. In: 2018 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–5. IEEE, Florence (2018). http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8350958&isnumber=8350884. Accessed 29 Jan 2018
LaSalle, J.P.: The Stability and Control of Discrete Processes. Springer, New York (1986)
Kalman, R.E., Bertram, J.E.: Control system analysis and design via the second method of lyapunov: I. Continuous-time systems. J. Basic Eng. 82(2), 371–393 (1960)
Graves, A.: Generating sequences with recurrent neural networks. arXiv (2013). ArXiv preprint arXiv:1308.0850
Horn, R.A., Johnson, C.R.: Matrix Analysis. Cambridge University Press, Cambridge (1990)
Fleming, W.: Functions of Several Variables. Springer, Berlin (2012)
Siljak, D.D.: Decentralized control of complex systems. Academic Press Inc, New York (1991)
Lakshmikantham, V., Trigiante, D.: Theory of Difference Equations: Numerical Methods and Applications, vol. 251. Marcel Dekker, New York (2002)
Laurent, T., von Brecht, J.: A recurrent neural network without chaos. arXiv preprint arXiv:1612.06212 (2016)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Deka, S.A., Stipanović, D.M., Murmann, B. et al. Global Asymptotic Stability and Stabilization of Long Short-Term Memory Neural Networks with Constant Weights and Biases. J Optim Theory Appl 181, 231–243 (2019). https://doi.org/10.1007/s10957-018-1447-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-018-1447-6