Abstract
This work addresses the problem of improving the generalization capabilities of continuous recurrent neural networks. The learning task is transformed into an optimal control framework in which the weights and the initial network state are treated as unknown controls. A new learning algorithm based on a variational formulation of Pontryagin’s maximum principle is proposed. Numerical examples are also given which demonstrate an essential improvement of generalization capabilities after the learning process of a recurrent network.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Morozov, V.A.: Methods for solving incorrectly posed problems, Springer-Verlag, Berlin, (1984)
Geman, S., Bienenstock, E., Doursat, R.: Neural networks and the bias/variance dillema, Neural Comp., vol. 4, no. 1, (1992) 1–58
Giles, C.L., Chen, D., Sun, G.Z.: Constructive learning of recurrent neural net-works. Limitations of recurrent cascade correlation and a simple solution, IEEE Trans. Neural Networks, vol. 6, no. 4, (1995), 829–836
Pearlmutter, B.A.: Gradient calculation for dynamic recurrent neural networks: A survey, IEEE Trans. Neural Networks, vol. 6, (1995), 1212–1228
Cohen, B., Saad, D., Marom, E.: Efficient training of recurrent neural network with time delays, Neural Networks, vol. 10, no. 1, (1997), 51–59
Galicki, M., Leistritz, L., Witte, H.: Learning continuous trajectories in recurrent neural networks with time-dependent weights, IEEE Trans. Neural Networks, vol. 10, (1999), 741–756
Holden S.B., Niranjan, M.: On the practical applicability of VC dimension bounds, Neural Comp., vol. 7, no. 6, (1995), 1265–1288
Czernichow, T.: A double gradient algorithm to optimize regularization, in Proc. Artificial Neural Networks-ICANN’7, (1997), 289–294
Bishop, C.: Improving the generalization properties of radial basis function neural networks, Neural Comp., 3, (1991), 579–588
Girosi, F., Jones, F., Poggio, T.: Regularization theory and neural networks architectures, Neural Comp., 7, (1995), 219–269
Bishop, C.: Training with noise is equivalent to Tikhonov regularization, Neural Comp., 7, (1995), 108–116
Galicki, M., Leistritz, L., Witte, H.: Improved learning of multiple continuous trajectories with initial network state, in Proc. IJCNN’2000, (2000), 15–20
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Galicki, M., Leistritz, L., Witte, H. (2002). Learning the Dynamic Neural Networks with the Improvement of Generalization Capabilities. In: Dorronsoro, J.R. (eds) Artificial Neural Networks — ICANN 2002. ICANN 2002. Lecture Notes in Computer Science, vol 2415. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-46084-5_61
Download citation
DOI: https://doi.org/10.1007/3-540-46084-5_61
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-44074-1
Online ISBN: 978-3-540-46084-8
eBook Packages: Springer Book Archive