Abstract
Learning neural algorithms are generally very simple, but the convergence is not very fast and robust. In this paper we address the important problem of optimum learning rate adjustement according to an adaptive procedure based on gradient method. The basic idea, very simple, which has already been successfully used in Signal Processing, is extended to 2 neural algorithms : Kohonen self-organizing maps and blind separation of sources (Hérault-Jutten algorithm). Although this procedure increases the algorithms complexity, it remains very interesting :
-
-the convergence speed is strongly boosted,
-
-the local nature of learning rule is retained,
-
-the method is applicable to some rule, even if we do not know the cost function (error) which is minimized.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
M. L. Honig, D. G. Messerschmitt, Adaptive Filters: Structures, Algorithms and Applications,Kluwer Academic Publishers, 1985 (pp. 40–47).
T. Kohonen, Self Organization and Associative Memory, Springer Verlag, Berlin, 1984.
I. D. Landau, Identification et commande des systèmes, Ed. Hermès, Paris, 1988.
Ph. Bragard, G. Jourdain, J. Martin, "Optimal adaptive algorithms behaviour used in unterwater communication signals equalization", Proceeding of EUSIPCO 88, Grenoble (France), Sept. 1988, Signal Processing IV: Theories and Applications, J.-L. Lacoume, A. Chehikian, N. Martin, and J. Malbos (Eds.), Elsevier Science Publishers B. V. (North-Holland), pp. 363–366.
J. Hérault, C. Jutten, B. Ans, "Détection de grandeurs primitives dans un message composite par une architecture de calcul neuromimétique en apprentissage non supervisé". Actes du Xème colloque GRETSI, Nice (France) 1985, Vol. 2, pp.1017–22.
C. Jutten, J. Hérault, "Une solution neuromimétique au problème de séparation de sources". Traitement du Signal, vol. 5, no 6 (1988), pp. 389–403.
C. Jutten, J. Hérault, "Blind separation of sources. Part I: an Adaptive Algorithm based on a Neuromimetic Architecture". Signal Processing. To appear in Vol. 24 (1991).
J.-C. Fort, Solving a combinatorial problem via self-organizing process: an application of the Kohonen algorithm to the Travelling Salesman Problem. Biol. Cybern., 59 (1988), pp. 33–40.
J. C. Houk, W. Z. Rymer, P. E. Crago, "Nature of the dynamic response and its relation to the high sensitivity of muscles spindles to small changes in length". In Muscles Receptors and Movement (H. Taylor and A. Prochazka, Eds.) MacMillan, London, 1981, pp. 33–43.
J.-P. Roll, Contribution de la proprioception musculaire à la perception et au contrôle du mouvement chez l'homme, Thèse d'Etat, Univ. d'Aix-Marseille I, 1981.
P. Duvaut, "Principes des méthodes de séparation fondées sur les moments d'ordre supérieur". Traitement du Signal, Vol. 7, No 5, 1990, pp. 407–418.
C. Jutten, H.L. Nguyen Thi, E. Djikstra, E. Vittoz, J. Caelen, "Blind Separation of Sources: an Algorithm for Separation of Convolutive Mixtures", Int. Workshop on High Order Statistics, Chamrousse (France), July 1991.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1991 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jutten, C., Guerin, A., Nguyen Thi, H.L. (1991). Adaptive optimization of neural algorithms. In: Prieto, A. (eds) Artificial Neural Networks. IWANN 1991. Lecture Notes in Computer Science, vol 540. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0035877
Download citation
DOI: https://doi.org/10.1007/BFb0035877
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-54537-8
Online ISBN: 978-3-540-38460-1
eBook Packages: Springer Book Archive