Abstract
This paper proposes an approach for entropy optimization by neural networks. A brief introduction to this problem is given. A simple neural algorithm based upon MSE minimization is provided. Validation of this algorithm is given by an application to the Source Separation problem.
Preview
Unable to display preview. Download preview PDF.
References
J-F. Cardoso and B. Laheld. Equivariant adaptive source separation. IEEE Trans. on S.P., 44(12):3017–3030, December 1996.
N. Charkani and Y. Deville. Optimization of the asymptotic performance of time-domain convolutive source separation algorithms. In ESANN'97, pages 273–278, Bruges, Belgium, April 1997.
P. Comon. Independent component analysis, a new concept? Signal Processing, 36(3):287–314, April 1994.
C. Fyfe and R. Baddeley. Non-linear data structure extraction using simple hebbian networks. Biological Cybernetics, (72):533–541, 1995.
P.J. Huber. Projection pursuit. The Annals of Statistics, 13(2):435–475, 1985.
C. Jutten and J. Hérault. Blind separation of sources, Part I: An adaptive algorithm based on a neuromimetic architecture. Signal Processing, 24(1):1–10, 1991.
J. Karhunen, E. Oja, L. Wang, R. Vigário, and J. Joutsensalo. A class of neural networks for independant component analysis. IEEE trans. N.N., 8(3):486–504, May 1997.
R. Linsker. Self-organization in a perceptual network. Computer, (21):105–117, 1988.
D. T. Pham, P. Garat, and C. Jutten. Separation of a mixture of independent sources through a maximum likelihood approach. In J. Vandewalle, R. Boite, M. Moonen, and A. Oosterlinck, editors, Signal Processing VI, Theories and Applications, pages 771–774, Brussels, Belgium, August 1992. Elsevier.
N. N. Schraudolph. Optimization of entropy with neural networks. PhD thesis, University of California, San Diego, 1995.
H.H. Yang and S.I. Amari. Adaptive on-line learning algorithms for blind separation-maximum entropy and minimum mutual information. Neural Computation, 1997. Accepted.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1997 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Taleb, A., Jutten, C. (1997). Entropy optimization. In: Gerstner, W., Germond, A., Hasler, M., Nicoud, JD. (eds) Artificial Neural Networks — ICANN'97. ICANN 1997. Lecture Notes in Computer Science, vol 1327. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0020208
Download citation
DOI: https://doi.org/10.1007/BFb0020208
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-63631-1
Online ISBN: 978-3-540-69620-9
eBook Packages: Springer Book Archive