Abstract
In this paper, we propose a new computational method for a supervised competitive learning method. In the supervised competitive learningmethod, information is controlled in an intermediate layer, and in an output layer, errors between targets and outputs are minimized. In the intermediate layer, competition is realized by maximizing mutual information between input patterns and competitive units with Gaussian functions. One problem is that a process of information maximization is computationally expensive. However, we have found that the method can produce appropriate performance with a small number of epochs. Thus, we restrict here the number of epochs to only one epoch for facilitating learning. This computational method can overcome the shortcoming of our information maximization method. We applied our method to chemical data processing. Experimental results showed that with only one epoch, the new computational method gave better performance than did the conventional methods.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Kamimura, R., Kamimura, T., Uchida, O.: Flexible feature discovery and structural information. Connection Science 13(4), 323–347 (2001)
Kamimura, R., Kamimura, T., Takeuchi, H.: Greedy information acquisition algorithm: A new information theoretic approach to dynamic information acquisition in neural networks. Connection Science 14(2), 137–162 (2002)
Kamimura, R.: Progressive feature extraction by greedy network-growing algorithm. Complex Systems 14(2), 127–153 (2003)
DeSieno, D.: Adding a conscience to competitive learning. In: Proceedings of IEEE International Conference on Neural Networks, San Diego, pp. 117–124. IEEE, Los Alamitos (1988)
Ahalt, S.C., Krishnamurthy, A.K., Chen, P., Melton, D.E.: Competitive learning algorithms for vector quantization. Neural Networks 3, 277–290 (1990)
Xu, L.: Rival penalized competitive learning for clustering analysis, RBF net, and curve detection. IEEE Transaction on Neural Networks 4(4), 636–649 (1993)
Hulle, M.M.V.: The formation of topographic maps that maximize the average mutual information of the output responses to noiseless input signals. Neural Computation 9(3), 595–606 (1997)
Rumelhart, D.E., Zipser, D.: Feature discovery by competitive learning. Cognitive Science 9, 75–112
Hecht-Nielsen, R.: Counterpropagation networks. Applied Optics 26, 4979–4984 (1987)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kamimura, R. (2004). One-Epoch Learning for Supervised Information-Theoretic Competitive Learning. In: Pal, N.R., Kasabov, N., Mudi, R.K., Pal, S., Parui, S.K. (eds) Neural Information Processing. ICONIP 2004. Lecture Notes in Computer Science, vol 3316. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30499-9_80
Download citation
DOI: https://doi.org/10.1007/978-3-540-30499-9_80
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23931-4
Online ISBN: 978-3-540-30499-9
eBook Packages: Springer Book Archive