Abstract
The α-divergence is utilized to derive a generalized expectation and maximization algorithm (EM algorithm). This algorithm has a wide range of applications. In this paper, neural network learning for mixture probabilities is focused. The α-EM algorithm includes the existing EM algorithm as a special case since that corresponds to α = −1. The parameter α specifies a probability weight for the learning. This number affects learning speed and local optimality. In the discussions of update equations of neural nets, extensions of basic statistics such as Fisher's efficient score, his measure of information and Cramér-Rao's inequality are also given. Besides, this paper unveils another new idea. It is found that the cyclic EM structure can be used as a building block to generate a learning systolic array. Attaching monitors to this systolic array makes it possible to create a functionally distributed learning system.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm, J. R. Stat. Soc. Sr. B. 39 (1978) 1–38
Jordan, M.J., Jacobs, R.A.: Hierarchical mixtures of experts and the EM algorithm, Neural Computation 6 (1994) 181–214
Jordan, M.J., Xu, L.: Convergence results for the EM approach to mixtures of experts architecture, Neural Networks 8 (1995) 1409–1431
Amari, S.: Information geometry of the EM and em algorithms for neural nets, Neural Networks 8 (1995) 1379–1408
Amari, S.: Information, geometry of neural networks — New Bayesian duality theory-, Proc. ICONIP'96 1 (1996) 3–6
Xu, L.: Bayesian-Kullback Ying-Yang Machine: Reviews and new results, Proc. ICONIP'96, 1 (1996) 59–67
Rényi, A.: On measures of entropy and information, Proc. 4th Berkeley Symp. Math. Stat. and Pr., 1 (1960) 547–561
Havrda, J.H., Chavat, F.: Qualification methods of classification processes: Concepts of structural α. entropy, Kybernetica, 3 (1967) 30–35
Amari, S., Nagaoka, H.: Methods of Information Geometry (in Japanese), Iwanami Publishing Co. (1993)
Csiszár, I: A class of measure of informativity of observation channels, Periodica Mathematica Hungarica 2 (1972) 191–213
Matsuyama, Y.: The α-EM algorithm and its block connections, Tech. Report, CML-96-1 (1996) Computation Mechanism Lab., Dept. of EECE, Waseda Univ.
Matsuyama, Y.: The weighted EM learning and monitoring structure, Info. Proc. Soc. of Japan, 54th Convention Record (1997) 6G-04
Matsuyama, Y.: The Weighted EM algorithm and Block Monitoring, Proc. ICNN, (1997)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1997 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Matsuyama, Y. (1997). The α-EM algorithm: A block connectable generalized leaning tool for neural networks. In: Mira, J., Moreno-Díaz, R., Cabestany, J. (eds) Biological and Artificial Computation: From Neuroscience to Technology. IWANN 1997. Lecture Notes in Computer Science, vol 1240. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0032507
Download citation
DOI: https://doi.org/10.1007/BFb0032507
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-63047-0
Online ISBN: 978-3-540-69074-0
eBook Packages: Springer Book Archive