Abstract
This paper analyzed convergence properties of an online learning method when teacher’s signal includes noise in the thermodynamic limit. The learning curve was analytically derived using a statistical mechanical method and its validity was confirmed by computer simulations. In this case, the learning curve shows an overshoot phenomenon. In order to elucidate why and how it occurs in this case, the asymptotic analysis of dynamical systems was applied to the differential equations that expresses the dynamics of the learning curve and showed that the phenomenon results from the properties of the system matrix of the equations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Nishimori, H.: Statistical Physics of Spin Glasses and Information Processing: An Introduction. Oxford Univ. Press, Oxford (2001)
Tanaka, T.: A Statistical-Mechanics Approach to Large-System Analysis of CDMA Multiuser Detectors. IEEE Trans. Information Theory 48(11), 2888–2910 (2002)
Rosenblatt, F.: Principle of Neurodynamics. Spartan, Washington, D.C. (1961)
Biehl, M., Schwarze, H.: Online Learning of a Time-Dependent Rule. Europhysics Letters 2, 733–738 (1992)
Hanzawa, H.: An Asymptotic Analysis of Perceptron Learning with Noisy Teacher. Bachelor’s thesis, Kansai University (2009)
Inoue, J., Nishimori, H.: On-line AdaTron Learning of Unlearnable Rules. Physical Review E 55(4), 4544–4551 (1997)
Hara, K., Okada, M.: On-line Learning through Simple Perceptron Learning with a Margin. Neural Networks 17, 215–223 (2004)
Miyoshi, S., Hara, K., Okada, M.: Analysis of Ensemble Learning Using Simple Perceptrons Based on Online Learning Theory. Physical Review E 71, 036116 (2005)
Miyoshi, S., Okada, M.: Analysis of On-line Learning When a Moving Teacher Goes around a True Teacher. J. Physical Society of Japan 75(2), 024003 (2006)
Miyoshi, S., Okada, M.: Statistical Mechanics of Online Learning for Ensemble Teachers. J. Physical Society of Japan 75(4), 044002 (2006)
Uezu, T., Miyoshi, S., Izuo, M., Okada, M.: Theory of Time Domain Ensemble On-line Learning of Perceptron under the Existence of External Noise. J. Physical Society of Japan 76(11), 114006 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ikeda, K., Hanzawa, H., Miyoshi, S. (2013). Convergence Properties of Perceptron Learning with Noisy Teacher. In: Yang, J., Fang, F., Sun, C. (eds) Intelligent Science and Intelligent Data Engineering. IScIDE 2012. Lecture Notes in Computer Science, vol 7751. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-36669-7_51
Download citation
DOI: https://doi.org/10.1007/978-3-642-36669-7_51
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-36668-0
Online ISBN: 978-3-642-36669-7
eBook Packages: Computer ScienceComputer Science (R0)