Abstract
This paper presents a different view on the issue of local minima and introduces a new search method for Backpropagation learning algorithm of Multi-Layer Perceptrons (MLP). As in conventional point of view, Back-propagation may be trapped at local minima instead of finding the global minimum. This concept often leads to less confidence that people may have on neural networks. However, one could argue that most of local minima may be caused by the limitation of search methods. Therefore a new search method to address this situation is proposed in this paper. This new method, “retreat and turn”, has been applied to several different types of data alone or combined with other techniques. The encouraging results are included in this paper.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Minsky, M., Papert, S.: Epilog: the new connectionism. In: Perceptrons, 3rd edn., pp. 247–280. MIT Press, Cambridge (1988)
Frasconi, P., Gori, M., Tesi, A.: Success and Failures of Backpropagation: A Theoretical Investigation. In: Omidvar, O. (ed.) Progress in Neural Networks, vol. 5. Ablex Publishing, Greenwich (1993)
Blum, E.K.: Approximation of Boolean Functions by Sigmoidal Networks: Part I: XOR and Other Two-Variable Functions. Neural Computation 1, 532–540 (1989)
Hush, D.R., Horne, B., Salas, J.M.: Error Surfaces for Multilayer Perceptrons. IEEE Transactions on Systems, Man, and Cybernetics 22, 1152–1161 (1992)
Kordos, M., Duch, W.: On Some Factors Influencing MLP Error Surface. In: 7th International Conference of Artificial Intelligence and Soft Computing, pp. 217–222 (2004)
Hamey, L.G.: The Structure of Neural Network Error Surface. In: 6th Australian Conference on Neural Networks, pp. 197–200 (1995)
Sprinkhuizen-Kuyper, I.G., Boers, E.J.: A Comment on Paper of Blum: Blum’s “local minima” are Saddle Points. Technical Report 94-34, Department of Computer Science, Leiden University (1994)
Chen, H.H., Manry, M.T.: Improving Healthcare Predictive Modeling using NeuroSequences. In: 16th Federal Forecasters Conference (in press, 2008)
Abdurrab, A.A., Manry, M.T., Li, J., Malalur, S.S., Gore, R.G.: A Piecewise Linear Network Classifier. In: 20th International Joint Conference on Neural Networks, pp. 1750–1755 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chen, HH. (2008). The Turning Points on MLP’s Error Surface. In: Sun, F., Zhang, J., Tan, Y., Cao, J., Yu, W. (eds) Advances in Neural Networks - ISNN 2008. ISNN 2008. Lecture Notes in Computer Science, vol 5263. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87732-5_57
Download citation
DOI: https://doi.org/10.1007/978-3-540-87732-5_57
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87731-8
Online ISBN: 978-3-540-87732-5
eBook Packages: Computer ScienceComputer Science (R0)