Abstract
There are several drawbacks of multilayer neural networks (MLNNs) including the difficulty of determining the number of hidden nodes and their black box nature. We propose a new dynamic construction mechanism for MLNNs to overcome such inherent drawbacks. The main goal of our work is to train a hidden neuron and assemble it to the network dynamically while making the learning error smaller and smaller. In this paper, a hidden neuron carries out the function of a linear classifier which answers yes(Y) or no(N) to whether the input data belongs to the specific class. We call such a linear classifier a Y/N classifier and call the hidden neuron a Y/N neuron. The number of Y/N neurons are determined self-adaptively according to the given learning error and then successfully avoid the overlearning problem. The dynamically constructed MLNN with Y/N neurons is called a Y/N neural network. We prove that a Y/N neural network can always converge to the required solution and illustrate that Y/N neural networks can be applied to very complex classification problems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Networks 2, 359–366 (1989)
Mirchandani, G., Cao, W.: On hidden nodes for neural nets. IEEE Transactions on Circuits and Systems 36, 661–664 (1989)
Geman, S., Bienenstock, E., Doursat, R.: Neural Networks and the Bias/Variance Dilemma. Neural Computation 4, 1–58 (1992)
Lawrence, S., Giles, C.L., Tsoi, A.C.: What Size Neural Network Gives Optimal Generalization? In: Convergence Properties of Backpropagation. Institute for Advanced Computer Studies, University of Maryland, College Park, MD 20742 (1996)
Bartlett, P.L.: The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. IEEE Transaction on Information Theory 44, 525–536 (1998)
Ash, T.: Dynamic Node Creation in Backpropagation Networks. Connection Science 1, 365–375 (1989)
Elisseeff, A., PaugamMoisy, H.: Size of multilayer networks for exact learning: analytic approach. In: Michael, I., Jordan, M.C.M., Petsche, T. (eds.) Advances in Neural Information Processing Systems, vol. 9, pp. 162–168. The MIT Press, Cambridge (1997)
Fletcher, L., Katkovnik, S.F., Engelbrecht, A.: Optimizing the number of hidden nodes of a feedforward artificial neural network. In: IEEE International Joint Conference on Neural Networks, vol. 2, pp. 1608–1612 (1998)
Sarle, W.S.: Stopped training and other remedies for overfitting. In: 27th Symposium on the Interface, pp. 352–360 (1995)
Xu, L.: Bayesian Ying-Yang System and Theory as A Unified Statistical Learning Approach (III) Models and Algorithms for Dependence Reduction, Data Dimension Reduction, ICA and Supervised Learning. In: Wong, K.M., King, I., Yeung, D.-Y. (eds.) Theoretical Aspects of Neural Computation: A Multidisciplinary Perspective, pp. 43–60. Springer, Heidelberg (1997)
Xu, S., Chen, L.: A novel approach for determining the optimal number of hidden layer neurons for FNN’s and its application in data mining. In: 5th International Conference on Information Technology and Applications, pp. 683–686 (2008)
Vapnik, V.: Support-vector networks. Machine Learning 20, 273–297 (1995)
Platt, J.: Sequential minimal optimization: A fast algorithm for training support vector machines. Advances in Kernel MethodsSupport Vector Learning 208, 1–21 (1998)
Valiant, L.G.: A theory of the learnable. Communications of the ACM 27, 1134–1142 (1984)
Schapire, R.E.: The Strength of Weak Learnability. In: 30th Annual Symposium on Foundations of Computer Science, pp. 28–33 (1989)
Schapire, R.E., Freund, Y., Bartlett, P., Lee, W.S.: Boosting the margin: a new explanation for the effectiveness of voting methods. The Annals of Statistics 26, 1651–1686 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liu, J., Jia, Y. (2011). Dynamic Construction of Multilayer Neural Networks for Classification. In: Liu, D., Zhang, H., Polycarpou, M., Alippi, C., He, H. (eds) Advances in Neural Networks – ISNN 2011. ISNN 2011. Lecture Notes in Computer Science, vol 6675. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21105-8_59
Download citation
DOI: https://doi.org/10.1007/978-3-642-21105-8_59
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21104-1
Online ISBN: 978-3-642-21105-8
eBook Packages: Computer ScienceComputer Science (R0)