Abstract
In a function approximation task using neural network, there exist many cases in which the distributions of data are so complex that the regression with single network does not perform the given task well. Employing multiple modules that performs regression in different regions respectively may be a reasonable solution for this case. This paper proposes a new adaptive modular architecture for regression model, and simulates its performance on difficult problems that are not easily approximated using traditional neural network learning algorithms. Regression modules are added as learning proceeds, depending on the local distribution of data. The use of a local distribution that captures the underlying local structure of the data offers the advantage of adaptive regression.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Jacobs, R.A., Jordan, M.I., Barto, A.G.: Task decomposition through competition in a modular connectionist architecture: The what and where vision task. Cognitive Science 15, 219–250 (1991)
Fahlman, S.E., Lebiere, C.: The cascade-correlation learning architecture. Advances in Neural Information Processing Systems II (Denver 1989), 524–532 (1990)
Le Cun, Y., Denker, J.S., Solla, S.A.: Optimal Brain Damage. Advances in Neural Information Processing system 2, 598–605 (1990)
Hassibi, B., Stork, D.G.: econd Order Derivatives for Network Pruning: Optimal Brain Surgeon. Advances in Neural Information Processing Systems 5, 164–171 (1993)
Carpenter, G., Grossberg, S.: The art of adaptive pattern recognition by a self-organizing neural network. Computer, 77–88 (March 1988)
~Frean, M.: The Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks. Neural Computation 2, 198–209 (1990)
~Sirat, J.-A., ~Nadal, J.-P.: Neural Trees: A New Tool for Classification, Preprint, Laboratoires d’Electronique Philips, Limeil-Brevannes, France" (1990)
Lee, H., Mehrotra, K., Mohan, C.K., Ranka, S.: An Incremental Network Construction Algorithm for Approximating Discontinuous Functions. In: Proc. Int’l. Conf. Neural Networks, vol. IV, pp. 2191–2196 (1994)
Plaut, D.C., Hinton, G.E.: Learning sets of filters using back-propagation. Computer Speech and Language 2, 35–61 (1987)
Sutton, R.S.: Two problems with back-propagation and other steepest-descent learning procedures for networks. In: Proc. Eighth Annual Conference of the Cognitive Science Society, pp. 823–832 (1986)
Sekiguchi, M., Nagata, S., Asakawa, K.: Behavior Control for a Mobile Robot by Multi-Hierarchcal Neural Network. In: Proc. IEEE Int’l. Conf. Robotics and Automation, pp. 1578–1583 (1989)
Pratt, L.Y., Kamm, C.A.L.: Improving Phoneme Classification Neural Network through Problem Decomposition. In: IEEE Int’l. Joint Conf. Neural, vol. 2, pp. 821–826 (1991)
Smieja, F.J.: Multiple network systems Minos modules: Task division and module discrimination. In: Proceedings of the 8th AISB conference on Artificial Intelligence (1991)
Reilly, D.L., Scofield, C., Elbaum, C., Cooper, L.N.: Learning System Architectures Composed of Multiple Learning Modules. In: IEEE Int’l. Conf. Neural Networks, vol. 2, pp. 495–503 (1989)
Jacobs, R.A., Jordan, M.I.: Learning piecewise control strategies in a modular neural network architecture. IEEE Trans. Systems, Man and Cybernetics 23, 337–345 (1993)
Kohonen, T.: Improved versions of Learning Vector Quantization. In: Proc. Int’l. Joint Conf. Neural Networks, San Diego (CA), vol. I, pp. 545–550 (1990)
Hecht-Nielsen, R.: Counterpropagation networks. In: IEEE International Conference on Neural Networks, vol. II, pp. 19–32 (1987b)
Clarkson, K.L.: Safe and effective determinant evaluation. In: Proc. 31st IEEE Symposium on Foundations of Computer Science, pp. 387–395 (1992)
Clarkson, K.L., Mehlhorn, K., Seidel, R.: Four results on randomized incremental constructions. In: Comp. Geom.: Theory and Applications, pp. 185–121 (1993)
Nomura, T., Miyoshi, T.: An adaptive rule extration with the fuzzy self-organizing map and a comparison with other methods. In: Proc. ISUMA-NAFIPS 1995, pp. 311–316 (1995)
Nomura, T., Miyoshi, T.: An adaptive fuzzy rule extraction using hybrid model of the fuzzy selforganizing map and the genetic algorithm with numerical chromosomes. In: Proc. Fourth International Conference on Soft Computing IIZUKA 1996, vol. I, pp. 70–73 (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kim, W., Hong, C., Jung, C. (2004). A Novel Approach to Function Approximation: Adaptive Multimodule Regression Networks. In: Lemaître, C., Reyes, C.A., González, J.A. (eds) Advances in Artificial Intelligence – IBERAMIA 2004. IBERAMIA 2004. Lecture Notes in Computer Science(), vol 3315. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30498-2_82
Download citation
DOI: https://doi.org/10.1007/978-3-540-30498-2_82
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23806-5
Online ISBN: 978-3-540-30498-2
eBook Packages: Springer Book Archive