Abstract
Determining the optimum amount of regularisation to obtain the best generalisation performance in feedforward neural networks is a difficult problem. This problem is addressed in the casper algorithm, a constructive cascade algorithm that uses regularisation. Previously the amount of regularisation used by this algorithm was set by a parameter prior to training. This work explores the use of an adaptive method to automatically set the amount of regularisation as the network is constructed. This technique is compared against the original method of user optimised regularisation settings and is shown to maintain, and some-times improve the generalisation results, while also constructing smaller networks. Further benchmarking on the Proben1 series of data sets is performed and the results compared to an optimised Cascade Correlation algorithm.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
N. K. Treadgold and T. D. Gedeon, “A cascade network algorithm employing progressive rprop,” in Proc. of the Int. Work-Conf. on Artificial and Natural Neural Systems, Lanzarote, Spain, June 1997, pp. 723–732.
N. K. Treadgold and T. D. Gedeon, “Extending casper: A regression survey,” in Proc. of the Int. Conf. on Neural Information Processing, Dunedin, New Zealand, Nov. 1997, pp. 310–313.
S. E. Fahlman and C. Lebiere, “The Cascade-Correlation learning architecture”, in Advances in Neural Information Processing Systems 2, D. S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1990, pp. 524–532.
L. Prechelt, “Proben1—a set of neural network benchmark problems and benchmarking rules,” Tech. Rep. 21/94, Fakultät für Informatik, Universität Kahlsruhe, 1994.
M. Riedmiller and H. Braun, “A direct adaptive method for faster backpropagation learning: The RPROP algorithm,” in Proc. of the IEEE Int. Conf. on Neural Networks, San Francisco, CA, Apr. 1993, pp. 586–591.
N. K. Treadgold and T. D. Gedeon, “Simulated annealing and weight decay in adaptive learning: The sarprop algorithm”, IEEE Transactions on Neural Networks, vol. 9, pp. 662–668, July 1998.
J. S. Bridle, “Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition”, in Neuro-computing: Algorithms, Architectures and Applications, F. Fogelman Soulié and J. Hérault, Eds. Berlin: Springer-Verlag, 1990, pp. 227–236.
C. Bishop, Neural Networks for Pattern Recognition. Oxford: Oxford University Press, 1995.
J.-N. Hwang, S.-R. Lay, M. Maechler, R. D. Martin, and J. Schimert, “Regression modeling in back-propagation and projection pursuit learning”, IEEE Transactions on Neural Networks, vol. 5, pp. 342–353, May 1994.
R. Steel and J. Torrie, Principles and Procedures of Statistics A Biomedical Approach. Singapore: McGraw-Hill, 1980.
L. Prechelt, “Investigation of the cascor family of learning algorithms”, Neural Networks, vol. 10, no. 5, pp. 885–896, 1997.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1999 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Treadgold, N.K., Gedeon, T.D. (1999). A constructive cascade network with adaptive regularisation. In: Mira, J., Sánchez-Andrés, J.V. (eds) Engineering Applications of Bio-Inspired Artificial Neural Networks. IWANN 1999. Lecture Notes in Computer Science, vol 1607. Springer, Berlin, Heidelberg . https://doi.org/10.1007/BFb0100470
Download citation
DOI: https://doi.org/10.1007/BFb0100470
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-66068-2
Online ISBN: 978-3-540-48772-2
eBook Packages: Springer Book Archive