Abstract
This paper deals with higher order feed-forward neural networks with a new activation function - neuron-adaptive activation function. Experiments with function approximation and stock market movement simulation have been conducted to justify the new activation function. Experimental results have revealed that higher order feed-forward neural networks with the new neuron-adaptive activation function present several advantages over traditional neuron-fixed higher order feed-forward networks such as much reduced network size, faster learning, and more accurate financial data simulation.
This work is supported by an IRGS (Institutional Research Grants Scheme) at University of Tasmania, Tasmania, Australia.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Lee, Y.C., Doolen, G., Chen, H., Sun, G., Maxwell, T., Lee, H., Giles, C.L. (1986) Machine learning using a higher order correlation network, Physica D: Nonlinear Phenomena, 22, 276–306.
Lippman, R.P. (1989) Pattern classification using neural networks, IEEE Commun. Mag., 27, 47–64.
Psaltis, D., Park, C.H., Hong, J. (1988) Higher order associative memories and their optical implementations, Neural Networks, 1, 149–163.
Reid, M.B., Spirkovska, L., Ochoa, E. (1989). Simultaneous position, scale, rotation invariant pattern classification using third-order neural networks, Int. J. Neural Networks, 1, 154–159.
Wood, J., Shawe-Taylor, J. (1996). A unifying framework for invariant pattern recognition, Pattern Recognition Letters, 17, 1415–1422.
Giles, C.L., Maxwell, T. (1987) Learning, invariance, and generalization in higher order neural networks, Applied Optics, 26(23), 4972–4978.
Redding, N.J., Kowalczyk, A., Downs, T. (1993). Constructive higher-order network algorithm that is polynomial time, Neural Networks, 6, 997–1010.
Kosmatopoulos, E.B., Polycarpou, M.M., Christodoulou, M.A., Ioannou, P.A. (1995). High-order neural network structures for identification of dynamical systems, IEEE Transactions on Neural Networks, 6(2), 422–431.
Thimm, G., Fiesler, E. (1997). High-order and multilayer perceptron initialization, IEEE Transactions on Neural Networks, 8(2), 349–359.
Vecci, L., Piazza, F., Uncini, A. (1998). Learning and approximation capabilities of adaptive spline activation function neural networks, Neural Networks, 11, 259–270.
Chen, C.T., Chang, W.D. (1996). A feedforward neural network with function shape autotuning, Neural Networks, 9(4), 627–641
Campolucci, P., Capparelli, F., Guarnieri, S., Piazza, F., & Uncini, A. (1996). Neural networks with adaptive spline activation function. Proceedings of IEEE MELECON 96, Bari, Italy, 1442–1445.
Hu, Z., Shao, H. (1992). The study of neural network adaptive control systems, Control and Decision, 7, 361–366.
Yamada, T., Yabuta, T. (1992). Remarks on a neural network controller which uses an auto-tuning method for nonlinear functions, IJCNN, 2, 775–780
Rumelhart, D.E., McClelland, J.L. (1986). Parallel distributed computing: exploration in the microstructure of cognition, Cambridge, MA: MIT Press.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Xu, S., Zhang, M. (2002). An Adaptive Activation Function for Higher Order Neural Networks. In: McKay, B., Slaney, J. (eds) AI 2002: Advances in Artificial Intelligence. AI 2002. Lecture Notes in Computer Science(), vol 2557. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36187-1_31
Download citation
DOI: https://doi.org/10.1007/3-540-36187-1_31
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-00197-3
Online ISBN: 978-3-540-36187-9
eBook Packages: Springer Book Archive