Abstract
In this paper, we present a preliminary experimental study of the generalization abilities of feedforward neural networks with median neuron input function (MIF). In these networks, proposed in our previous work, the signals fed to a neuron are not summed but a median of input signals is calculated. The MIF networks were designed to be fault tolerant but we expect them to have also improved generalization ability. Results of first experimental simulations are presented and described in this article. Potentially improved performance of the MIF networks is demonstrated.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Amari, S., Murata, N., Muller, K.-R., Finke, M., Yang, H.H.: Asymptotic statistical theory of overtraining and cross-validation. IEEE Transactions on Neural Networks 8(5), 985–996 (1997)
Bartlett, P.: For valid generalization the size of the weights is more important than the size of the network. In: Advances in Neural Information Processing Systems, vol. 9, p. 134. The MIT Press (1997)
Baum, E.B., Haussler, D.: What size net gives valid generalization? Neural Computation 1, 151–160 (1989)
Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press (1995)
Caruana, R., Lawrence, S., Lee Giles, C.: Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping. In: Proc. Neural Information Processing Systems Conference, pp. 402–408 (2000)
Hagan, M.T., Demuth, H.B., Beale, M.H.: Neural Network Design. PWS Publishing, Boston (1996)
Hagan, M.T., Menhaj, M.B.: Training Feedforward Networks with the Marquardt Algorithm. IEEE Trans. on Neural Networks 5(6), 989–993 (1994)
Liano, K.: Robust error measure for supervised neural network learning with outliers. IEEE Transactions on Neural Networks 7, 246–250 (1996)
MacKay, D.: A Practical Bayesian Framework for Backprop Networks. Neural Computation 4(3), 448–472 (1992)
Mackey, M.C., Glass, L.: Oscillations and chaos in physiological control systems. Science 197, 287–289 (1977)
Neal, R.M.: Bayesian Learning for Neural Networks. Springer, New York (1996)
Prechelt, L.: Early stopping - But when? In: Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 1524, pp. 55–69. Springer, Heidelberg (1998)
Riedmiller, M., Braun, H.: A direct adaptive method for faster backpropagation learning: The RPROP algorithm. In: Proceedings of the IEEE International Conference on Neural Networks (ICNN), San Francisco, pp. 586–591 (1993)
Rusiecki, A.: Robust MCD-based backpropagation learning algorithm. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2008. LNCS (LNAI), vol. 5097, pp. 154–163. Springer, Heidelberg (2008)
Rusiecki, A.: Fast Robust Learning Algorithm Dedicated to LMLS Criterion. In: Rutkowski, L., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2010, Part II. LNCS (LNAI), vol. 6114, pp. 96–103. Springer, Heidelberg (2010)
Rusiecki, A.L.: Fault tolerant feedforward neural network with median neuron input function. Electronics Letters 41(10), 603–605 (2005)
Wang, W., Van Gelder, P.H.A.J.M., Vrijling, J.K.: Some issues about the generalization of neural networks for time series prediction. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005, Part II. LNCS, vol. 3697, pp. 559–564. Springer, Heidelberg (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Rusiecki, A. (2013). Testing the Generalization of Feedforward Neural Networks with Median Neuron Input Function. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing. ICAISC 2013. Lecture Notes in Computer Science(), vol 7894. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38658-9_16
Download citation
DOI: https://doi.org/10.1007/978-3-642-38658-9_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-38657-2
Online ISBN: 978-3-642-38658-9
eBook Packages: Computer ScienceComputer Science (R0)