Abstract
Abstract. In support vector machine there exist four attractive techniques: kernel idea to construct nonlinear algorithm using Mercer kernels, large margin or regularization to control generalization ability, convex objective functional to obtain unique solution, and support vectors or sparseness to reduce computation time. The kernel neuron is the nonlinear version of McCulloch-Pitts neuron based on kernels. In this paper we define a regularized risk functional including empirical risk functional and Gaussian regularizer for kernel neuron. On the basis of gradient descent method, single sample correction and momentum term, the corresponding learning algorithm is designed, which can realize four ideas in support vector machine with a simple iterative scheme and can handle the classification and regression problems effectively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Vapnik, V.N.: Statistical Learning Theory. Wiley, New York (1998)
Vapnik, V.N.: The Nature of Statistical Learning Theory, 2nd edn. Springer, New York (1999)
Scholkopf, B., Smola, A.J.: Learning with Kernels - Support Vector Machines, Regularization, Optimization and Beyond. MIT Press, Cambridge (2002)
Scholkopf, B., Smola, A., Muller, K.-R.: Nonlinear Component Analysis as a Kernel Eignvalue Problem. Neural Computation 10(5), 1299–1319 (1998)
Mika, S., Ratsch, G., Weston, J., Scholkopf, B., Muller, K.-R.: Fisher Discriminant Analysis with Kernels. Neural Networks for Signal Processing IX, 41–48 (1999)
Suykens, J.A.K., Vandewalle, J.: Least Squares Support Vector Machines. Neural Processing Letters 9, 293–300 (1999)
Xu, J., Zhang, X., Li, Y.: Large Margin Kernel Pocket Algorithm. In: Proceedings of 2001 International Joint Conference on Neural Networks, pp. 1480–1485. IEEE Press, New York (2001)
McCulloch, W.S., Pitts, W.: A Logical Calculus of the Ideas Immanent in Nervous Activity. Bulletin of Mathematical Biophysics 5, 115–131 (1943)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning Representations by Back- Propagating Errors. Nature 323(9), 533–536 (1986)
Theodoridis, S., Koutroumbas, K.: Pattern Recognition. Academic Press, San Diego (1999)
Xu, J., Zhang, X., Li, Y.: Kernel Neuron and its Training Algorithm. In: Proceedings of 8th International Conference on Neural Information Processing, vol. 2, pp. 861–866. Fudan University Press, Shanghai China (2001)
Tikhonov, A.N., Arsenin, V.Y.: Solution of Ill-Posed Problem. W. H. Wanston, Washington (1977)
Pault, D.C., Nowlan, S.J., Hinton, G.E.: Experiments on Learning by Back Propagation. Technical Report. CMU-CS-86-126, Carnegie-Mallon University (1986)
Saito, K., Nakano, R.: Second Order Learning Algorithm with Squared Penalty Term. Neural Computation 12(3), 709–729 (2000)
Evgeniou, T., Pontil, M., Poggio, T.: Regularization Networks and Support Vector Machines. In: Smola, A.J., Bartlett, P.L., Schölkopf, B., Schuurmans, D. (eds.) Advances in Large Margin Classifiers, pp. 171–204. MIT Press, Cambridge (2000)
Girosi, F., Jones, M., Poggio, T.: Regularization Theory and Neural Networks Architectures. Neural Computation 7(2), 219–269 (1995)
Reed, R.: Pruning Algorithms - A Survey. IEEE Transactions on Neural Networks 4(5), 740–747 (1993)
Ishikawa, M.: Structural Learning with Forgetting. Neural Networks 9(3), 509–521 (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Xu, J., Zhang, X. (2004). A Learning Algorithm with Gaussian Regularizer for Kernel Neuron. In: Yin, FL., Wang, J., Guo, C. (eds) Advances in Neural Networks – ISNN 2004. ISNN 2004. Lecture Notes in Computer Science, vol 3173. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-28647-9_43
Download citation
DOI: https://doi.org/10.1007/978-3-540-28647-9_43
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22841-7
Online ISBN: 978-3-540-28647-9
eBook Packages: Springer Book Archive