Abstract
In the kernel approach, any N vectorial or non-vectorial data can be converted to N vectors with feature dimension N. The promise of the kernel approach hinges upon its representation vector space, leading to a “cornerized” data structure. Furthermore, the nonsingular kernel matrix basically assures a theoretically linear separability, critical to supervised learning. The main results are two folds: In terms of unsupervised clustering, the kernel approach allows dimension reduction in the spectral space and, moreover, a simple error analysis for the fast kernel K-means. As to supervised classification, by imposing uncorrelated perturbation to the training vector in the spectral space, a perturbed (Fisher) discriminant analysis (PDA) is proposed. This ultimately leads to a hybrid classier which includes PDA and SVM as specials cases, thus offering more flexibility for improving the prediction performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bach, F., Jordan, M.: Learning spectral clustering. In: Proc. of NIPS-16. MIT Press, Cambridge (2004)
Courant, R., Hilbert, D.: Methods of Mathematical Physics, volume I and II. Wiley Interscience, New York (1970)
Duda, R.O., Hart, P.E.: Pattern Classification. John Wiley, Chichester (1973/2001)
Fisher, R.A.: The Use of Multiple Measurements in Taxonomic Problems. Annals of Eugenics 7, 179–188 (1936)
Gestel, T.V., et al.: Bayesian framework for least squares support vector machine classifiers, gaussian processs and kernel fisher discriminant analysis. Neural Computation 15(5), 1115–1148 (2002)
Kailath, T., Sayed, A.H., Hassibi, B.: Estimation Theory. Prentice Hall, Englewood Cliffs (2000)
McLachlan, G.: Discriminant Analysis and statistical pattern recognition. John Wiley & Sons, Chichester (1992)
Mercer, J.: Functions of positive and negative type, and their connection with the theory of integral equations. Trans. of the London Philosophical Society (A) 209, 415–446 (1909)
Mika, S.M., Ratsch, G., Muller, K.-R.: A mathematical programming approach to the Kernel Fisher algorithm. In: Leen, T.K., Dietterich, T.G., Tresp, V. (eds.) Advances in Neural Information Processing Systems, vol. 13, pp. 591–597. MIT Press, Cambridge (2001)
Mika, S.M., Smola, A.J., Scholkopf, B.: An improved training algorithm for kernel fisher discriminants. In: Jaakkola, T., Richardson, T. (eds.) Proceedings AISTATS 2001, pp. 98–104. Morgan Kaufmann, San Francisco (2001)
Mika, S., et al.: Fisher discriminant analysis with kernels. In: Hu, Y.-H., Larsen, J., Wilson, E., Douglas, S. (eds.) Neural Networks for Signal Processing IX, pp. 41–48. IEEE, Los Alamitos (1999)
Muller, K.-R., et al.: An introduction to kernel-based learning algorithms. IEEE Transactions on Neural Networks 12(2), 181–201 (2001)
Ng, A.Y., Jordan, M., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: Proc. of NIPS-14 (2001)
Scholkopf, B., Smola, A., Muller, K.-R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10, 1299–1319 (1998)
Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Analysis and Machine Intelligence 22(8), 888–905 (2000)
Suykens, J.A.K., Vanderwalle, J.: Least Squares Support Vector Machine Classifiers. Neural Processing Letters 9(3), 293–300 (1999)
Tsuda, K.: Support vector classification with asymmetric kernel function. In: Proceedings ESANN, pp. 183–188 (1999)
Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer-Verlag, New York (1995)
Vapnik, V.N.: Statistical Learning Theory. Wiley, New York (1998)
Vert, J.-P., Tsuda, K., Scholkopf, B.: A Primer on Kernel Methods. In: Scholkopf, B., Tsuda, K., Vert, J.-P. (eds.) Kernel Methods in Computational Biology, ch. 1. MIT Press, Cambridge (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kung, SY. (2009). Kernel Approaches to Unsupervised and Supervised Machine Learning. In: Muneesawang, P., Wu, F., Kumazawa, I., Roeksabutr, A., Liao, M., Tang, X. (eds) Advances in Multimedia Information Processing - PCM 2009. PCM 2009. Lecture Notes in Computer Science, vol 5879. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-10467-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-10467-1_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-10466-4
Online ISBN: 978-3-642-10467-1
eBook Packages: Computer ScienceComputer Science (R0)