Abstract
Spectral methods, as an unsupervised technique, have been used with success in data mining such as LSI in information retrieval, HITS and PageRank in Web search engines, and spectral clustering in machine learning. The essence of success in these applications is the spectral information that captures the semantics inherent in the large amount of data required during unsupervised learning. In this paper, we ask if spectral methods can also be used in supervised learning, e.g., classification. In an attempt to answer this question, our research reveals a novel kernel in which spectral clustering information can be easily exploited and extended to new incoming data during classification tasks. From our experimental results, the proposed Spectral Kernel has proved to speedup classification tasks without compromising accuracy.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press, Cambridge (2000)
Scholkopf, B., Smola, A., Muller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10, 1299–1319 (1998)
Bach, F.R., Jordan, M.I.: Kernel independent component analysis. The Journal of Machine Learning Research 3, 1–48 (2003)
Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)
Li, W., Ng, W.K., Ong, K.L., Lim, E.P.: A spectroscopy of texts for effective clustering. In: Boulicaut, J.-F., Esposito, F., Giannotti, F., Pedreschi, D. (eds.) PKDD 2004. LNCS (LNAI), vol. 3202, Springer, Heidelberg (2004)
Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, T.K., Harshman, R.: Indexing by latent semantic analysis. J. American Society of Information Science 41, 391–407 (1990)
Kleinberg, J.M.: Authoritative Sources in a Hyperlinked Environment. Journal of the ACM 46, 604–632 (1999)
Lawrence, P., Sergey, B., Rajeev, M., Terry, W.: The pagerank citation ranking: Bringing order to the web. Technical report, Stanford Digital Library Technologies Project (1999)
Siolas, G., d’Alché Buc, F.: Support vector machines based on a semantic kernel for text categorization. In: Proc. Int. Joint Conf. on Neural Networks (IJCNN), pp. 5205–5210 (2000)
Moschitti, A., Bejan, C.A.: A semantic kernel for predicate argument classification. In: Proc. Natural Language Learning (CoNLL), Boston, MA, USA, pp. 17–24 (2004)
Gosselin, P.H., Cord, M.: Semantic kernel updating for content-based image retrieval. In: Proc. of IEEE 6th International Symposium on Multimedia Software Engineering (ISMSE), Miami, Florida, pp. 537–544 (2004)
Cristianini, N., Shawe-Taylor, J., Lodhi, H.: Latent semantic kernels. In: Proc. of 18st International Conference on Machine Learning (ICML), Williams College, US, pp. 66–73 (2001)
Chung, F.R.K.: Spectral Graph Theory. CBMS Regional Conference Series in Mathematics, vol. 92. American Mathematical Society, Providence (1997)
Ding, C.: Tutorial: Spectral clustering. In: Proc. of 21st International Conference on Machine Learning (ICML), Alberta, Canada (2004)
Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 888–905 (2000)
Maila, M., Shi, J.: A random walks view of spectral segmentation. In: Proc. of the 8th International Workshop on Artificial Intelligence and Statistics, Florida (2001)
Ng, A.Y., Jordan, M.I., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: Proc. of Advances in Neural Information Processing Systems, vol. 14 (2001)
Papadimitriou, C.H., Tamaki, H., Raghavan, P., Vempala, S.: Latent semantic indexing: A probabilistic analysis. In: Proc. ACM PODS, Seattle, Washington, USA, pp. 159–168 (1998)
Golub, G., Reinsch, C.: Handbook for Matrix Computation II, Linear Algebra. Springer, New York (1971)
Li, W., Ong, K.L., Sun, A., Ng, W.K.: Spectral kernels for classification. Technical Report TRC05/05, Deakin University (2005), http://www.deakin.edu.au/~leong/tr
Kamvar, S.D., Klein, D., Manning, C.D.: Spectral learning. In: Proc. of the 18th International Joint Conference on Artificial Intelligence (IJCAI) (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Li, W., Ong, KL., Ng, WK., Sun, A. (2005). Spectral Kernels for Classification. In: Tjoa, A.M., Trujillo, J. (eds) Data Warehousing and Knowledge Discovery. DaWaK 2005. Lecture Notes in Computer Science, vol 3589. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11546849_51
Download citation
DOI: https://doi.org/10.1007/11546849_51
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28558-8
Online ISBN: 978-3-540-31732-6
eBook Packages: Computer ScienceComputer Science (R0)