Abstract
During recent years much effort has been spent in incorporating problem specific a-priori knowledge into kernel methods for machine learning. A common example is a-priori knowledge given by a distance measure between objects. A simple but effective approach for kernel construction consists of substituting the Euclidean distance in ordinary kernel functions by the problem specific distance measure. We formalize this distance substitution procedure and investigate theoretical and empirical effects. In particular we state criteria for definiteness of the resulting kernels. We demonstrate the wide applicability by solving several classification tasks with SVMs. Regularization of the kernel matrices can additionally increase the recognition accuracy.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Bahlmann, C., Haasdonk, B., Burkhardt, H.: On-line Handwriting Recognition with Support Vector Machines—A Kernel Approach. In: Proc. of the 8th IWFHR, pp. 49–54 (2002)
Belongie, S., Fowlkes, C., Chung, F., Malik, J.: Spectral partitioning with indefinite kernels using the Nyström extension. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 21–31. Springer, Heidelberg (2002)
Canu, S.: Learning with non-positive kernels. Submitted to ICML (2004)
Chapelle, O., Haffner, P., Vapnik, V.: Support vector machines for histogram-based image classification. IEEE-NN 10(5), 1055–1064 (1999)
Graepel, T., Herbrich, R., Bollmann-Sdorra, P., Obermayer, K.: Classification on pairwise proximity data. In: NIPS 12, pp. 438–444. MIT Press, Cambridge (1999)
Graepel, T., Herbrich, R., Schölkopf, B., Smola, A., Bartlett, P., Müller, K.-R., Obermayer, K., Williamson, B.: Classification on proximity data with LP–machines. In: Proc. of the 9th ICANN, pp. 304–309 (1999)
Haasdonk, B.: Feature space interpretation ofSVMswith non positive definite kernels. Internal report 1/03, IIF-LMB, University Freiburg, Submitted to IEEE TPAMI (October 2003)
Haasdonk, B., Keysers, D.: Tangent distance kernels for support vector machines. In: Proc. of the 16th ICPR, vol. 2, pp. 864–868 (2002)
Lin, H.-T., Lin, C.-J.: A study on sigmoid kernels for SVM and the training of non-PSD kernels by SMO-type methods. Technical report, National Taiwan University (March 2003)
Mary, X.: Hilbertian subspaces, subdualities and applications. PhD thesis, INSA Rouen (2003)
Moreno, P.J., Ho, P., Vasconcelos, N.: A Kullback-Leibler divergence based kernel for SVM classification in multimedia applications. In: NIPS 17 (2003)
Pekalska, E., Paclik, P., Duin, R.: A generalized kernel approach to dissimilarity based classification. J. of Mach. Learn. Research 2, 175–211 (2001)
Roth, V., Laub, J., Kawanabe, M., Buhmann, J.M.: Optimal cluster preserving embedding of nonmetric proximity data. IEEE TPAMI 25(12), 1540–1551 (2003)
Schölkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002)
Simard, P.Y., LeCun, Y.A., Denker, J.S.: Efficient pattern recognition using a new transformation distance. In: NIPS 5, pp. 50–58. Morgan Kaufmann, San Francisco (1993)
Typke, R., Giannopoulos, P., Veltkamp, R.C., Wiering, F., van Oostrum, R.: Using transportation distances for measuring melodic similarity. In: ISMIR, pp. 107–114 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Haasdonk, B., Bahlmann, C. (2004). Learning with Distance Substitution Kernels. In: Rasmussen, C.E., Bülthoff, H.H., Schölkopf, B., Giese, M.A. (eds) Pattern Recognition. DAGM 2004. Lecture Notes in Computer Science, vol 3175. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-28649-3_27
Download citation
DOI: https://doi.org/10.1007/978-3-540-28649-3_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22945-2
Online ISBN: 978-3-540-28649-3
eBook Packages: Springer Book Archive