Abstract
Kernel matching pursuit (KMP) is a relatively new learning algorithm to produce non-linear version of conventional supervised and unsupervised learning algorithm. However, it also contains some defects such as storage problem (in training process) and sparsity problem. In this paper, a new method is proposed to pre-select the base vectors from the original data according to vector correlation principle, which could greatly reduce the scale of the optimization problem and improve the sparsity of the solution. The method could capture the structure of the data space by approximating a basis of the subspace of the data; therefore, the statistical information of the training samples is preserved. In the paper, the deduction of mathematical process is given in details and the number of simulation results on artificial data and practical data has been done to validate the performance of base vector selection (BVS) algorithm. The experimental results show the combination of such algorithm with KMP can make great progress while leave the performance almost unchanged.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Vincent, P., Bengio, Y.: Kernel matching pursuit. Machine Learning 48, 165–187 (2002)
Mallat, S.: A theory for nuliresolution signal decomposition: The wavelet representation. IEEE Trans. Pattern Anal. Machine Intell. 11, 674–693 (1989)
Davis, G., Mallat, S., Zhang, Z.: Adaptive time-frequency decompositions. Optical Engineering 33(7), 2183–2191
Vapnik, V.N.: An overview of statistical learning theory. IEEE Trans. Neural Network 10(5), 988–999 (1999)
Vapnik, V.: Three remarks on support vector machine. In: Solla, S.A., Leen, T.K., Müller, K.R. (eds.) Advances in Neural Comput., vol. 10, pp. 1299–1319 (1998)
Burges, C.J.C.: Geometry and invariance in kernel based method. In: Advance in Kernel Method-Support Vector Learning, pp. 86–116. MIT Press, Cambridge, MA (1999)
Mallat, S., Zhang, Z.: Matching pursuit with time-frequency dictionaries. IEEE Trans. Signal Proc. 41(12), 3397–3415 (1993)
Burges, C.J.C.: Geometry and invariance in kernel based method. In: Advance in Kernel Method-Support Vector Learning, pp. 86–116. MIT Press, Cambridge, MA (1999)
Baudat, G., Anouar, F.: Generalized Discriminant Analysis Using a Kernel Approach. Neural Computation 12(10), 2385–2404 (2000)
Engel, Y., Mannor, S., Meir, R.: The kernel recursive least-squares algorithm. IEEE Trans. Signal Processing 52(8), 2275–2285 (2004)
Graepel, T., Herbrich, R., Shawe-Taylor, J.: Generalization Error Bounds for Sparse Linear Classifiers. In: Thirteenth Annual Conference on Computational Learning Theroy, Morgan Kanfmann, San Francisco (in press, 2000)
Baudat, G., Anouar, F.: Kernel based methods and function approximation. In: International Joint Conference on Neural Network, IJCNN 2001, Washington, DC, pp. 1244–1249 (2001)
Baudat, G., Anouar, F.: Feature vector selection and projection using kernels. Neurocomputing 55(1-2), 21–28 (2003)
Lang, K.J., Witbrock, M.J.: Learning to tell two spirals apart. In: Proc. 1989 Connectionist Models Summer School, pp. 52–61 (1989)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Li, Q., Jiao, L. (2006). Base Vector Selection for Kernel Matching Pursuit. In: Li, X., Zaïane, O.R., Li, Z. (eds) Advanced Data Mining and Applications. ADMA 2006. Lecture Notes in Computer Science(), vol 4093. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11811305_105
Download citation
DOI: https://doi.org/10.1007/11811305_105
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-37025-3
Online ISBN: 978-3-540-37026-0
eBook Packages: Computer ScienceComputer Science (R0)