Abstract
Kernel combination is meant to improve the performance of single kernels and avoid the difficulty of kernel selection. The most common way of combining kernels is to compute their weighted sum. Usually, the kernels are assumed to exist in independent empirical feature spaces and therefore were combined without considering their relationships.
To take these relationships into consideration in kernel combination, we propose the generalized augmentation kernel which is extended by all the single kernels considering their correlations. The generalized augmentation kernel, unlike the weighted sum kernel, does not need to find out the weight of each kernel, and also would not suffer from information loss due to the average of kernels.
In the experiments, we observe that the generalized augmentation kernel usually can achieve better performances than other combination methods that do not consider relationship between kernels.
We acknowledge financial support from the FET programme within the EU FP7, under the SIMBAD project (contract 213250).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bennett, K.P., Momma, M., Embrechts, M.J.: MARK: A Boosting Algorithm for Heterogeneous Kernel Models. In: Proc. 8th ACMSIGKDD Int. Conf. Knowledge Discovery and Data Mining, pp. 24–31 (2002)
Bousquet, O., Herrmann, D.: On the Complexity of Learning the Kernel Matrix. In: Proc. Advances in Neural Information Processing Systems, pp. 415–422 (2003)
Burges, C.J.C.: A Tutorial on Support Vector Machines for Pattern Recognition. Knowledge Discovery and Data Mining 2(2), 1–43 (1998)
Camps-Valls, G., Gomez-Chova, L., Muñoz-Marí, J., Vila-Francés, J., Calpe-Maravilla, J.: Composite Kernels for Hyperspectral Image Classification. IEEE Geoscience and Remote Sensing Letters 3(1), 93–97 (2006)
Chang, C.C., Lin, C.J.: LIBSVM: A Library for Support Vector Machines, Taiwan, National Taiwan University (2001), http://www.csie.ntu.edu.tw/cjlin/libsvm
Crammer, K., Keshet, J., Singer, Y.: Kernel Design Using Boosting. In: Proc. of the Fifteenth Annual Conference on Neural Information Processing Systems (2002)
Cristianini, N., Kandola, J., Elisseeff, A., Shawe-Taylor, J.: On Kernel Target Alignment. Technical Report NeuroColt, pp. 2001–2099. Royal Holloway University, London (2001)
de Diego, I.M., Moguerza, J.M., Mu noz, A.: Combining Kernel Information for Support Vector Classification. In: Proc. Multiple Classifier Systems, pp. 102–111 (2004)
Fung, G., Dundar, M., Bi, J., Rao, B.: A Fast Iterative Algorithm for Fisher Discriminant Using Heterogeneous Kernels. In: Proc. 21st Int. Conf. Machine Learning (2004)
Lanckriet, G.R.G., Cristianini, N., Bartlett, P., Ghaoui, L.E., Jordan, M.I.: Learning the Kernel Matrix with Semidefinite Programming. Journal of Machine Learning Research 5, 27–72 (2004)
Lee, W.-J., Verzakov, S.A., Duin, R.P.W.: Kernel Combination Versus Classifier Combination. In: Proc. Multiple Classifier Systems, pp. 22–31 (2007)
Lin, C.T., Yeh, C.M., Liang, S.F., Chung, J.F., Kumar, N.: Support-Vector-Based Fuzzy Neural Network for Pattern Classification. IEEE Trans. on Fuzzy Systems 14(1), 31–41 (2006)
Micchelli, C.A., Pontil, M.: Learning the Kernel Function via Regularization. Journal of Machine Learning Research 6, 1099–1125 (2005)
Moguerza, J.M., Munoz, A., de Diego, I.M.: Improving Support Vector Classification via the Combination of Multiple Sources of Information. In: SSPR/SPR, pp. 592–600 (2004)
Asuncion, A., Newman, D.J.: UCI Machine Learning Repository. University of California, Department of Information and Computer Science, Irvine, CA (2007), http://www.ics.uci.edu/mlearn/MLRepository.html
Jain, A.K., Ramaswami, M.D.: Classifier design with Parzen window. Pattern Recogition and Artificial Intelligence (1988)
Duin, R.P.W., Juszczak, P., Paclik, P., Pȩkalska, E., de Ridder, D., Tax, D.M.J.: PRTOOLS4, A Matlab Toolbox for Pattern Recognition, Delft University of Technology, Pattern Recognition Laboratory, The Netherlands (2004), http://www.prtools.org
Ong, C.S., Smola, A.J., Williamson, R.C.: Learning the Kernel with Hyperkernels. Journal of Machine Learning Research 6, 1043–1071 (2005)
Tsang, I.W.H., Kwok, J.T.Y.: Efficient Hyperkernel Learning Using Second-Order Cone Programming. IEEE Trans. on Neural Networks 17(1), 48–58 (2006)
Vapnik, V.: The Nature of Statistical Learning Theory. Springer, Heidelberg (1995)
Yan, F., Mikolajczyk, K., Kittler, J., Tahir, M.A.: Combining Multiple Kernels by Augmenting the Kernel Matrix. In: Proc. Multiple Classifier Systems, pp. 175–184 (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lee, WJ., Duin, R.P.W., Loog, M. (2011). Generalized Augmentation of Multiple Kernels. In: Sansone, C., Kittler, J., Roli, F. (eds) Multiple Classifier Systems. MCS 2011. Lecture Notes in Computer Science, vol 6713. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21557-5_14
Download citation
DOI: https://doi.org/10.1007/978-3-642-21557-5_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21556-8
Online ISBN: 978-3-642-21557-5
eBook Packages: Computer ScienceComputer Science (R0)