Abstract
Motivated by the importance of kernel-based methods for multi-task learning, we provide here a complete characterization of multi-task finite rank kernels in terms of the positivity of what we call its associated characteristic operator. Consequently, we are led to establishing that every continuous multi-task kernel, defined on a cube in an Euclidean space, not only can be uniformly approximated by multi-task polynomial kernels, but also can be extended as a multi-task kernel to all of the Euclidean space. Finally, we discuss the interpolation of multi-task kernels by multi-task finite rank kernels.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Argyriou, A., Evgeniou, T., Pontil, M.: Convex multi-task feature learning. Mach. Learn. 73, 243–272 (2008)
Argyriou, A., Micchelli, C.A., Pontil, M.: When is there a representer theorem? Vector versus matrix regularizers. J. Mach. Learn. Res. 10, 2507–2529 (2009)
Argyriou, A., Micchelli, C.A., Pontil, M.: On spectral learning. J. Mach. Learn. Res. 11, 935–953 (2010)
Aronszajn, N.: Theory of reproducing kernels. Trans. Am. Math. Soc. 68, 334–404 (1950)
Bakker, B., Heskes, T.: Task clustering and gating for Bayesian multi-task learning. J. Mach. Learn. Res. 4, 83–99 (2003)
Blanz, V., Schölkopf, B., Bülthoff, H., Burges, C., Vapnik, V., Vetter, T.: Comparison of view-based object recognition algorithms using realistic 3D models. In: von der Malsburg, C., von Seelen, W., Vorbrüggen, J.C., Sendhoff, B. (eds.) Artificial Neural Networks–ICANN’96. Lecture Notes in Computer Science, vol. 1112, pp. 251–256. Springer, Berlin (1996)
Boser, B., Guyon, I., Vapnik, V.: A training algorithm for optimal margin classifiers. In: Proc. of the 5th Annual Workshop of Computational Learning Theory, vol. 5, pp. 144–152. ACM, Pittsburgh (1992)
Burges, C., Schölkopf, B.: Improving the accuracy and speed of support vector machines. In: Mozer, M., Jordan, M., Petsche, T. (eds.) Advances in Neural Information Processing Systems, vol. 9, pp. 375–381. MIT Press, Cambridge, MA (1997)
Caponnetto, A., Micchelli, C.A., Pontil, M., Ying, Y.: Universal multi-task kernels. J. Mach. Learn. Res. 9, 1615–1646 (2008)
Caruana, R.: Multi-task learning. Mach. Learn. 28, 41–75 (1997)
Cortes, C., Vapnik, V.: Support vector networks. Mach. Learn. 20, 273–297 (1995)
Evgeniou, T., Micchelli, C.A., Pontil, M.: Learning multiple tasks with kernel methods. J. Mach. Learn. Res. 6, 615–637 (2005)
Evgeniou, T., Pontil, M.: Regularized multi-task learning. In: International Conference on Knowledge Discovery and Data Mining (2004)
Goldberg, Y., Elhadad, M.: splitSVM: fast, space-efficient, non-heuristic, polynomial kernel computation for NLP applications. In: Proceedings of ACL-08: HLT, Short Papers (Companion Volume), pp. 237–240 (2008)
Isozaki, H., Kazawa, H.: Efficient support vector classifiers for named entity recognition. In: Proceedings of the 19th International Conference on Computational Linguistics (COLING02) (2002)
Joachims, T.: Text categorization with support vector machines: learning with many relevant features. Technical report, LS VIII Number 23. University of Dortmund (1997)
Kudo, T., Matsumoto, Y.: Fast methods for kernel-based text analysis. In: Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pp. 24–31 (2003)
Lorentz, G.G.: Bernstein Polynomials. University of Toronto Press, Toronto (1953)
Micchelli, C.A., Pontil, M.: On learning vector-valued functions. Neural Comput. 17, 177–204 (2005)
Micchelli, C.A., Xu, Y., Zhang, H.: Universal kernels. J. Mach. Learn. Res. 7, 2651–2667 (2006)
Schölkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge, MA (2002)
Thrun, S., Pratt, L.: Learning to Learn. Kluwer Academic Publishers (1997)
Tong, H., Chen, D., Peng, L.: Learning rates for regularized classifiers using multivariate polynomial kernels. J. Complex. 24, 619–631 (2008)
Vapnik, V.N.: Statistical Learning Theory. Wiley, New York (1998)
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Lixin Shen.
Rights and permissions
About this article
Cite this article
Liu, J., Micchelli, C.A., Wang, R. et al. Finite rank kernels for multi-task learning. Adv Comput Math 38, 427–439 (2013). https://doi.org/10.1007/s10444-011-9244-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10444-011-9244-x
Keywords
- Multi-task polynomial kernels
- Characteristic operator
- Weierstrass approximation theorem
- Continuous kernel extension