Abstract
Multi-task averaging deals with the problem of estimating the means of a set of distributions jointly. It has its roots in the fifties when it was observed that leveraging data from related distributions can yield superior performance over learning from each distribution independently. Stein’s paradox showed that, in an average square error sense, it is better to estimate the means of T Gaussian random variables using data sampled from all of them. This phenomenon has been largely disregarded and has recently emerged again in the field of multi-task learning. In this paper, we extend recent results for multi-task averaging to the n-dimensional case and propose a method to detect from data which tasks/distributions should be considered as related. Our experimental results indicate that the proposed method compares favorably to the state of the art.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Argyriou, A., Maurer, A., Pontil, M.: An algorithm for transfer learning in a heterogeneous environment. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008, Part I. LNCS (LNAI), vol. 5211, pp. 71–85. Springer, Heidelberg (2008)
Gretton, A., Sriperumbudur, B., Sejdinovic, D., Strathmann, H., Balakrishnan, S., Pontil, M., Fukumizu, K.: Optimal kernel choice for large-scale two-sample tests. In: Advances in Neural Information Processing Systems 25, pp. 1214–1222 (2012)
Efron, B., Morris, C.N.: Stein’s paradox in statistics. Scientific American 236(5), 119–127 (1977)
Feldman, S., Gupta, M.R., Frigyik, B.A.: Multi-task averaging. In: Advances in Neural Information Processing Systems 25, pp. 1178–1186 (2012); MMM This paper seems obscure. can you add volume, page number etc? I would otherwise remove this citation unless strictly needed PPP
He, X., Mourot, G., Maquin, D., Ragot, J.: One-class SVM in multi-task learning. In: Advances in Safety, Reliability and Risk Management (2012)
James, W., Stein, C.: Estimation with quadratic loss. In: Proc. Fourth Berkeley Symposium on Mathematical Statistics and Probability, pp. 361–379 (1961)
Lehmann, E.L., Casella, G.: Theory of Point Estimation. Springer, New York (1998)
Maurer, A., Pontil, M., Romera-Paredes, B.: Sparse coding for multitask and transfer learning. In: Proceedings of the 30th International Conference on Machine Learning (2013)
Ng, A.Y., Jordan, M., Weiss, Y.: On spectral clustering: analysis and an algorithm. In: Advances in Neural Information Processing Systems (2002)
Principe, J.C.: Information Theoretic Learning. Renyi’s Entropy and Kernel Perspectives. Springer (2000)
Parzen, E.: On Estimation of a Probability Density Function and Mode. Annals of Mathematics and Statistics 33(3), 1065–1076 (1962)
Stein, C.: Inadmissibility of the usual estimator for the mean of a multivariate distribution. In: Proc. Third Berkeley Symposium on Mathematical Statistics and Probability, pp. 197–206 (1956)
Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8), 888–905 (2000)
von Luxburg, U.: A Tutorial on Spectral Clustering. Statistics and Computing 17(4), 395–416 (2007)
Yang, H., King, I., Lyu, M.R.: Multi-task Learning for One-class Classification. In: International Joint Conference on Neural Networks (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Martínez-Rego, D., Pontil, M. (2013). Multi-task Averaging via Task Clustering. In: Hancock, E., Pelillo, M. (eds) Similarity-Based Pattern Recognition. SIMBAD 2013. Lecture Notes in Computer Science, vol 7953. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39140-8_10
Download citation
DOI: https://doi.org/10.1007/978-3-642-39140-8_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39139-2
Online ISBN: 978-3-642-39140-8
eBook Packages: Computer ScienceComputer Science (R0)