Abstract
One of the basic ideas of present transfer learning algorithms is to find a common underlying representation among multiple related tasks as a bridge of transferring knowledge. Different with most transfer learning algorithms which are designed to solve classification problems, a new algorithm is proposed in this paper to solve multiple regression tasks. First, based on ”self-consistency” of principal curves, this algorithm utilizes non-parametric approach to find the principal curve passing through data sets of all tasks. We treat this curve as common-across-tasks representation. Second, the importance of every sample in target task is determined by computing the deviation from the principal curve and finally the weighted support vector regression is used to obtain a regression model. We simulate multiple related regression tasks using noisy Sinc data sets with various intensities and report experiments which demonstrate that the proposed algorithm can draw the useful information of multiple tasks and dramatically improve the performance relative to learning target task independently. Furthermore, we replace principal curve with support vector regression with model selection to find common representation and show the comparative results of these two algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Dai, W., Xue, G., Yang, Q., Yu, Y.: Co-clustering based classification for out-of-domain documents. In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 210–219. ACM, New York (2007)
Yin, J., Yang, Q., Ni, L.M.: Adaptive temporal radio maps for indoor location estimation. In: Proceedings of the 3rd IEEE International Conference on Pervasive Computing and Communications (PerCom 2005), pp. 85–94 (2005)
Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering 99 (2009)
Dai, W., Yang, Q., Xue, G., Yu, Y.: Boosting for transfer learning. In: Proceedings of the 24th International Conference on Machine Learning (ICML), pp. 193–200. ACM, New York (2007)
Shi, Y., Wang, J.: Similarity-based weighting approach for transfer learning. Technical report, Sun Yat-sen University, China (2009)
Argyriou, A., Evgeniou, T., Pontil, M.: Multi-task feature learning. In: Proceedings of the 19th Annual Conference on Neural Information Processing Systems (NIPS), pp. 41–48. MIT Press, Cambridge (2007)
Pan, S.J., Kwok, J.T., Yang, Q.: Transfer learning via dimensionality reduction. In: Proceedings of the 23rd AAAI Conference on Artificial Intelligence, pp. 677–682. AAAI Press, Menlo Park (2008)
Hastie, T.: Principal curves and surfaces. Technical report, Department of Statistics, Stanford University (1983)
Hermann, T., Meinicke, P., Ritter, H.: Principal curve sonification. In: Proceedings of International Conference on Auditory Display, International Community for Auditory Displayn, pp. 81–86 (2000)
De’ath, G.: Principal curves: A new technique for indirect and direct gradient analysis. Ecology 80, 2237–2253 (1999)
Zhang, J., Wang, J.: An overview of principal curves. Chinese Journal of Computers 26, 129–146 (2003)
Kégl, B., Krzyzak, A., Linder, T., Zeger, K.: Learning and design of principal curves. IEEE Transactions on Pattern Recognition and Machine Intelligence 22, 281–297 (2000)
Qi, H., Wang, J.: A model for mining outliers from complex data sets. In: Proceedings of the 19th Annual ACM Symposium on Applied Computing, pp. 595–599. ACM, New York (2004)
Mao, W., Dong, L., Zhang, G.: Weighted solution path algorithm of support vector regression for abnormal data. In: Proceedings of the 19th International Conference on Pattern Recognition (ICPR 2008), pp. 1–4 (2008)
Tay, F., Cao, L.: Modified support vector machines in financial time series forecasting. Neurocomputing 48, 847–862 (2003)
Wang, G., Yeung, D.Y., Lochovsky, F.H.: A new solution path algorithm in support vector regression. IEEE Transactions on Neural Networks 19(10), 1753–1767 (2008)
Hu, D., Mao, W., Zhao, J., Yan, G.: Application of lssvm-pso to load identification in frequency domain. In: Proceedings of Artificial Intelligence and Computational Intelligence, pp. 231–240. Springer, Heidelberg (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mao, W., Yan, G., Bai, J., Li, H. (2010). Regression Transfer Learning Based on Principal Curve. In: Zhang, L., Lu, BL., Kwok, J. (eds) Advances in Neural Networks - ISNN 2010. ISNN 2010. Lecture Notes in Computer Science, vol 6063. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13278-0_47
Download citation
DOI: https://doi.org/10.1007/978-3-642-13278-0_47
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13277-3
Online ISBN: 978-3-642-13278-0
eBook Packages: Computer ScienceComputer Science (R0)