Skip to main content
SpringerLink
Log in

A kernel learning framework for domain adaptation learning

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Domain adaptation learning (DAL) methods have shown promising results by utilizing labeled samples from the source (or auxiliary) domain(s) to learn a robust classifier for the target domain which has a few or even no labeled samples. However, there exist several key issues which need to be addressed in the state-of-theart DAL methods such as sufficient and effective distribution discrepancy metric learning, effective kernel space learning, and multiple source domains transfer learning, etc. Aiming at the mentioned-above issues, in this paper, we propose a unified kernel learning framework for domain adaptation learning and its effective extension based on multiple kernel learning (MKL) schema, regularized by the proposed new minimum distribution distance metric criterion which minimizes both the distribution mean discrepancy and the distribution scatter discrepancy between source and target domains, into which many existing kernel methods (like support vector machine (SVM), v-SVM, and least-square SVM) can be readily incorporated. Our framework, referred to as kernel learning for domain adaptation learning (KLDAL), simultaneously learns an optimal kernel space and a robust classifier by minimizing both the structural risk functional and the distribution discrepancy between different domains. Moreover, we extend the framework KLDAL to multiple kernel learning framework referred to as MKLDAL. Under the KLDAL or MKLDAL framework, we also propose three effective formulations called KLDAL-SVM or MKLDAL-SVM with respect to SVM and its variant µ-KLDALSVM or µ-MKLDALSVM with respect to v-SVM, and KLDAL-LSSVM or MKLDAL-LSSVM with respect to the least-square SVM, respectively. Comprehensive experiments on real-world data sets verify the outperformed or comparable effectiveness of the proposed frameworks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Quanz B, Huan J. Large margin transductive transfer learning. In: Proceedings of the 18th ACM Conference on Information and Knowledge Management (CIKM). New York: ACM, 2009. 1327–1336

    Chapter  Google Scholar 

  2. Pan S J, Tsang I W, Kwok J T, et al. Domain adaptation via transfer component analysis. IEEE Trans Neural Netw, 2011, 22: 199–210

    Article  Google Scholar 

  3. Xiang E W, Cao B, Hu D H, et al. Bridging domains using world wide knowledge for transfer learning. IEEE Trans Knowl Data Eng, 2010, 22: 770–783

    Article  Google Scholar 

  4. Bruzzone L, Marconcini M. Domain adaptation problems: A DASVM classification technique and a circular validation strategy. IEEE Trans Pattern Anal Mach Intell, 2010, 32: 770–787

    Article  Google Scholar 

  5. Blitzer J, Crammer K, Kulesza A, et al. Learning bounds for domain adaptation. In: Proceedings of The Neural Information Processing Systems (NIPS). Cambridge: MIT Press, 2007. 129–136

    Google Scholar 

  6. Gao J, Fan W, Jiang J, et al. Knowledge transfer via multiple model local structure mapping. In: Proceedings of the 14th ACM SIGKDD conference on Knowledge Discovery and Data Mining. New York: ACM, 2008

    Google Scholar 

  7. Ling X, Dai W, Xue G, et al. Spectral domain transfer learning. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2008

    Google Scholar 

  8. Pan S J, Yang Q. A survey on transfer learning. IEEE Trans Knowl Data Eng, 2010, 22: 1345–1359

    Article  Google Scholar 

  9. Blitzer J, McDonald R, Pereira F. Domain adaptation with structural correspondence learning. In: Proceedings of 2006 Conference Empirical Methods Natural Lang, Sydney, 2006. 120–128

  10. Blitzer J, Dredze M, Pereira F. Boom-Boxes and Blenders: domain adaptation for sentiment classification. In: Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL’ 07), Pereira, 2007. 440–447

  11. Mansour Y, Mohri M, Rostamizadeh A. Domain adaptation: learning bounds and algorithms. In: The 22nd Annual Conference on Learning Theory (COLT 2009), Montreal, 2009

  12. Daume’ III H. Frustratingly easy domain adaptation. In: Proceedings of Annual Meeting Association for Computational Linguistics, Prague, 2007. 256–263

  13. Duan L X, Tsang I W, Xu D, et al. Domain transfer SVM for video concept detection. In: Proc IEEE Int’l Conf Computer Vision and Pattern Recognition, Miami, 2009. 1375–1381

  14. Gretton A, Harchaoui Z, Fukumizu K, et al. A fast, consistent kernel two-sample test. In: Lafferty J, Williams C K I, Shawe-Taylor J, et al, eds. Advances in Neural Information Processing Systems, Vancouver, 2010. 673–681

  15. Huang J, Smola A, Gretton A, et al. Correcting sample selection bias by unlabeled data. In: Proceedings of Twentieth Annual Conference on Neural Information Processing Systems, Vancouver, 2006

  16. Duan L X, Xu D, Tsang I W. Domain adaptation from multiple sources: a domain-dependent regularization approach. IEEE Trans Neural Netw Learn Syst, 2012, 23: 504–518

    Article  Google Scholar 

  17. Duan L X, Tsang I W, Xu D, et al. Domain adaptation from multiple sources via auxiliary classifiers. In: Proceedings of the 26th International Conference on Machine Learning (ICML 2009), Montreal, 2009

  18. Duan L X, Tsang I W, Xu D. Domain transfer multiple kernel learning. IEEE Trans Pattern Anal Mach Intell, 2012, 34: 465–479

    Article  Google Scholar 

  19. Bach F R, lanckriet G R G, Jordan M. Multiple kernel learning, conic duality, and the SMO algorithm. In: Proc Int’l Conf Machine Learning. Banff: IEEE Press, 2004

    Google Scholar 

  20. Rakotomamonjy A, Bach F R, Canu S, et al. SimpleMKL. Mach Learn Res, 2008, 9: 2491–2521

    MathSciNet  MATH  Google Scholar 

  21. Sonnenburg S, Ra tsch G, Scha fer C, et al. Large scale multiple kernel learning. J Mach Learn Res, 2006, 7: 1531–1565

    MathSciNet  MATH  Google Scholar 

  22. Hu M Q, Chen Y Q, Kwok J T Y. Building sparse multiple-kernel SVM classifiers. IEEE Trans Neural Netw, 2009, 20

  23. Rosenstein M T, Marx Z, Kaelbling L P. To transfer or not to transfer. In: Advances in Neural Information Processing Systems. Cambridge: MIT Press, 2005

    Google Scholar 

  24. Seah C W, Tsang I W, Ong Y S, et al. Predictive distribution matching SVM for multi-domain learning. In: Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2010), Barcelona, 2010

  25. Luo P, Zhuang F, Xiong H, et al. Transfer learning from multiple source domains via consensus regularization. In: Proc. ACM Conf. Inf. Knowledge. Management, Napa Valley, 2008. 103–112

  26. Mansour Y, Mohri M, Rostamizadeh A. Domain adaptation with multiple sources. In: Advances in Neural Information Processing Systems 21. Cambridge: MIT Press, 2009. 1041–1048

    Google Scholar 

  27. Crammer K, Kearns M, Wortman J. Learning from multiple sources. J Mach Learn Res, 2008, 9: 1757–1774

    MathSciNet  MATH  Google Scholar 

  28. Scholkopf B, Herbrich R, Smola A J. A generalized representer theorem. In: Proc. COLT’2001. Amsterdam: Springer Press, 2001. 416–426

    Google Scholar 

  29. Vapnik V. Statistical Learning Theory. New York: John Wiley and Sons, 1998

    MATH  Google Scholar 

  30. Schölkopf B, Smola A J, Williamson R, et al. New support vector algorithms. Neural Comput, 2000, 12: 1207–1245

    Article  Google Scholar 

  31. Joachims T. Transductive inference for text classification using support vector machines. In: Bratko I, Dzeroski S, eds. Proceedings of ICML-99, 16th International Conference on Machine Learning. San Francisco: Morgan Kaufmann Publishers, 1999. 200–209

    Google Scholar 

  32. Suykens J A K, Vandewalle J. Least squares support vector machine classifiers. Neural Process Lett, 1999, 9: 293–300

    Article  MathSciNet  Google Scholar 

  33. Jiang W, Zavesky E, Chang S F, et al. Cross-domain learning methods for high-level visual concept classification. In: Proc IEEE Int’l Conf Image Processing, San Diego, 2008. 161–164

  34. Yang J, Yan R, Hauptmann A G. Cross-domain video concept detection using adaptive SVMs. In: Proc ACM Int’l Conf Multimedia, Augsburg, 2007. 188–197

  35. Ben-David S, Blitzer J, Crammer K, et al. Analysis of representations for domain adaptation. In: The Neural Information Processing Systems, Cambridge: MIT Press, 2007

    Google Scholar 

  36. Ben-David S, Luu T, Lu T, et al. Impossibility theorems for domain adaptation. J Mach Learn Res, 2010, 9: 129–136

    Google Scholar 

  37. Ben-David S, Blitzer J, Crammer K, et al. A theory of learning from different domains. Mach Learn, 2010, 79: 151–175

    Article  Google Scholar 

  38. Zhu X. Semi-supervised learning literature survey. Madison: Department of Computer Science, University of Wisconsin. Technical Report. 2008

    Google Scholar 

  39. Hofmann T, Schölkopf B, Smola A J. Kernel methods in machine learning. Annal Stat, 2007, 36: 1171–1220

    Article  Google Scholar 

  40. Sriperumbudur B K, Gretton A, Fukumizu K, et al. Hilbert space embeddings and metrics on probability measures. J Mach Learn Res, 2010, 11: 1517–1561

    MathSciNet  MATH  Google Scholar 

  41. Sriperumbadur B K, Fukumizu K, Gretton A, et al. Kernel choice and classifiability for RKHS embeddings of probability distributions. In: Advances in Neural Information Processing Systems 21. Cambridge: MIT Press, 2010. 1750–1758

    Google Scholar 

  42. Wu Y, Liu Y. Robust truncated hinge loss support vector machines. J Am Stat Assoc, 2007, 102: 974–983

    Article  MATH  Google Scholar 

  43. Tao J W, Wang S T. Locality-preserved maximum information variance v-support vector machine. Acta Autom Sin, 2012, 38: 97–108

    Google Scholar 

  44. Szedmak S, Shawe-Taylor J. Muticlass learning at one-class complexity. Southampton: School of Electronics and Computer Science. Technical Report No: 1508. 2005

    Google Scholar 

  45. Chang C C, Lin C J. Training v-support vector classifiers: Theory and algorithms. Neural Comput, 2001, 13: 2119–2147

    Article  MATH  Google Scholar 

  46. Dai Q Y W, Xue G R, Yu Y. Co-clustering based classification for out-of-domain documents. In: Proceedings of the Thirteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Jose, 2007. 210–219

  47. Cai D, He X F, Han J W. Orthogonal Laplacianfaces for face recognition. IEEE Trans Image Process, 2006, 15: 3608–3614

    Article  Google Scholar 

  48. Schweikert G, Widmer C, Schölkopf B, et al. An empirical analysis of domain adaptation algorithms for genomic sequence analysis. In: Advances in Neural Information Processing Systems 21. Cambridge: MIT Press, 2009. 1433–1440

    Google Scholar 

  49. Naphade M R, Smith J, Tesic J, et al. Large-scale concept ontology for multimedia. IEEE Multimed Mag, 2006,13: 86–91

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Digital Media, Jiangnan University, Wuxi, 214122, China

    JianWen Tao & ShiTong Wang

  2. Department of Computing, Hong Kong Polytechnic University, Hong Kong, China

    FuLai Chung & ShiTong Wang

  3. School of Information Engineering, Zhejiang Business Technology Institute, Ningbo, 315012, China

    JianWen Tao

Authors
  1. JianWen Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. FuLai Chung
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. ShiTong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to JianWen Tao.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tao, J., Chung, F. & Wang, S. A kernel learning framework for domain adaptation learning. Sci. China Inf. Sci. 55, 1983–2007 (2012). https://doi.org/10.1007/s11432-012-4611-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11432-012-4611-x

Keywords

Search

Navigation