Skip to main content
SpringerLink
Log in

Transfer Learning via Multi-View Principal Component Analysis

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Transfer learning aims at leveraging the knowledge in labeled source domains to predict the unlabeled data in a target domain, where the distributions are different in domains. Among various methods for transfer learning, one kind of algorithms focus on the correspondence between bridge features and all the other specific features from different domains, and later conduct transfer learning via the single-view correspondence. However, the single-view correspondence may prevent these algorithms from further improvement due to the problem of incorrect correlation discovery. To tackle this problem, we propose a new method for transfer learning in a multi-view correspondence perspective, which is called Multi-View Principal Component Analysis (MVPCA) approach. MVPCA discovers the correspondence between bridge features representative across all domains and specific features from different domains respectively, and conducts the transfer learning by dimensionality reduction in a multi-view way, which can better depict the knowledge transfer. Experiments show that MVPCA can significantly reduce the cross domain prediction error of a baseline non-transfer method. With multi-view correspondence information incorporated to the single-view transfer learning method, MVPCA can further improve the performance of one state-of-the-art single-view method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Blitzer J, McDonald R, Pereira F. Domain adaptation with structural correspondence learning. In Proc. Conference on Empirical Methods in Natural Language, Sydney, Australia, Jul. 22-23, 2006, pp. 120–128.

  2. Blitzer J, Dredze M, Pereira F. Biographies, Bollywood, Boomboxes and Blenders: Domain adaptation for sentiment classification. In Proc. the 45th Annual Meeting of the Association of Computational Linguistics, Prague, Czech, Jun. 25-27, 2007, pp. 432–439.

  3. Dhillon I S, Mallela S, Modha D S. Information-theoretic co-clustering. In Proc. the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), Washington DC, USA, Aug. 24-27, 2003, pp. 89–98.

  4. Pan S J, Tsang I W, Kwok J T, Yang Q. Domain adaptation via transfer component analysis. In Proc. the 21st International Joint Conference on Arti¯cial Intelligence, Pasadena, USA, Jul. 11-17, 2009, pp. 1187–1192.

  5. Daumé H. Frustratingly easy domain adaptation. In Proc. the 45th Annual Meeting of the Association for Computational Linguistics, Prague, Czech, Jun. 25-27, 2007, pp. 256–263.

  6. Daumé H, Marcu D. Domain adaptation for statistical classifiers. Journal of Artificial Intelligence Research, 2006, 26: 101–126.

    MATH  MathSciNet  Google Scholar 

  7. Dai W Y, Xue G R, Yang Q, Yu Y. Co-clustering based classification for out-of-domain documents. In Proc. the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Jose, USA, Aug. 12-15, 2007, pp. 210–219.

  8. Pan S J, Kwok J T, Yang Q, Pan J J. Adaptive localization in a dynamic WiFi environment through multi-view learning. In Proc. the 22nd AAAI Conference on Artificial Intelligence, Vancouver, Canada, Jul. 22-26, 2007, pp. 1108–1113.

  9. DaiWY, Yang Q, Xue G R, Yu Y. Boosting for transfer learning. In Proc. the 24th International Conference on Machine Learning, Corvalis, USA, Jun. 20-24, 2007, pp. 193–200.

  10. Huang J, Smola A, Gretton A, Borgwardt K M, Schölkopf B. Correcting sample selection bias by unlabeled data. In Proc. the 19th Annual Conference on Neural Information Processing Systems, Vancouver, Canada, Dec. 4-6, 2006, pp. 601–608.

  11. Zadrozny B. Learning and evaluating classifiers under sample selection bias. In Proc. the 21st International Conference on Machine Learning, Banff, Canada, Jul. 4-8, 2004, pp. 903–910.

  12. Jiang J, Zhai C X. Instance weighting for domain adaptation in NLP. In Proc. the 45th Annual Meeting of the Association of Computational Linguistics, Prague, Czech, Jun. 25-27, 2007, pp. 264–271.

  13. Raina R, Battle A, Lee H, Packer B, Ng A Y. Self-taught learning: Transfer learning from unlabeled data. In Proc. the 24th International Conference on Machine Learning, Corvalis, USA, Jun. 20-24, 2007, pp. 759–766.

  14. Lee H, Battle A, Raina R, Ng A Y. Efficient sparse coding algorithms. In Proc. the 19th Annual Conference on Neural Information Processing Systems, Vancouver, Canada, Dec. 4-6, 2006, pp. 801–808.

  15. Tan S, Cheng X, Wang Y, Xu H. Adapting naive Bayes to domain adaptation for sentiment analysis. In Proc. the 31st European Conference on IR Research, Toulouse, France, Apr. 7-9, 2009, pp. 337–349.

  16. Dai W Y, Xue G R, Yang Q, Yu Y. Transferring Naive Bayes classifiers for text classification. In Proc. the 22nd AAAI Conference on Artificial Intelligence (AAAI 2007), Vancouver, Canada, Jul. 22-26, 2007, pp. 540–545.

  17. Sandler T, Blitzer J, Talukdar P, Pereira F. Regularized learning with networks of features. In Proc. the 22nd Annual Conference on Neural Information Processing Systems, Vancouver, Canada, Dec. 8-11, 2008, pp. 1401–1408.

  18. Vinokourov A, Shawe-Taylor J, Cristianini N. Inferring a semantic representation of text via cross-language correlation analysis. In Proc. the 15th Advances of Neural Information Processing Systems, Vancouver, Canada, Dec. 9-12, 2002, pp. 1473–1480.

  19. Li Y, Shawe-Taylor J. Using KCCA for Japanese-English cross-language information retrieval and classification. Journal of Intelligent Information Systems, 2006, 27(2): 117–133.

    Article  Google Scholar 

  20. Hardoon D R, Szedmak S, Shawe-Taylor J. Canonical correlation analysis: An overview with application to learning methods, Neural Computation, 2004, 16(12): 2639–2664.

    Article  MATH  Google Scholar 

  21. Hotelling H. Relations between two sets of variates. Biometrika, 1936, 28(3/4): 321–377.

    Article  MATH  Google Scholar 

  22. Diethe T, Hardoon D R, Shawe-Taylor J. Multiview Fisher discriminant analysis. In Proc. Learning from Multiple Sources Workshop (NIPS 2008), Whistler, Canada, Dec. 13, 2008.

  23. Zhou Z H, Zhan D C, Yang Q. Semi-supervised learning with very few labeled training examples. In Proc. the 22nd AAAI Conference on Arti¯cial Intelligence (AAAI 2007), Vancouver, Canada, Jul. 22-26, 2007, pp. 675–680.

  24. Zhang D Q, Zhou Z H, Chen S C. Semi-supervised dimensionality reduction. In Proc. the 7th SIAM International Conference on Data Mining (SDM2007), Minneapolis, USA, Apr. 26-28, 2007, pp. 629–634.

  25. Long C, Huang M L, Zhu X Y, Ming Li. A new approach for multi-document update summarization. Journal of Computer Science and Technology, 2010, 25(4): 739–749.

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Nanjing University, Nanjing, 210093, China

    Yang-Sheng Ji, Jia-Jun Chen (Member, CCF), Gang Niu, Lin Shang (Member, CCF) & Xin-Yu Dai (Member, CCF)

  2. National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210093, China

    Yang-Sheng Ji, Jia-Jun Chen (Member, CCF), Gang Niu, Lin Shang (Member, CCF) & Xin-Yu Dai (Member, CCF)

Authors
  1. Yang-Sheng Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jia-Jun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gang Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lin Shang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xin-Yu Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yang-Sheng Ji.

Additional information

This work is supported by the National Natural Science Foundation of China under Grant No. 61003112, the National Basic Research 973 Program of China under Grant No. 2010CB327903, the National High-Tech Research and Development 863 Program of China under Grant No. 2006AA010109, and the Natural Science Foundation of Jiangsu Province under Grant No. 2009233.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(PDF 81 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ji, YS., Chen, JJ., Niu, G. et al. Transfer Learning via Multi-View Principal Component Analysis. J. Comput. Sci. Technol. 26, 81–98 (2011). https://doi.org/10.1007/s11390-011-9417-6

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-011-9417-6

Keywords

Search

Navigation