Jintao Zhang
ABSTRACT
In many real-world applications, it is becoming common to have data extracted from multiple diverse sources, known as "multi-view" data. Multi-view learning (MVL) has been widely studied in many applications, but existing MVL methods learn a single task individually. In this paper, we study a new direction of multi-view learning where there are multiple related tasks with multi-view data (i.e. multi-view multi-task learning, or MVMT Learning). In our MVMT learning methods, we learn a linear mapping for each view in each task. In a single task, we use co-regularization to obtain functions that are in-agreement with each other on the unlabeled samples and achieve low classification errors on the labeled samples simultaneously. Cross different tasks, additional regularization functions are utilized to ensure the functions that we learn in each view are similar. We also developed two extensions of the MVMT learning algorithm. One extension handles missing views and the other handles non-uniformly related tasks. Experimental studies on three real-world data sets demonstrate that our MVMT methods significantly outperform the existing state-of-the-art methods.
Supplemental Material
- S. Abney. Bootstrapping. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pages 360--367, 2002. Google Scholar
Digital Library
- M.-R. Amini, N. Usunier, and C. Goutte. Learning from multiple partially observed views - an application to multilingual text categorization. In NIPS'09, pages 28--36, 2009.Google Scholar
- R. Ando and T. Zhang. A framework for learning predictive structures from multiple tasks and unlabeled data. J. Mach. Learn. Res., 6:1817--1853, 2005. Google ScholarDigital Library
- A. Argyriou, T. Evgeniou, and M. Pontil. Multi-task feature learning. In NIPS'06, pages 41--48, 2006.Google Scholar
- M.-f. Balcan and A. Blum. A pac-style model for learning from labeled and unlabeled data. In Proceedings of COLT'05, pages 111--126, 2005. Google ScholarDigital Library
- A. Blum and T. Mitchell. Combining labeled and unlabeled data with co-training. In COLT'98, pages 92--100, 1998. Google ScholarDigital Library
- M. R. Boutell, J. Luo, X. Shen, and C. M. Brown. Learning multi-label scene classification. Pattern Recognition, 37(9):1757--71, 2004.Google ScholarCross Ref
- G. Cavallanti, N. Cesa-Bianchi, and C. Gentile. Linear algorithms for online multitask classification. J. Mach. Learn. Res., 11:2901--2934, 2010. Google ScholarDigital Library
- J. Chen, J. Liu, and J. Ye. Learning incoherent sparse and low-rank patterns from multiple tasks. In KDD'10, pages 1179--1188, 2010. Google ScholarDigital Library
- X. Chen, S. Kim, Q. Lin, J. Carbonell, and E. Xing. Graph-structured multi-task regression and an efficient optimization method for general fused lasso. Stat., 1050:21, 2010.Google Scholar
- C. M. Christoudias, R. Urtasun, and T. Darrell. Multi-view learning in the presence of view disagreement. In Proceedings of UAI'08, 2008.Google Scholar
- T.-S. Chua, J. Tang, R. Hong, H. Li, Z. Luo, and Y.-T. Zheng. Nus-wide: A real-world web image database from national university of singapore. In CIVR'09, 2009. Google ScholarDigital Library
- M. Culp, G. Michailidis1, and K. Johnson. On multi-view learning with additive models. Ann. Applied Stat., 3(1):292--318, 2009.Google ScholarCross Ref
- S. Dasgupta, M. Littman, and D. McAllester. Pac generalization bounds for co-training. In NIPS'01, pages 375--382, 2001.Google Scholar
- T. Evgeniou and M. Pontil. Regularized multi-task learning. In KDD'04, pages 109--117, 2004. Google ScholarDigital Library
- J. He and R. Lawrence. A graph-based framework for multi-task multi-view learning. In ICML'11, 2011.Google Scholar
- S. Kim and E. P. Xing. Tree-guided group lasso for multi-task regression with structured sparsity. In ICML'10, pages 543--550, 2010.Google Scholar
- A. McCallum, K. Nigam, J. Rennie, and S. Kim. Automating the construction of internet portals with machine learning. Information Retrieval, 3:127--163, 2000. Google ScholarDigital Library
- I. Muslea, S. Minton, and C. A. Knoblock. Adaptive view validation: A first step towards automatic view detection. In Proceedings of ICML'02, pages 443--450, 2002. Google ScholarDigital Library
- K. Nigam and R. Ghani. Analyzing the effectiveness and applicability of co-training. In CIKM'00, pages 86--93, 2000. Google ScholarDigital Library
- V. Sindhwani and P. Niyogi. A co-regularized approach to semi-supervised learning with multiple views. In ICML Workshop on Learning with Multiple Views, 2005.Google Scholar
- V. Sindhwani and D. Rosenberg. An rkhs for multi-view learning and manifold co-regularization. In ICML'08, pages 976--983, 2008. Google ScholarDigital Library
- W. Wang and Z. H. Zhou. Analyzing co-training style algorithms. In Proceedings of ECML'07, pages 454--465, 2007. Google ScholarDigital Library
- W. Wang and Z. H. Zhou. A new analysis of co-training. In Proceedings of ICML'10, 2010.Google Scholar
- Y. Zhang and D.-Y. Yeung. A convex formulation for learning task relationships in multi-task learning. In Proceedings of UAI'10, pages 733--442, Corvallis, Oregon, 2010.Google Scholar
Index Terms
- Inductive multi-task learning with multiple view data
Recommendations
Multi-task Multi-view Learning for Heterogeneous Tasks
CIKM '14: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge ManagementMulti-task multi-view learning deals with the learning scenarios where multiple tasks are associated with each other through multiple shared feature views. All previous works for this problem assume that the tasks use the same set of class labels. ...
Multi-view learning overview
This overview reviews theoretical underpinnings of multi-view learning.It provides comprehensive introduction for the recent developments of multi-view learning methods.It attempts to identify promising venues and point out some specific challenges. ...
Multi-task and multi-view learning of user state
Several computational approaches have been proposed for inferring the affective state of the user, motivated for example by the goal of building improved interfaces that can adapt to the user@?s needs and internal state. While fairly good results have ...
Comments