Wei-Xuan Bao
ABSTRACT
Partial label learning deals with training examples each associated with a set of candidate labels, among which only one is valid. Most existing works focus on manipulating the label space by estimating the labeling confidences of candidate labels, while the task of manipulating the feature space by dimensionality reduction has been rarely investigated. In this paper, a novel partial label dimensionality reduction approach named CENDA is proposed via confidence-based dependence maximization. Specifically, CENDA adapts the Hilbert-Schmidt Independence Criterion (HSIC) to help identify the projection matrix, where the dependence between projected feature information and confidence-based labeling information is maximized iteratively. In each iteration, the projection matrix admits closed-form solution by solving a tailored generalized eigenvalue problem, while the labeling confidences of candidate labels are updated by conducting kNN aggregation in the projected feature space. Extensive experiments over a broad range of benchmark data sets show that the predictive performance of well-established partial label learning algorithms can be significantly improved by coupling with the proposed dimensionality reduction approach.
Supplemental Material
- K. Altun and B. Barshan. 2010. Human activity recognition using inertial/magnetic sensor units. In Proceedings of the 1st International Conference on Human Behavior Understanding. Istanbul, Turkey, 38--51.Google Scholar
Digital Library
- J. Amores. 2013. Multiple instance classification: Review, taxonomy and comparative study. Artificial intelligence, Vol. 201 (2013), 81--105.Google Scholar
- S. Boyd, S. P. Boyd, and L. Vandenberghe. 2004. Convex optimization. New York: Cambridge University Press.Google ScholarDigital Library
- F. Briggs, X. Z. Fern, and R. Raich. 2012. Rank-loss support instance machines for MIML instance annotation. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Beijing, China, 534--542.Google Scholar
- M.-A. Carbonneaua, V. Cheplyginabc, E. Granger, and G. Gagnon. 2018. Multiple instance learning: A survey of problem characteristics and applications. Pattern Recognition, Vol. 77 (2018), 329--353.Google ScholarDigital Library
- J. Chai, I. W. Tsang, and W. Chen. 2020. Large margin partial label machine. IEEE Transactions on Neural Networks and Learning Systems, Vol. 31, 7 (2020), 2594--2608.Google ScholarCross Ref
- B. Chang, U. Kruger, R. Kustra, and J. Zhang. 2013. Canonical correlation analysis based on hilbert-schmidt independence criterion and centered kernel target alignment. In Proceedings of the 30th International Conference on Machine Learning. Atlanta, Georgia, 316--324.Google Scholar
- C.-H. Chen, V. M. Patel, and R. Chellappa. 2018. Learning from ambiguously labeled face images. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 40, 7 (2018), 1653--1667.Google ScholarCross Ref
- J.-H. Chen, S.-W. Ji, B Ceran, Q Li, M.-R. Wu, and J.-P. Ye. 2008. Learning subspace kernels for classification. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Vegas, Nevada. 106--114.Google ScholarDigital Library
- Y.-C. Chen, V. M. Patel, R. Chellappa, and P. J. Phillips. 2014. Ambiguously labeled learning using dictionaries. IEEE Transactios on Information Forensics and Security, Vol. 9, 12 (2014), 2076--2088.Google ScholarDigital Library
- T. Cour, B. Sapp, and B. Taskar. 2011. Learning from partial labels. Journal of Machine Learning Research, Vol. 12 (2011), 1501--1536.Google ScholarDigital Library
- D. Dheeru and E. Karra Taniskidou. 2017. UCI Machine Learning Repository. http://archive.ics.uci.edu/mlGoogle Scholar
- L. Feng and B. An. 2019. Partial label learning with self-guided retraining. In Proceedings of the 13th AAAI Conference on Artificial Intelligence. Honolulu, Hawaii. 3542--3549.Google Scholar
- M. J. Gangeh, H. Zarkoob, and A. Ghodsi. 2017. Fast and scalable feature selection for gene expression data using hilbert-schmidt independence criterion. IEEE/ACM Transactions on Computational Biology and Bioinformatics, Vol. 14, 1 (2017), 167--181.Google ScholarDigital Library
- B. Ghojogh, F. Karray, and M. Crowley. 2019. Eigenvalue and generalized eigenvalue problems: Tutorial. ArXiv:1903.11240 (2019).Google Scholar
- C. Gong, T. Liu, Y. Tang, J. Yang, J. Yang, and D. Tao. 2018. A regularization approach for instance-based superset label learning. IEEE Transactions on Cybernetics, Vol. 48, 3 (2018), 967--978.Google ScholarCross Ref
- D. Greenfeld and U. Shalit. 2020. Robust learning with the hilbert-schmidt independence criterion. In Proceedings of the 37th International Conference on Machine Learning, Virtual Event. 3759--3768.Google Scholar
- A. Gretton, O.r Bousquet, A. Smola, and B. Schölkopf. 2005. Measuring statistical dependence with Hilbert-Schmidt norms. In Proceedings of the 16th International Conference on Algorithmic Learning Theory. Berlin, Heidelberg, 63--77.Google Scholar
- M. Guillaumin, J. Verbeek, and C. Schmid. 2010. Multiple instance metric learning from automatically labeled bags of faces. In Lecture Notes in Computer Science 6311, K. Daniilidis, P. Maragos, and N. Paragios (Eds.). Springer, Berlin, 634--647.Google Scholar
- M. J. Huiskes and M. S. Lew. 2008. The MIR Flickr retrieval evaluation. In Proceedings of the 1st ACM International Conference on Multimedia Information Retrieval. Vancouver, Canada, 39--43.Google Scholar
- E. Hüllermeier and J. Beringer. 2006. Learning from ambiguously labeled examples. Intelligent Data Analysis, Vol. 10, 5 (2006), 419--439.Google ScholarDigital Library
- S.-W. Ji and J.-P. Ye. 2009. Linear dimensionality reduction for multi-label classification. In Proceedings of the 21st International Joint Conference on Artificial Intelligence, Pasadena, California, USA.Google ScholarDigital Library
- L. Jie and F. Orabona. 2010. Learning from candidate labeling sets. In Advances in Neural Information Processing Systems 23, J. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. S. Zemel, and A. Culotta (Eds.). MIT Press, Cambridge, MA, 1504--1512.Google Scholar
- R. Jin and Z. Ghahramani. 2003. Learning with multiple labels. In Advances in Neural Information Processing Systems 15, S. Becker, S. Thrun, and K. Obermayer (Eds.). MIT Press, Cambridge, MA, 897--904.Google Scholar
- I. Katakis, G. Tsoumakas, and I. Vlahavas. 2008. Multilabel text classification for automated tag suggestion. In Proceedings of the ECML/PKDD 2008 Discovery Challenge. Antwerp, Belgium.Google Scholar
- J. Leskovec, K. Lang, A. Dasgupta, and M. Mahoney. 2009. Community structure in large networks: natural cluster sizes and the absence of large well-defined clusters. Internet Mathematics, Vol. 6, 1 (2009), 29--123.Google ScholarCross Ref
- L. Liu and T. Dietterich. 2012. A conditional multinomial mixture model for superset label learning. In Advances in Neural Information Processing Systems 25, P. Bartlett, F. C. N. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger (Eds.). MIT Press, Cambridge, MA, 557--565.Google Scholar
- J. Lv, M. Xu, L. Feng, G. Niu, X. Geng, and M. Sugiyama. 2020. Progressive identification of true labels for partial-label learning. In Proceedings of the 37th International Conference on Machine Learning. Virtual Conference, 6500--6510.Google Scholar
- G. Lyu, S. Feng, T. Wang, and C. Lang. 2021, in press. A self-paced regularization framework for partial-label learning. IEEE Transactions on Cybernetics (2021, in press).Google Scholar
- N. Nguyen and R. Caruana. 2008. Classification with partial labels. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Las Vegas, NV, 381--389.Google Scholar
- G. Panis and A. Lanitis. 2015. An overview of research activities in facial age estimation using the FG-NET aging database. In Lecture Notes in Computer Science 8926, C. Rother L. Agapito, M. M. Bronstein (Ed.). Springer, Berlin, 737--750.Google Scholar
- R. B. Pereira, A. Plastino, B. Zadrozny, and L. H. Merschmann. 2018. Categorizing feature selection methods for multi-label classification. Artificial Intelligence Review, Vol. 49, 1 (2018), 57--78.Google ScholarDigital Library
- B.-Y. Qian and I. Davidson. 2010. Semi-supervised dimension reduction for multi-label classification. In Proceedings of the 24th AAAI Conference on Artificial Intelligence. Atlanta, GA, 569--574.Google ScholarDigital Library
- X. Ren, W. He, M. Qu, C. R. Voss, H. Ji, and J. Han. 2016. Label noise reduction in entity typing by heterogeneous partial-label embedding. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. San Francisco, CA, 1825--1834.Google Scholar
- J. D. M. Rennie and R. Rifkin. 2001. Improving multiclass text classification with the support vector machines. Technical Report AIM-2001-026. Artificial Intelligence Laboratory, Massachusetts Institute of Technology.Google Scholar
- C. G. M. Snoek, M. Worring, J. C. van Gemert, J.-M. Geusebroek, and A. W. M. Smeulders. 2006. The challenge problem for automated detection of 101 semantic concepts in multimedia. In Proceedings of the 14th ACM International Conference on Multimedia. Santa Barbara, CA, 421--430.Google ScholarDigital Library
- H. Soleimani and D. J. Miller. 2016. Semi-supervised multi-label topic models for document classification and sentence labeling. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. Indianapolis, IN, 105--114.Google Scholar
- A. N. Srivastava and B. Zane-Ulman. 2005. Discovering recurring anomalies in text reports regarding complex space systems. In Proceedings of the 26th IEEE Aerospace Conference. Big Sky, MT, 3853--3862.Google Scholar
- K. Sun, Z. Min, and J. Wang. 2019. PP-PLL: Probability propagation for partial label learning. In Lecture Notes in Computer Science 11907, U. Brefeld, E. Fromont, A. Hotho, A. Knobbe, M. Maathuis, and C. Robardet (Eds.). Springer, Berlin, 123--137.Google Scholar
- L. Sun, S.-W. Ji, and J.-P. Ye. 2013. Multi-label Dimensionality Reduction. CRC Press.Google Scholar
- C.-Z. Tang and M.-L. Zhang. 2017. Confidence-rated discriminative partial label learning. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. San Francisco, CA, 2611--2617.Google ScholarDigital Library
- D.-B. Wang, L. Li, and M.-L. Zhang. 2019. Adaptive graph guided disambiguation for partial label learning. In Proceedings of the 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Anchorage, AK, 83--91.Google ScholarDigital Library
- J.-H. Wu and M.-L. Zhang. 2019. Disambiguation enabled linear discriminant analysis for partial label dimensionality reduction. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, NY, 416--424.Google ScholarDigital Library
- M. Xu and L.-Z. Guo. 2021. Learning from group supervision: the impact of supervision deficiency on multi-label learning. Science China Information Sciences, Vol. 64, 3 (2021), 1--13.Google ScholarCross Ref
- F. Yu and M.-L. Zhang. 2017. Maximum margin partial label learning. Machine Learning, Vol. 106, 4 (2017), 573--593.Google ScholarDigital Library
- K. Yu, S. Yu, and V. Tresp. 2005. Multi-label informed latent semantic indexing. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. Salvador, Brazil, 258--265.Google Scholar
- Z. Zeng, S. Xiao, K. Jia, T.-H. Chan, S. Gao, D. Xu, and Y. Ma. 2013. Learning by associating ambiguously labeled images. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition . Portland, OR, 708--715.Google Scholar
- M.-L. Zhang and F. Yu. 2015. Solving the partial label learning problem: An instance-based approach. In Proceedings of the 24th International Joint Conference on Artificial Intelligence. Buenos Aires, Argentina, 4048--4054.Google ScholarDigital Library
- M.-L. Zhang, F. Yu, and C.-Z. Tang. 2017. Disambiguation-free partial label learning. IEEE Transactions on Knowledge and Data Engineering, Vol. 29, 10 (2017), 2155--2167.Google ScholarDigital Library
- M.-L. Zhang and Z.-H. Zhou. 2007. ML-KNN: A lazy learning approach to multi-label learning. Pattern Recognition, Vol. 40, 7 (2007), 2038--2048.Google ScholarDigital Library
- M.-L. Zhang and Z.-H. Zhou. 2014. A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering, Vol. 26, 8 (2014), 1819--1837.Google ScholarCross Ref
- Y. Zhang and Z.-H. Zhou. 2010. Multilabel dimensionality reduction via dependence maximization. ACM Transactions on Knowledge Discovery from Data, Vol. 4, 3 (2010), 1--21.Google Scholar
- S. Zhao, P. Ni, H. Chen, C. Li, and Z. Dai. 2021, in press. Partial label learning via conditional-label-aware disambiguation. Journal of Computer Science and Technology (2021, in press).Google Scholar
- D. Zhou, Z. Zhang, M.-L. Zhang, and Y. He. 2018. Weakly supervised POS tagging without disambiguation. ACM Transactions on Asian and Low-Resource Language Information Processing, Vol. 17, 4 (2018), Article 35.Google ScholarDigital Library
- X.-Y. Zhou and M. Belkin. 2014. Semi-supervised learning. In Academic Press Library in Signal Processing. Vol. 1. Elsevier, 1239--1269.Google Scholar
- Z.-H. Zhou. 2018. A brief introduction to weakly supervised learning. National Science Review, Vol. 5, 1 (2018), 44--53.Google ScholarCross Ref
- Z.-H. Zhou and M.-L. Zhang. 2017. Multi-label learning. In Encyclopedia of Machine Learning and Data Mining, C. Sammut and G. I. Webb (Eds.). Springer, Berlin, 875--881.Google Scholar
- X.-J. Zhu and A. B. Goldberg. 2009. Introduction to semi-supervised learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, Vol. 3, 1 (2009), 1--130.Google ScholarCross Ref
Index Terms
- Partial Label Dimensionality Reduction via Confidence-Based Dependence Maximization
Recommendations
Multilabel dimensionality reduction via dependence maximization
Multilabel learning deals with data associated with multiple labels simultaneously. Like other data mining and machine learning tasks, multilabel learning also suffers from the curse of dimensionality. Dimensionality reduction has been studied for many ...
Disambiguation Enabled Linear Discriminant Analysis for Partial Label Dimensionality Reduction
KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data MiningPartial label learning is an emerging weakly-supervised learning framework where each training example is associated with multiple candidate labels among which only one is valid. Dimensionality reduction serves as an effective way to help improve the ...
Disambiguation Enabled Linear Discriminant Analysis for Partial Label Dimensionality Reduction
As an emerging weakly supervised learning framework, partial label learning considers inaccurate supervision where each training example is associated with multiple candidate labels among which only one is valid. In this article, a first attempt toward ...
Comments