Deep Incomplete Multi-View Network Semi-Supervised Multi-Label Learning with Unbiased Loss
Pages 9048 - 9056
Abstract
Due to the explosive growth in data sources and label categories, multi-view multi-label learning has garnered widespread attention. However, multi-view multi-label data often exhibits incomplete features and a huge number of unlabeled instances, due to the technical limitations and high cost of manual labeling in practice. Learning for such simultaneous missing of view features and labels is crucial but rarely studied, particularly when the labeled samples are limited. In this paper, we tackle this problem by proposing a novel Deep Incomplete Multi-View Semi-Supervised Multi-Label Learning method (DIMvSML). Specifically, to improve high-level representations of missing features, deep graph network is firstly employed to recover the feature information with structural similarity relations. Meanwhile, we design the structure-specific deep feature extractors to obtain discriminative information and preserve the cross-view consistency for the recovered data with instance-level contrastive loss. Furthermore, to eliminate the bias of the estimate of the risk that the semi-supervised multi-label methods minimise, we design a safe estimate framework with an unbiased loss and improve its empirical performance by using pseudo-labels of unlabeled data. Besides, we provide both the theoretical proof of better estimate variance and the intuitive explanation of our debiased framework. Finally, extensive experimental results on public datasets validate the superiority of DIMvSML compared with state-of-the-art methods.
References
[1]
Luis Von Ahn and Laura Dabbish. 2004. Labeling images with a computer game. In Human Factors in Computing Systems.
[2]
Ricardo Cabral, Fernando Torre, Joao P Costeira, and Alexandre Bernardino. 2011. Matrix completion for multi-label image classification. Advances in Neural Information Processing Systems 24 (2011).
[3]
Wei-Cheng Chang, Hsiang-Fu Yu, Kai Zhong, Yiming Yang, and Inderjit S Dhillon. 2020. Taming pretrained transformers for extreme multi-label text classification. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 3163--3171.
[4]
Man-Sheng Chen, Tuo Liu, Chang-Dong Wang, Dong Huang, and Jian-Huang Lai. 2022. Adaptively-weighted integral space for fast multiview clustering. In Proceedings of the 30th ACM International Conference on Multimedia. 3774--3782.
[5]
Tianshui Chen, Muxin Xu, Xiaolu Hui, Hefeng Wu, and Liang Lin. 2019. Learning semantic-specific graph representation for multi-label image recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 522-- 531.
[6]
Pinar Duygulu, Kobus Barnard, Joao FG de Freitas, and David A Forsyth. 2002. Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. In European Conference on Computer Vision. Springer, 97--112.
[7]
Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. 2010. The pascal visual object classes (voc) challenge. International Journal of Computer Vision 88 (2010), 303--338.
[8]
Xiang Fang, Yuchong Hu, Pan Zhou, and Dapeng Wu. 2021. Animc: A soft approach for autoweighted noisy and incomplete multiview clustering. IEEE Transactions on Artificial Intelligence 3, 2 (2021), 192--206.
[9]
Hongchang Gao, Feiping Nie, Xuelong Li, and Heng Huang. 2015. Multi-view subspace clustering. In Proceedings of the IEEE International Conference on Computer Vision. 4238--4246.
[10]
Yunchao Gong, Yangqing Jia, Thomas Leung, Alexander Toshev, and Sergey Ioffe. 2013. Deep convolutional ranking for multilabel image annotation. arXiv preprint arXiv:1312.4894 (2013).
[11]
Michael Grubinger, Paul Clough, Henning Müller, and Thomas Deselaers. 2006. The iapr tc-12 benchmark: A new evaluation resource for visual information systems. In International Workshop onto Image, Vol. 2.
[12]
Matthieu Guillaumin, Jakob Verbeek, and Cordelia Schmid. 2010. Multimodal semi-supervised learning for image classification. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, 902--909.
[13]
Mark J Huiskes and Michael S Lew. 2008. The mir flickr retrieval evaluation. In Proceedings of the 1st ACM International Conference on Multimedia Information Retrieval. 39--43.
[14]
Xu Ji, Joao F Henriques, and Andrea Vedaldi. 2019. Invariant information clustering for unsupervised image classification and segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 9865--9874.
[15]
Xiaodong Jia, Xiao-Yuan Jing, Xiaoke Zhu, Songcan Chen, Bo Du, Ziyun Cai, Zhenyu He, and Dong Yue. 2020. Semi-supervised multi-view deep discriminant representation learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 7 (2020), 2496--2509.
[16]
Zhangqi Jiang, Tingjin Luo, and Xinyan Liang. 2024. Deep Incomplete Multi-View Learning Network with Insufficient Label Information. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 12919--12927.
[17]
Xiang Li and Songcan Chen. 2021. A concise yet effective model for non-aligned incomplete multi-view and missing multi-label learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 10 (2021), 5918--5932.
[18]
Yu-Feng Li and Zhi-Hua Zhou. 2014. Towards making unlabeled data never hurt. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 1 (2014), 175--188.
[19]
Wei Liang, Yuhui Li, Kun Xie, Dafang Zhang, Kuan-Ching Li, Alireza Souri, and Keqin Li. 2022. Spatial-temporal aware inductive graph neural network for C-ITS data recovery. IEEE Transactions on Intelligent Transportation Systems (2022).
[20]
Yijie Lin, Yuanbiao Gou, Xiaotian Liu, Jinfeng Bai, Jiancheng Lv, and Xi Peng. 2022. Dual contrastive prediction for incomplete multi-view representation learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 4 (2022), 4447--4461.
[21]
Chengliang Liu, Jie Wen, Xiaoling Luo, Chao Huang, Zhihao Wu, and Yong Xu. 2023. Dicnet: Deep instance-level contrastive network for double incomplete multi-view multi-label classification. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 8807--8815.
[22]
Chengliang Liu, Jie Wen, Xiaoling Luo, and Yong Xu. 2023. Incomplete multiview multi-label learning via label-guided masked view-and category-aware transformers. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 8816--8824.
[23]
Yang Liu, Kaiwen Wen, Quanxue Gao, Xinbo Gao, and Feiping Nie. 2018. SVM based multi-label learning with missing labels for image annotation. Pattern Recognition 78 (2018), 307--317.
[24]
Tingjin Luo, Chenping Hou, Feiping Nie, Hong Tao, and Dongyun Yi. 2018. Semi-Supervised Feature Selection via Insensitive Sparse Regression with Application to Video Semantic Recognition. IEEE Transactions on Knowledge and Data Engineering 30, 10 (2018), 1943--1956. https://doi.org/10.1109/TKDE.2018.2810286
[25]
Yong Luo, Dacheng Tao, Chang Xu, Chao Xu, Hong Liu, and Yonggang Wen. 2013. Multiview vector-valued manifold regularization for multilabel image classification. IEEE Transactions on Neural Networks and Learning Systems 24, 5 (2013), 709--722.
[26]
Alexander Mey and Marco Loog. 2022. Improved generalization in semisupervised learning: A survey of theoretical results. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 4 (2022), 4747--4767.
[27]
Hunter A Miller, John Lowengrub, and Hermann B Frieboes. 2022. Modeling of tumor growth with input from patient-specific metabolomic data. Annals of Biomedical Engineering 50, 3 (2022), 314--329.
[28]
Ryoma Sato. 2023. Graph neural networks can recover the hidden features solely from the graph structure. In International Conference on Machine Learning. PMLR, 30062--30079.
[29]
Hugo Schmutz, Olivier Humbert, and Pierre-Alexandre Mattei. 2022. Don't fear the unlabelled: safe semi-supervised learning via debiasing. In The Eleventh International Conference on Learning Representations.
[30]
Connor Shorten and Taghi M Khoshgoftaar. 2019. A survey on image data augmentation for deep learning. Journal of Big Data 6, 1 (2019), 1--48.
[31]
Xiangbo Shu, Jinhui Tang, Guo-Jun Qi, Zechao Li, Yu-Gang Jiang, and Shuicheng Yan. 2016. Image classification with tailored fine-grained dictionaries. IEEE Transactions on Circuits and Systems for Video Technology 28, 2 (2016), 454--467.
[32]
Ningzhao Sun, Tingjin Luo, Wenzhang Zhuge, Hong Tao, Chenping Hou, and Dewen Hu. 2023. Semi-Supervised Learning With Label Proportion. IEEE Transactions on Knowledge and Data Engineering 35, 1 (2023), 877--890. https: //doi.org/10.1109/TKDE.2021.3076457
[33]
Qiaoyu Tan, Guoxian Yu, Carlotta Domeniconi, Jun Wang, and Zili Zhang. 2018. Incomplete multi-view weak-label learning. In International Joint Conference on Artificial Intelligence. 2703--2709.
[34]
Naonori Ueda and Kazumi Saito. 2002. Parametric mixture models for multilabeled text. Advances in Neural Information Processing Systems 15 (2002).
[35]
Jesper E Van Engelen and Holger H Hoos. 2020. A survey on semi-supervised learning. Machine Learning 109, 2 (2020), 373--440.
[36]
Yiming Wang, Dongxia Chang, Zhiqiang Fu, Jie Wen, and Yao Zhao. 2022. Incomplete multiview clustering via cross-view relation transfer. IEEE Transactions on Circuits and Systems for Video Technology 33, 1 (2022), 367--378.
[37]
Jie Wen, Chengliang Liu, Shijie Deng, Yicheng Liu, Lunke Fei, Ke Yan, and Yong Xu. 2023. Deep double incomplete multi-view multi-label learning with incomplete labels and missing views. IEEE Transactions on Neural Networks and Learning Systems (2023).
[38]
Jie Wen, Yong Xu, and Hong Liu. 2018. Incomplete multiview spectral clustering with adaptive graph learning. IEEE Transactions on Cybernetics 50, 4 (2018), 1418--1429.
[39]
Yanshan Xiao, Junfeng Chen, Bo Liu, Liang Zhao, Xiangjun Kong, and Zhifeng Hao. 2024. A new multi-view multi-label model with privileged information learning. Information Sciences 656 (2024), 119911.
[40]
Junyuan Xie, Ross Girshick, and Ali Farhadi. 2016. Unsupervised deep embedding for clustering analysis. In International Conference on Machine Learning. PMLR, 478--487.
[41]
Changqing Zhang, Ziwei Yu, Qinghua Hu, Pengfei Zhu, Xinwang Liu, and Xiaobo Wang. 2018. Latent semantic aware multi-view multi-label classification. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.
[42]
Dawei Zhao, Qingwei Gao, Yixiang Lu, and Dong Sun. 2021. Two-step multi-view and multi-label learning with missing label via subspace learning. Applied Soft Computing 102 (2021), 107120.
[43]
Zhi-Hua Zhou. 2018. A brief introduction to weakly supervised learning. National science review 5, 1 (2018), 44--53.
[44]
Zhi-Hua Zhou. 2022. Open-environment machine learning. National Science Review 9, 8 (2022), nwac123.
[45]
Zhi-Hua Zhou. 2022. Rehearsal: learning from prediction to decision. Frontiers of Computer Science 16, 4 (2022), 164352.
[46]
Pengfei Zhu, Qi Hu, Qinghua Hu, Changqing Zhang, and Zhizhao Feng. 2018. Multi-view label embedding. Pattern Recognition 84 (2018), 126--135.
[47]
Wenzhang Zhuge, Tingjin Luo, Ruidong Fan, Hong Tao, Chenping Hou, and Dongyun Yi. 2024. Absent Multiview Semisupervised Classification. IEEE Transactions on Cybernetics 54, 3 (2024), 1708--1721. https://doi.org/10.1109/TCYB. 2023.3241171
Index Terms
- Deep Incomplete Multi-View Network Semi-Supervised Multi-Label Learning with Unbiased Loss
Recommendations
Uncertainty-Aware Pseudo-Labeling and Dual Graph Driven Network for Incomplete Multi-View Multi-Label Classification
MM '24: Proceedings of the 32nd ACM International Conference on MultimediaMulti-view multi-label classification has recently received extensive attention due to its wide-ranging applications across various fields, such as medical imaging and bioinformatics. However, views and labels are usually incomplete in practical ...
Semi-supervised multi-label classification using incomplete label information
Highlights- An inductive semi-supervised method called Smile is proposed for multi-label classification using incomplete label information.
AbstractClassifying multi-label instances using incompletely labeled instances is one of the fundamental tasks in multi-label learning. Most existing methods regard this task as supervised weak-label learning problem and assume sufficient ...
Inductive Semi-supervised Multi-Label Learning with Co-Training
KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data MiningIn multi-label learning, each training example is associated with multiple class labels and the task is to learn a mapping from the feature space to the power set of label space. It is generally demanding and time-consuming to obtain labels for training ...
Comments
Information & Contributors
Information
Published In
October 2024
11719 pages
ISBN:9798400706868
DOI:10.1145/3664647
- General Chairs:
- Jianfei Cai,
- Mohan Kankanhalli,
- Balakrishnan Prabhakaran,
- Susanne Boll,
- Program Chairs:
- Ramanathan Subramanian,
- Liang Zheng,
- Vivek K. Singh,
- Pablo Cesar,
- Lexing Xie,
- Dong Xu
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 October 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
MM '24
Sponsor:
MM '24: The 32nd ACM International Conference on Multimedia
October 28 - November 1, 2024
Melbourne VIC, Australia
Acceptance Rates
MM '24 Paper Acceptance Rate 1,150 of 4,385 submissions, 26%;
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 148Total Downloads
- Downloads (Last 12 months)148
- Downloads (Last 6 weeks)93
Reflects downloads up to 03 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in