ABSTRACT
The semi-supervised multi-label classification problem primarily deals with Euclidean data, such as text with a 1D grid of tokens and images with a 2D grid of pixels. However, the non-Euclidean graph-structured data naturally and constantly appears in semi-supervised multi-label learning tasks from various domains like social networks, citation networks, and protein-protein interaction (PPI) networks. Moreover, the existing popular node embedding methods, like Graph Neural Networks (GNN), focus on graphs with simplex labels and tend to neglect label correlations in the multi-label setting, so the easy adaption proves empirically ineffective. Therefore, graph representation learning for the semi-supervised multi-label learning task is crucial and challenging. In this work, we incorporate the idea of label embedding into our proposed model to capture both network topology and higher-order multi-label correlations. The label embedding is generated along with the node embedding based on the topological structure to serve as the prototype center for each class. Moreover, the similarity of the label embedding and node embedding can be used as a confidence vector to guide the label smoothing process, formulating as a margin ranking optimization problem to learn the second-order relations between labels. Extensive experiments on real-world datasets from various domains demonstrate that our model significantly outperforms the state-of-the-art models for node-level tasks.
Supplemental Material
- Uchenna Akujuobi, Yufei Han, Qiannan Zhang, and Xiangliang Zhang. 2019. Collaborative Graph Walk for Semi-Supervised Multi-label Node Classification. In ICDM. IEEE, 1--10.Google Scholar
- Matthew R. Boutell, Jiebo Luo, Xipeng Shen, and Christopher M. Brown. 2004. Learning multi-label scene classification. Pattern Recognit. 37, 9 (2004), 1757--1771.Google ScholarCross Ref
- Bobby-Joe Breitkreutz, Chris Stark, Teresa Reguly, Lorrie Boucher, Ashton Bre- itkreutz, Michael S. Livstone, Rose Oughtred, Daniel H. Lackner, Jürg Bähler, Valerie Wood, Kara Dolinski, and Mike Tyers. 2008. The BioGRID Interaction Database: 2008 update. Nucleic Acids Res. 36, Database-Issue (2008), 637--640.Google Scholar
- Sandro Cavallari, Vincent W. Zheng, Hongyun Cai, Kevin Chen-Chuan Chang, and Erik Cambria. 2017. Learning Community Embedding with Community Detection and Node Embedding on Graphs. Association for Computing Machinery, New York, NY, USA, 377--386. https://doi.org/10.1145/3132847.3132925 Google ScholarDigital Library
- Peng Cui, Xiao Wang, Jian Pei, and Wenwu Zhu. 2019. A Survey on Network Embedding. IEEE Trans. Knowl. Data Eng. 31, 5 (2019), 833--852.Google ScholarCross Ref
- Kunal Dahiya, Deepak Saini, Anshul Mittal, Ankush Shaw, Kushal Dave, Akshay Soni, Himanshu Jain, Sumeet Agarwal, and Manik Varma. 2021. DeepXML: A Deep Extreme Multi-Label Learning Framework Applied to Short Text Documents. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining (Virtual Event, Israel) (WSDM '21). Association for Computing Machinery, New York, NY, USA, 31--39. https://doi.org/10.1145/3437963.3441810 Google ScholarDigital Library
- Janez Demsar. 2006. Statistical Comparisons of Classifiers over Multiple Data Sets. J. Mach. Learn. Res. 7 (2006), 1--30. Google ScholarDigital Library
- Jie Du and Chi-Man Vong. 2020. Robust Online Multilabel Learning Under Dynamic Changes in Data Distribution With Labels. IEEE Trans. Cybern. 50, 1 (2020), 374--385.Google ScholarCross Ref
- Kaisheng Gao, Jing Zhang, and Cangqi Zhou. 2019. Semi-supervised Graph Embedding for Multi-label Graph Node Classification. In Web Information Systems Engineering -- WISE 2019, Reynold Cheng, Nikos Mamoulis, Yizhou Sun, and Xin Huang (Eds.). Springer International Publishing, Cham, 555--567.Google ScholarDigital Library
- Aditya Grover and Jure Leskovec. 2016. Node2vec: Scalable Feature Learning for Networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (San Francisco, California, USA) (KDD '16). Association for Computing Machinery, New York, NY, USA, 855--864. https://doi.org/10.1145/2939672.2939754 Google ScholarDigital Library
- William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive Representation Learning on Large Graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems (Long Beach, California, USA) (NIPS'17). Curran Associates Inc., Red Hook, NY, USA, 1025--1035. Google ScholarDigital Library
- Matthew D. Hoffman, David M. Blei, Chong Wang, and John W. Paisley. 2013. Stochastic variational inference. J. Mach. Learn. Res. 14, 1 (2013), 1303--1347. Google ScholarDigital Library
- Qian Huang, Horace He, Abhay Singh, Ser-Nam Lim, and Austin R. Benson. 2021. Combining Label Propagation and Simple Models out-performs Graph Neural Networks. In ICLR. OpenReview.net.Google Scholar
- Tony Jebara, Jun Wang, and Shih-Fu Chang. 2009. Graph Construction and b-Matching for Semi-Supervised Learning. In Proceedings of the 26th Annual International Conference on Machine Learning (Montreal, Quebec, Canada) (ICML '09). Association for Computing Machinery, New York, NY, USA, 441--448. https://doi.org/10.1145/1553374.1553432 Google ScholarDigital Library
- Jyun-Yu Jiang, Zeyu Li, Chelsea J.-T. Ju, and Wei Wang. 2020. MARU: Meta-Context Aware Random Walks for Heterogeneous Network Representation Learning. Association for Computing Machinery, New York, NY, USA, 575--584. https://doi.org/10.1145/3340531.3412040 Google ScholarDigital Library
- Diederik P. Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In ICLR.Google Scholar
- Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR (Poster). OpenReview.net.Google Scholar
- M. Mahoney. 2011. Large text compression benchmark. http://www. mattmahoney.net/dc/textdataGoogle Scholar
- Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In NeurIPS. 8024--8035. Google ScholarDigital Library
- Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: Online Learning of Social Representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (New York, New York, USA) (KDD '14). Association for Computing Machinery, New York, NY, USA, 701--710. https://doi.org/10.1145/2623330.2623732 Google ScholarDigital Library
- José Ramón Quevedo, Oscar Luaces, and Antonio Bahamonde. 2012. Multilabel classifiers with a probabilistic thresholding strategy. Pattern Recognit. 45, 2 (2012), 876--883. Google ScholarDigital Library
- Niloofar Rastin, Mansoor Zolghadri Jahromi, and Mohammad Taheri. 2021. A generalized weighted distance k-Nearest Neighbor for multi-label problems. Pattern Recognit. 114 (2021), 107526.Google ScholarCross Ref
- Jesse Read, Bernhard Pfahringer, Geoffrey Holmes, and Eibe Frank. 2009. Clas- sifier Chains for Multi-label Classification. In ECML/PKDD (2) (Lecture Notes in Computer Science, Vol. 5782). Springer, 254--269.Google Scholar
- Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic Backpropagation and Approximate Inference in Deep Generative Models. In Proceedings of the 31st International Conference on International Conference on Ma- chine Learning - Volume 32 (Beijing, China) (ICML'14). JMLR.org, II-1278-II-1286. Google ScholarDigital Library
- Andrea Rossi, Denilson Barbosa, Donatella Firmani, Antonio Matinata, and Paolo Merialdo. 2021. Knowledge Graph Embedding for Link Prediction: A Comparative Analysis. ACM Trans. Knowl. Discov. Data 15, 2 (2021), 14:1--14:49. Google ScholarDigital Library
- Xiao Shen, Quanyu Dai, Sitong Mao, Fu-Lai Chung, and Kup-Sze Choi. 2021. Network Together: Node Classification via Cross-Network Deep Network Em- bedding. IEEE Trans. Neural Networks Learn. Syst. 32, 5 (2021), 1935--1948.Google ScholarCross Ref
- Min Shi, Yufei Tang, and Xingquan Zhu. 2020. MLNE: Multi-Label Network Embedding. IEEE Trans. Neural Networks Learn. Syst. 31, 9 (2020), 3682--3695.Google ScholarCross Ref
- Min Shi, Yufei Tang, Xingquan Zhu, and Jianxun Liu. 2020. Multi-Label Graph Convolutional Network Representation Learning. IEEE Transactions on Big Data (2020).Google ScholarCross Ref
- Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjing Wang, and Yu Sun. 2021. Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, Zhi-Hua Zhou (Ed.). International Joint Conferences on Artificial Intelligence Organization, 1548--1554. https://doi.org/10.24963/ijcai.2021/214 Main Track.Google ScholarCross Ref
- Zixing Song, Xiangli Yang, Zenglin Xu, and Irwin King. 2021. Graph-based Semi- supervised Learning: A Comprehensive Review. CoRR abs/2102.13303 (2021). arXiv:2102.13303 https://arxiv.org/abs/2102.13303Google Scholar
- Fan-Yun Sun, Meng Qu, Jordan Hoffmann, Chin-Wei Huang, and Jian Tang. 2019. vGraph: A Generative Model for Joint Community Detection and Node Representation Learning. In NeurIPS. 512--522. Google ScholarDigital Library
- Heli Sun, Fang He, Jianbin Huang, Yizhou Sun, Yang Li, Chenyu Wang, Liang He, Zhongbin Sun, and Xiaolin Jia. 2020. Network Embedding for Community Detection in Attributed Networks. ACM Trans. Knowl. Discov. Data 14, 3 (2020), 36:1--36:25. Google ScholarDigital Library
- Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. LINE: Large-Scale Information Network Embedding. In Proceedings of the 24th International Conference on World Wide Web (Florence, Italy) (WWW '15). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 1067--1077. https://doi.org/10.1145/2736277.2741093 Google ScholarDigital Library
- Lei Tang and Huan Liu. 2009. Relational Learning via Latent Social Dimensions. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Dis- covery and Data Mining (Paris, France) (KDD '09). Association for Computing Machinery, New York, NY, USA, 817--826. https://doi.org/10.1145/1557019.1557109 Google ScholarDigital Library
- Lei Tang and Huan Liu. 2009. Scalable learning of collective behavior based on sparse social dimensions. In CIKM. ACM, 1107--1116. Google ScholarDigital Library
- Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, 11 (2008).Google Scholar
- Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In ICLR (Poster). OpenReview.net.Google Scholar
- Wei Wang and Min-Ling Zhang. 2020. Semi-Supervised Partial Label Learning via Confidence-Rated Margin Maximization. In NeurIPS.Google Scholar
- Xi Wang and Gita Sukthankar. 2013. Multi-Label Relational Neighbor Classifi- cation Using Social Context Features. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (Chicago, Illinois, USA) (KDD '13). Association for Computing Machinery, New York, NY, USA, 464--472. https://doi.org/10.1145/2487575.2487610 Google ScholarDigital Library
- Qingyao Wu, Yunming Ye, Shen-Shyang Ho, and Shuigeng Zhou. 2014. Semisupervised multi-label collective classification ensemble for functional genomics. BMC genomics 15, 9 (2014), 1--14.Google Scholar
- Ming-Kun Xie and Sheng-Jun Huang. 2018. Partial Multi-Label Learning. In AAAI. AAAI Press, 4302--4309.Google Scholar
- Yuanyuan Xu, Jun Wang, Shuai An, Jinmao Wei, and Jianhua Ruan. 2018. Semi- Supervised Multi-Label Feature Selection by Preserving Feature-Label Space Consistency. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management (Torino, Italy) (CIKM '18). Association for Computing Machinery, New York, NY, USA, 783--792. https://doi.org/10.1145/3269206.3271760 Google ScholarDigital Library
- M. Yang, Z. Meng, and I. King. 2020. FeatureNorm: L2 Feature Normalization for Dynamic Graph Embedding. In 2020 IEEE International Conference on Data Mining (ICDM). IEEE Computer Society, Los Alamitos, CA, USA, 731--740. https://doi.org/10.1109/ICDM50108.2020.00082Google ScholarCross Ref
- Menglin Yang, Min Zhou, Marcus Kalander, Zengfeng Huang, and Irwin King. 2021. Discrete-Time Temporal Network Embedding via Implicit Hierarchical Learning in Hyperbolic Space. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (Virtual Event, Singapore) (KDD '21). Association for Computing Machinery, New York, NY, USA, 1975--1985. https://doi.org/10.1145/3447548.3467422 Google ScholarDigital Library
- Xiangli Yang, Zixing Song, Irwin King, and Zenglin Xu. 2021. A Survey on Deep Semi-supervised Learning. CoRR abs/2103.00550 (2021). arXiv:2103.00550 https://arxiv.org/abs/2103.00550Google Scholar
- Chih-Kuan Yeh, Wei-Chieh Wu, Wei-Jen Ko, and Yu-Chiang Frank Wang. 2017. Learning Deep Latent Space for Multi-Label Classification. In AAAI. AAAI Press, 2838--2844. Google ScholarDigital Library
- Jing Zhang and Xindong Wu. 2021. Multi-Label Truth Inference for Crowdsourcing Using Mixture Models. IEEE Trans. Knowl. Data Eng. 33, 5 (2021), 2083--2095.Google Scholar
- Min-Ling Zhang and Zhi-Hua Zhou. 2007. ML-KNN: A lazy learning approach to multi-label learning. Pattern Recognit. 40, 7 (2007), 2038--2048. Google ScholarDigital Library
- Min-Ling Zhang and Zhi-Hua Zhou. 2014. A Review on Multi-Label Learning Algorithms. IEEE Trans. Knowl. Data Eng. 26, 8 (2014), 1819--1837.Google ScholarCross Ref
- M. L. Zhang, Y. K. Li, H. Yang, and X. Y. Liu. 2020. Towards Class-Imbalance Aware Multi-Label Learning. IEEE Transactions on Cybernetics (2020), 1--13. https://doi.org/10.1109/TCYB.2020.3027509Google Scholar
- Cangqi Zhou, Hui Chen, Jing Zhang, Qianmu Li, Dianming Hu, and Victor S. Sheng. 2021. Multi-label graph node classification with label attentive neighborhood convolution. Expert Systems with Applications 180 (2021), 115063. https://doi.org/10.1016/j.eswa.2021.115063Google ScholarCross Ref
- Xiaojin Zhu, Zoubin Ghahramani, and John Lafferty. 2003. Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions. In Proceedings of the Twentieth International Conference on International Conference on Machine Learn- ing (Washington, DC, USA) (ICML'03). AAAI Press, 912--919. Google ScholarDigital Library
- Yue Zhu, James T. Kwok, and Zhi-Hua Zhou. 2018. Multi-Label Learning with Global and Local Label Correlation. IEEE Trans. Knowl. Data Eng. 30, 6 (2018), 1081--1094.Google ScholarCross Ref
Index Terms
- Semi-supervised Multi-label Learning for Graph-structured Data
Recommendations
Inductive Semi-supervised Multi-Label Learning with Co-Training
KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data MiningIn multi-label learning, each training example is associated with multiple class labels and the task is to learn a mapping from the feature space to the power set of label space. It is generally demanding and time-consuming to obtain labels for training ...
Semi-supervised multi-label classification using incomplete label information
Highlights- An inductive semi-supervised method called Smile is proposed for multi-label classification using incomplete label information.
AbstractClassifying multi-label instances using incompletely labeled instances is one of the fundamental tasks in multi-label learning. Most existing methods regard this task as supervised weak-label learning problem and assume sufficient ...
Local-driven semi-supervised learning with multi-label
ICME'09: Proceedings of the 2009 IEEE international conference on Multimedia and ExpoIn this paper, we present a local-driven semi-supervised learning framework to propagate the labels of the training data (with multi-label) to the unlabeled data. Instead of using each datum as a vertex of graph, we encode each extracted local feature ...
Comments