Zhihui Hu
ABSTRACT
Graph Convolutional Networks (GCNs) are powerful representation learning methods for non-Euclidean data. Compared with the Euclidean data, labeling the non-Euclidean data is more expensive. Meanwhile, most existing GCNs only utilize few labeled data but ignore most of the unlabeled data. To address this issue, we design a novel end-to-end Iterative Feature Clustering Graph Convolutional Networks (IFC-GCN) that enhances the standard GCN with an Iterative Feature Clustering (IFC) module. The proposed IFC module constrains node features iteratively based on the predicted pseudo labels and feature clustering. Further, we design an EM-like framework for IFC-GCN training, which improves the network performance by rectifying the pseudo labels and the node features alternately. Theoretical analysis and experimental results show that our proposed IFC module can effectively modify the node features. Experimental results on public datasets demonstrate that IFC-GCN outperforms state-of-the-art methods on the semi-supervised node classification task.
Supplemental Material
- Sami Abu-El-Haija, Bryan Perozzi, Amol Kapoor, Nazanin Alipourfard, Kristina Lerman, Hrayr Harutyunyan, Greg Ver Steeg, and Aram Galstyan. 2019. MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing. In ICML. 21--29.Google Scholar
- Marcel R. Ackermann, Johannes Blömer, Daniel Kuntze, and Christian Sohler. 2014. Analysis of Agglomerative Clustering. Algorithmica, Vol. 69, 1 (2014), 184--215.Google ScholarCross Ref
- Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. 2014. Spectral Networks and Locally Connected Networks on Graphs. In ICLR.Google Scholar
- Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze. 2018. Deep Clustering for Unsupervised Learning of Visual Features. In ECCV. 139--156.Google Scholar
- Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. Enhanced LSTM for Natural Language Inference. In ACL. 1657--1668.Google Scholar
- Michael Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. In NIPS. 3837--3845. Google ScholarDigital Library
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT. 4171--4186.Google Scholar
- Luca Franceschi, Mathias Niepert, Massimiliano Pontil, and Xiao He. 2019. Learning Discrete Structures for Graph Neural Networks. In ICML. 1972--1982.Google Scholar
- Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive Representation Learning on Large Graphs. In NIPS. 1024--1034. Google ScholarDigital Library
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In CVPR. 770--778.Google Scholar
- R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. 2019. Learning deep representations by mutual information estimation and maximization. In ICLR.Google Scholar
- Anil K. Jain. 2010. Data Clustering: 50 Years Beyond K-means. Pattern Recognition Letters , Vol. 31, 8 (2010), 651--666. Google ScholarDigital Library
- Bowen Jin, Chen Gao, Xiangnan He, Depeng Jin, and Yong Li. 2020. Multi-behavior Recommendation with Graph Convolutional Networks. In SIGIR. 659--668. Google ScholarDigital Library
- Ming Jin, Yizhen Zheng, Yuan-Fang Li, Chen Gong, Chuan Zhou, and Shirui Pan. 2021. Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning. In IJCAI. 1477--1483.Google Scholar
- Diederik P. Kingma and Prafulla Dhariwal. 2018. Glow: Generative Flow with Invertible 1x1 Convolutions. In NIPS. 10236--10245. Google ScholarDigital Library
- Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR.Google Scholar
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2017. ImageNet Classification with Deep Convolutional Neural Networks. Commun. ACM, Vol. 60, 6 (2017), 84--90. Google ScholarDigital Library
- Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In ICLR.Google Scholar
- Dong-Hyun Lee. 2013. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In ICML.Google Scholar
- Qimai Li, Zhichao Han, and Xiao-Ming Wu. 2018. Deeper Insights Into Graph Convolutional Networks for Semi-Supervised Learning. In AAAI. 3538--3545.Google Scholar
- Xiao Liu, Fanjin Zhang, Zhenyu Hou, Zhaoyu Wang, Li Mian, Jing Zhang, and Jie Tang. 2020. Self-supervised Learning: Generative or Contrastive. CoRR, Vol. abs/2006.08218 (2020).Google Scholar
- Sina Mohseni, Mandar Pitale, J. B. S. Yadawa, and Zhangyang Wang. 2020. Self-Supervised Learning for Generalizable Out-of-Distribution Detection. In AAAI. 5216--5223.Google Scholar
- Andrew Y. Ng, Michael I. Jordan, and Yair Weiss. 2001. On Spectral Clustering: Analysis and an algorithm. In NIPS. 849--856. Google ScholarDigital Library
- A"aron van den Oord, Nal Kalchbrenner, Oriol Vinyals, Lasse Espeholt, Alex Graves, and Koray Kavukcuoglu. 2016. Conditional Image Generation with PixelCNN Decoders. In NIPS. 4790--4798. Google ScholarDigital Library
- Jiwoong Park, Minsik Lee, Hyung Jin Chang, Kyuewang Lee, and Jin Young Choi. 2019. Symmetric Graph Convolutional Autoencoder for Unsupervised Graph Representation Learning. In ICCV. 6518--6527.Google Scholar
- Zhen Peng, Wenbing Huang, Minnan Luo, Qinghua Zheng, Yu Rong, Tingyang Xu, and Junzhou Huang. 2020. Graph Representation Learning via Graphical Mutual Information Maximization. In WWW. 259--270. Google ScholarDigital Library
- Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: Online Learning of Social Representations. In SIGKDD. 701--710. Google ScholarDigital Library
- Gö zde Gü l Sahin and Mark Steedman. 2019. Data Augmentation via Dependency Tree Morphing for Low-Resource Languages. CoRR, Vol. abs/1903.09460 (2019).Google Scholar
- D. Sculley. 2010. Web-Scale k-Means Clustering. In WWW. 1177--1178. Google ScholarDigital Library
- Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Gü nnemann. 2018. Pitfalls of Graph Neural Network Evaluation. CoRR, Vol. abs/1811.05868 (2018).Google Scholar
- Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang. 2020a. InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization. In ICLR.Google Scholar
- Ke Sun, Zhouchen Lin, and Zhanxing Zhu. 2020b. Multi-Stage Self-Supervised Learning for Graph Convolutional Networks on Graphs with Few Labeled Nodes. In AAAI. 5892--5899.Google Scholar
- Yu Sun, Shuohuan Wang, Yu-Kun Li, Shikun Feng, Xuyi Chen, Han Zhang, Xin Tian, Danxiang Zhu, Hao Tian, and Hua Wu. 2019. ERNIE: Enhanced Representation through Knowledge Integration. CoRR, Vol. abs/1904.09223 (2019).Google Scholar
- Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang, and Zhong Su. 2008. ArnetMiner: Extraction and Mining of Academic Social Networks. In KDD. 990--998. Google ScholarDigital Library
- Yonglong Tian, Dilip Krishnan, and Phillip Isola. 2020. Contrastive Multiview Coding. In ECCV. 776--794.Google Scholar
- Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In ICLR.Google Scholar
- Petar Velickovic, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R. Devon Hjelm. 2019. Deep Graph Infomax. In ICLR.Google Scholar
- Hongwei Wang, Jia Wang, Jialin Wang, Miao Zhao, Weinan Zhang, Fuzheng Zhang, Xing Xie, and Minyi Guo. 2018. GraphGAN: Graph Representation Learning With Generative Adversarial Nets. In AAAI. 2508--2515.Google Scholar
- Xiao Wang, Meiqi Zhu, Deyu Bo, Peng Cui, Chuan Shi, and Jian Pei. 2020. AM-GCN: Adaptive Multi-Channel Graph Convolutional Networks. In KDD. 1243--1253.Google Scholar
- Felix Wu, Amauri H. Souza Jr., Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Q. Weinberger. 2019. Simplifying Graph Convolutional Networks. In ICML. 6861--6871.Google Scholar
- Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019. XLNet: Generalized Autoregressive Pretraining for Language Understanding. In NIPS. 5754--5764. Google ScholarDigital Library
- Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton, and Jure Leskovec. 2018. Graph Convolutional Neural Networks for Web-Scale Recommender Systems. In SIGKDD. 974--983. Google ScholarDigital Library
- Yuning You, Tianlong Chen, Zhangyang Wang, and Yang Shen. 2020. When Does Self-Supervision Help Graph Convolutional Networks?. In ICML. 10871--10880.Google Scholar
- Haoyu Zhang, Dingkun Long, Guangwei Xu, Muhua Zhu, Pengjun Xie, Fei Huang, and Ji Wang. 2020. Learning with Noise: Improving Distantly-Supervised Fine-grained Entity Typing via Automatic Relabeling. In IJCAI. 3808--3815.Google Scholar
- Muhan Zhang and Yixin Chen. 2018. Link Prediction Based on Graph Neural Networks. In NIPS. 5171--5181. Google ScholarDigital Library
- Amy Zhao, Guha Balakrishnan, Fré do Durand, John V. Guttag, and Adrian V. Dalca. 2019. Data Augmentation Using Learned Transformations for One-Shot Medical Image Segmentation. In CVPR. 8543--8553.Google Scholar
- Lingxiao Zhao and Leman Akoglu. 2020. PairNorm: Tackling Oversmoothing in GNNs. In ICLR.Google Scholar
Index Terms
- Rectifying Pseudo Labels: Iterative Feature Clustering for Graph Representation Learning
Recommendations
Multi-teacher Self-training for Semi-supervised Node Classification with Noisy Labels
MM '23: Proceedings of the 31st ACM International Conference on MultimediaGraph neural networks (GNNs) have achieved promising results for semi-supervised learning tasks on the graph-structured data. However, most existing methods assume that the training data are with correct labels, but in the real world, the graph-...
Semi-Supervised Deep Learning Using Pseudo Labels for Hyperspectral Image Classification
Deep learning has gained popularity in a variety of computer vision tasks. Recently, it has also been successfully applied for hyperspectral image classification tasks. Training deep neural networks, such as a convolutional neural network for ...
Collaborative Learning with Pseudo Labels for Robust Classification in the Presence of Noisy Labels
Computer Vision – ECCV 2020 WorkshopsAbstractSupervised learning depends on labels of dataset to train models with desired properties. Therefore, data containing mislabeled samples (a.k.a. noisy labels) can deteriorate supervised learning performance significantly as it makes models to be ...
Comments