Abstract
Semi-supervised object classification has been a fundamental problem in relational data modeling recently. The problem has been extensively studied in the literature of graph neural networks (GNNs). Based on the homophily assumption, GNNs smooth the features of the adjacent nodes, resulting in hybrid class distributions in the feature space when the labeled nodes are scarce. Besides, the existing methods inherently suffer from the non-robustness, due to the deterministic propagation. To address the above two limitations, we propose a novel method Concordant Contrastive Learning (CCL) for semi-supervised node classification on graph. Specifically, we generate two group data augmentations by randomly masking node features and separately perform node feature propagation with low- and high-order graph topology information. Further, we design two granularity regularization losses. The coarse-grained regularization loss (i.e., center-level contrastive loss) preserves the identity of each class against the rest, which benefits to guide the discriminative class distributions. The fine-grained regularization loss (i.e., instance-level contrastive loss) enforces consistency between soft assignments for different augmentations of the same node. Extensive experiments on different benchmark datasets imply that CCL significantly outperforms a wide range of state-of-the-art baselines on the task of semi-supervised node classification.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abu-El-Haija, S., et al.: MixHop: higher-order graph convolutional architectures via sparsified neighborhood mixing. In: Proceedings of the International Conference on Machine Learning (ICML) (2019)
Berthelot, D., et al.: ReMixMatch: semi-supervised learning with distribution alignment and augmentation anchoring. arXiv preprint arXiv:1911.09785 (2019)
Bruna, J., Zaremba, W., Szlam, A., LeCun, Y.: Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203 (2013)
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., Joulin, A.: Unsupervised learning of visual features by contrasting cluster assignments. In: Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS) (2020)
Chen, J., Yang, Z., Yang, D.: MixText: linguistically-informed interpolation of hidden space for semi-supervised text classification. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL) (2020)
Chen, M., Wei, Z., Huang, Z., Ding, B., Li, Y.: Simple and deep graph convolutional networks. In: Proceedings of the International Conference on Machine Learning (ICML) (2020)
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: Proceedings of the International Conference on Machine Learning (ICML) (2020)
Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering. In: Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS) (2016)
Feng, W., et al.: Graph random neural networks for semi-supervised learning on graphs. In: Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS) (2020)
Hamilton, W.L., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS) (2017)
Hassani, K., Khasahmadi, A.H.: Contrastive multi-view representation learning on graphs. In: Proceedings of the International Conference on Machine Learning (ICML) (2020)
He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Jin, W., Derr, T., Wang, Y., Ma, Y., Liu, Z., Tang, J.: Node similarity preserving graph convolutional networks. In: Proceedings of the ACM International Conference on Web Search and Data Mining (WSDM) (2021)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: Proceedings of the International Conference on Learning Representations (ICLR) (2017)
Klicpera, J., Bojchevski, A., Günnemann, S.: Predict then propagate: graph neural networks meet personalized PageRank. In: Proceedings of the International Conference on Learning Representations (ICLR) (2019)
Lam, W., Keung, C.K., Liu, D.: Discovering useful concept prototypes for classification based on filtering and abstraction. IEEE Trans. Pattern Anal. Mach. Intell. 24(8), 1075–1090 (2002)
Luo, F., Yang, P., Li, S., Ren, X., Sun, X.: CAPT: contrastive pre-training for learning denoised sequence representations. arXiv preprint arXiv:2010.06351 (2020)
Luo, X., et al.: CIMON: towards high-quality hash codes. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) (2021)
van den Oord, A., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)
Qiu, J., et al.: GCC: graph contrastive coding for graph neural network pre-training. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2020)
Qu, M., Bengio, Y., Tang, J.: GMNN: graph Markov neural networks. In: Proceedings of the International Conference on Machine Learning (ICML) (2019)
Qu, Y., Shen, D., Shen, Y., Sajeev, S., Han, J., Chen, W.: CoDA: contrast-enhanced and diversity-promoting data augmentation for natural language understanding. arXiv preprint arXiv:2010.08670 (2020)
Rong, Y., Huang, W., Xu, T., Huang, J.: DropEdge: towards deep graph convolutional networks on node classification. In: Proceedings of the International Conference on Learning Representations (ICLR) (2019)
Sen, P., Namata, G., Bilgic, M., Getoor, L., Galligher, B., Eliassi-Rad, T.: Collective classification in network data. AI Mag. 29(3), 93–93 (2008)
Sohn, K., et al.: FixMatch: simplifying semi-supervised learning with consistency and confidence. In: Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS) (2020)
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. In: Proceedings of the International Conference on Learning Representations (ICLR) (2018)
Velickovic, P., Fedus, W., Hamilton, W.L., Liò, P., Bengio, Y., Hjelm, R.D.: Deep graph infomax. In: Proceedings of the International Conference on Learning Representations (ICLR) (2019)
Wan, S., Pan, S., Yang, J., Gong, C.: Contrastive and generative graph convolutional networks for graph-based semi-supervised learning. arXiv preprint arXiv:2009.07111 (2020)
Wu, F., Souza, A., Zhang, T., Fifty, C., Yu, T., Weinberger, K.: Simplifying graph convolutional networks. In: Proceedings of the International Conference on Machine Learning (ICML) (2019)
You, Y., Chen, T., Sui, Y., Chen, T., Wang, Z., Shen, Y.: Graph contrastive learning with augmentations. In: Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS) (2020)
Zeng, J., Xie, P.: Contrastive self-supervised learning for graph classification. In: Proceedings of the AAAI Conference on Artificial Intelligence (2021)
Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., Wang, L.: Graph contrastive learning with adaptive augmentation. In: Proceedings of the World Wide Web Conference (WWW) (2021)
Zügner, D., Günnemann, S.: Adversarial attacks on graph neural networks via meta learning. In: Proceedings of the International Conference on Learning Representations (ICLR) (2018)
Acknowledgments
This work was partially supported by the National Key Research and Development Program of China under grant 2018AAA0100205.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Wu, D., Luo, X., Guo, X., Chen, C., Deng, M., Ma, J. (2021). Concordant Contrastive Learning for Semi-supervised Node Classification on Graph. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Lecture Notes in Computer Science(), vol 13108. Springer, Cham. https://doi.org/10.1007/978-3-030-92185-9_48
Download citation
DOI: https://doi.org/10.1007/978-3-030-92185-9_48
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92184-2
Online ISBN: 978-3-030-92185-9
eBook Packages: Computer ScienceComputer Science (R0)