Abstract
Graph Neural Networks (GNNs) have received much attention in the graph deep learning. However, there are some issues in extending traditional aggregation-based GNNs to large-scale graphs. With the rapid increase of neighborhood width, we find that the direction of aggregation can be disrupted and quite unbalanced, which compromises graphic structure and feature representation. This phenomenon is referred to Receptive Field Collapse. In order to preserve more structural information on large-scale graphs, we propose a novel Global Variational Convolutional Networks (GVCNs) for Semi-Supervised Node Classifications, which consists of a variational aggregation mechanism and a guidance learning mechanism. Variational aggregation can moderately map the unbalanced neighborhood distribution to a prior distribution. And the guidance learning mechanism, based on positive pointwise mutual information (PPMI), encourages the model to concentrate on more prominent graphic structures, which increases information entropy and alleviates Receptive Field Collapse. In addition, we propose a variational convolutional kernel to achieve effective global aggregation. Finally, we evaluate GVCNs on the Open Graph Benchmark (OGB) Arxiv and Products datasets. Up to the submission date (Jan 20, 2023), GVCNs achieve significant performance improvements compared to other aggregation-based GNNs, even state-of-the-art decoupling-based methods, the performance of GVCNs remains competitive with moderate spatiotemporal complexity. Our code can be obtained from: https://github.com/Yide-Qiu/GVCN.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Battaglia, P.W., et al.: Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261 (2018)
van den Berg, R., Kipf, T.N., Welling, M.: Graph convolutional matrix completion. arXiv preprint arXiv:1706.02263 (2017)
Bouma, G.: Normalized (pointwise) mutual information in collocation extraction. Proc. GSCL 30, 31–40 (2009)
Chen, J., Ma, T., Xiao, C.: FastGCN: fast learning with graph convolutional networks via importance sampling. arXiv preprint arXiv:1801.10247 (2018)
Chiang, W.-L., Liu, X., Si, S., Li, Y., Bengio, S., Hsieh, C.-J.: Cluster-GCN: an efficient algorithm for training deep and large graph convolutional networks. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 257–266 (2019)
Carl Doersch. Tutorial on variational autoencoders. arXiv preprint arXiv:1606.05908 (2016)
Duan, K., et al.: A comprehensive study on large-scale graph training: benchmarking and rethinking. arXiv preprint arXiv:2210.07494 (2022)
Fout, A., Byrd, J., Shariat, B., Ben-Hur, A.: Protein interface prediction using graph convolutional networks. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Frasca, F., Rossi, E., Eynard, D., Chamberlain, B., Bronstein, M., Monti, F.: Sign: scalable inception graph neural networks. arXiv preprint arXiv:2004.11198 (2020)
Grover, A., Leskovec, J.: node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 855–864 (2016)
Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Weihua, H., et al.: Open graph benchmark: datasets for machine learning on graphs. In: Advances in Neural Information Processing Systems, vol. 33, pp. 22118–22133 (2020)
Huang, Q., He, H., Singh, A., Lim, S.-N., Benson, A.R.: Combining label propagation and simple models out-performs graph neural networks. arXiv preprint arXiv:2010.13993 (2020)
Joyce, J.M.: Kullback-leibler divergence. In: Lovric, M. (ed.) International Encyclopedia of Statistical Science, pp. 720–722. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-04898-2_327
Kipf, T.N., Welling, M.: Variational graph auto-encoders. arXiv preprint arXiv:1611.07308 (2016)
Li, G., Xiong, C., Thabet, A., Ghanem, B.: DeeperGCN: all you need to train deeper GCNs. arXiv preprint arXiv:2006.07739 (2020)
Liang, D., Krishnan, R.G., Hoffman, M.D., Jebara, T.: Variational autoencoders for collaborative filtering. In: Proceedings of the 2018 World Wide Web Conference, pp. 689–698 (2018)
Sun, C., Gu, H., Hu, J.: Scalable and adaptive graph neural networks with self-label-enhanced training. arXiv preprint arXiv:2104.09376 (2021)
Sun, C., Wu, G.: Adaptive graph diffusion networks with hop-wise attention. arXiv e-prints, p. arXiv–2012 (2020)
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)
Wu, F., Souza, A., Zhang, T., Fifty, C., Yu, T., Weinberger, K.: Simplifying graph convolutional networks. In: International Conference on Machine Learning, pp. 6861–6871. PMLR (2019)
Xu, K., Li, C., Tian, Y., Sonobe, T., Kawarabayashi, K., Jegelka, S.: Representation learning on graphs with jumping knowledge networks. In: International Conference on Machine Learning, pp. 5453–5462. PMLR (2018)
Le, Yu., Bowen, D., Xiao, H., Sun, L., Han, L., Lv, W.: Deep spatio-temporal graph convolutional network for traffic accident prediction. Neurocomputing 423, 135–147 (2021)
Zeng, H., Zhou, H., Srivastava, A., Kannan, R., Prasanna, V.: Graphsaint: graph sampling based inductive learning method. arXiv preprint arXiv:1907.04931 (2019)
Zhang, W., et al.: Graph attention multi-layer perceptron. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 4560–4570 (2022)
Zheng, C., et al.: ByteGNN: efficient graph neural network training at large scale. Proc. VLDB Endowment 15(6), 1228–1242 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Qiu, Y., Zhang, T., Huang, B., Cui, Z. (2024). Global Variational Convolution Network for Semi-supervised Node Classification on Large-Scale Graphs. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14432. Springer, Singapore. https://doi.org/10.1007/978-981-99-8543-2_16
Download citation
DOI: https://doi.org/10.1007/978-981-99-8543-2_16
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8542-5
Online ISBN: 978-981-99-8543-2
eBook Packages: Computer ScienceComputer Science (R0)