Skip to main content
SpringerLink
Log in

Topological enhanced graph neural networks for semi-supervised node classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The complexity and non-Euclidean structure of graph data hinders the development of data augmentation methods similar to those in computer vision. In this paper, we propose a feature augmentation method for graph nodes based on topological regularization, in which topological structure information is introduced into an end-to-end model to promote better learning of node representation. Specifically, we first obtain topology embedding of nodes through Node2vec, an unsupervised graph feature learning method based on random walk. Then, the topological embedding as additional features and the original node features are input into a Symmetric Graph Neural Network framework for propagation, and two different high-order neighborhood representations of the nodes are obtained. On this basis, we propose a regularization technique to bridge the differences between the two different node representations, eliminate the adverse effects caused by the topological features of graphs directly used, and greatly improve the performance. Our framework can be effectively combined with other graph neural network models, and can effectively prevent over-smoothing of deep graph network. Experimental results on five datasets confirm the effectiveness of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Fout A, Byrd J, Shariat B, Ben-Hur A (2017) Protein interface prediction using graph convolutional networks. In: ADvances In Neural Information Processing Systems 30 (NIPS 2017) 6533–6542

  2. Do K, Tran T, Venkatesh S(2019) Graph transformation policy network for chemical reaction prediction. In:Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining pp. 750–760

  3. ZhangY, Chen X,Yang Y, Ramamurthy A, Li B, Qi Y, Song L (2020) Efficient probabilistic logic reasoning with graph neural networks. In: ICLR 2020 : Eighth International Conference on Learning Representations

  4. Xu X, Feng W, Jiang Y, Xie X, Sun Z,Deng ZH (2020) Dynamically pruned message passing networks for large-scale knowledge graph reasoning, ICLR

  5. Qi X, Liao R, Jia J, Fidler S, Urtasun R,(2017) 3d graph neural networks for rgbd semantic segmentation, ICCV 5209–5218

  6. Wang Y, Sun Y, Liu Z, Sarma ES, Bronstein MM, Solomon MJ (2019) Dynamic graph cnn for learning on point clouds, ACM Transactions on Graphics 1–12

  7. Yao L, Mao C, Luo Y (209) Graph convolutional networks for text classification. In: National conference on artificial intelligence

  8. Tu M, Wang G, Huang J, Tang Y, He X, Zhou B (2019) Multi-hop reading comprehension across multiple documents by reasoning over heterogeneous graphs. ACL 1:2704–2713

    Google Scholar 

  9. Qiu J, Tang J, Ma H, Dong Y, Wang K, Tang J (2018) Deepinf: Social influence prediction with deep learning, KDD 2110–2119

  10. Li C, Goldwasser D (2019) Encoding social information with graph convolutional networks forpolitical perspective detection in news media. ACL 1:2594–2604

    Google Scholar 

  11. Wu Y, Song Y, Huang H, Ye F, Xie X, Jin H (2021) Enhancing graph neural networks via auxiliary training for semi-supervised node classification. Knowledge Based Systems 220:106884

    Article  Google Scholar 

  12. Zhao A, Balakrishnan G, Durand F, Guttag VJ, Dalca VA (2019) Data augmentation using learned transforms for one-shot medical image segmentation, CVPR 8543–8553

  13. Cubuk DE, Zoph B, Mane D, Vasudevan V, Le VQ (2019) Autoaugment - learning augmentation strategies from data, CVPR 113–123

  14. Tong Z, Yozen L, Leonardo N, Oliver W, Meng J, Neil S (2021) Data augmentation for graph neural networks. In: Thirty-fifth aaai conference on artificial intelligence, thirty-third conference on innovative applications of artificial intelligence and the eleventh symposium on educational advances in artificial intelligence 11015–11023

  15. Rong Y, Huang W, Xu T, Huang J (2020) Dropedge: Towards deep graph convolutional networks on node classification, ICLR

  16. Chen D, Lin Y, Li W, Li P, Zhou J, Sun X (2020) Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. In: THirty-Fourth Aaai Conference On Artificial Intelligence, The Thirty-Second Innovative Applications Of Artificial Intelligence Conference And The Tenth Aaai Symposium On Educational Advances In Artificial Intelligence 3438–3445

  17. Luo D, Cheng W, Yu W, Zong B, Ni J, Chen H, Zhang X (2021) Learning to drop: Robust graph neural network via topological denoising, WSDM 779–787

  18. Feng W, Zhang J, Dong Y, Han Y, Luan H, Xu Q, Yang Q, Kharlamov E, Tang J (2020) Graph random neural networks for semi-supervised learning on graphs 33 22092–22103. https://proceedings.neurips.cc/paper/2020/file/fb4c835feb0a65cc39739320d7a51c02-Paper.pdfhttps://proceedings.neurips.cc/paper/2020/file/fb4c835feb0a65cc39739320d7a51c02-Paper.pdf

  19. Thda B, Dmna B, Gba B, Am A, Nda B (2021) Graph convolutional neural networks with node transition probability-based message passing and dropnode regularization. Expert Systems with Applications 174:114711

    Article  Google Scholar 

  20. Liu S, Ying R, Dong H, Li L, Xu T, Rong Y, Zhao P, Huang J, Wu D (2022) Local augmentation for graph neural networks. In: International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, Vol. 162 of Proceedings of Machine Learning Research pp. 14054–14072

  21. Lin Z, Tian C, Hou Y, Zhao WX (2022) Improving graph collaborative filtering with neighborhood-enriched contrastive learning. In: WWW ’22: The ACM Web Conference 2022, Virtual Event, Lyon, France, pp. 2320–2329 25-29 Apr 2022

  22. Xia L, Huang C, Xu Y, Zhao J, Yin D, Huang JX (2022) Hypergraph contrastive collaborative filtering. In: SIGIR ’22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain pp. 70–79 11-15 Jul 2022

  23. Buffelli D, Vandin F (2022) The impact of global structural information in graph neural networks applications, Vol. 7

  24. Jin Y, Song G, Shi C (2020) Gralsp: Graph neural networks with local structural patterns. Proceedings of the AAAI Conference on Artificial Intelligence 34:4361–4368

    Article  Google Scholar 

  25. Singh N, Singh A, Sharma R (2020) Predicting information cascade on twitter using random walk, Procedia Computer Science 201–209

  26. Kipf NT, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: International conference on learning representations

  27. Velickovic P, Cucurull G, Casanova A,Romero A, Lió Bengio PY (2018) Graph attention networks, ICLR

  28. Klicpera J, Bojchevski S (2019) Predict then propagate: Graph neural networks meet personalized pagerank, ICLR

  29. Chen M, Wei Z, Huang Z, Ding B, Li Y (2020) Simple and deep graph convolutional networks, ICML 1725–1735

  30. Gori M, Monfardini G, Scarselli F (2005) A new model for learning in graph domains. PRoceedings Of The International Joint Conference On Neural Networks (IJCNN) 1–5:729–734

    Google Scholar 

  31. Scarselli F, Tsoi CA, Gori M, Hagenbuchner M (2004) Graphical-based learning environments for pattern recognition, SSPR/SPR 42–56

  32. Bruna J, Zaremba W, Szlam A, LeCun Y (2014) Spectral networks and locally connected networks on graphs. In: International conference on learning representations

  33. Xu K, Hu W, Leskovec J, Jegelka S (2019) How powerful are graph neural networks?. In: International conference on learning representations (2019)

  34. Wu F, Souza HA, Zhang T, Fifty C, Yu T, Weinberger QK (2019) Simplifying graph convolutional networks. In: International conference on machine learning 6861–6871

  35. Pei H, Wei B, Chang CCK, Lei Y, Yang B (2020) Geom-gcn: Geometric graph convolutional networks, ICLR

  36. Zhu J, Rossi AR, Rao A, Mai T, Lipka N, Ahmed KN, Koutra D (2021) Graph neural networks with heterophily. In: THirty-Fifth Aaai Conference On Artificial Intelligence, Thirty-Third Conference On Innovative Applications Of Artificial Intelligence And The Eleventh Symposium On Educational Advances In Artificial Intelligence 11168–11176

  37. Sun K, Lin Z, Zhu Z (2021) Adagcn: Adaboosting graph convolutional networks into deep models, ICLR

  38. Zhao J, Wang X, Shi C, Hu B, Song G, Ye Y (2021) Heterogeneous graph structure learning for graph neural networks. In: THirty-Fifth Aaai Conference On Artificial Intelligence, Thirty-Third Conference On Innovative Applications Of Artificial Intelligence And The Eleventh Symposium On Educational Advances In Artificial Intelligence 4697–4705

  39. Xie Q, Dai Z, Hovy E, Luong T, Le Q (2020) Unsupervised data augmentation for consistency training 33,6256–6268.https://proceedings.neurips.cc/paper/2020/file/44feb0096faa8326192570788b38c1d1-Paper.pdfhttps://proceedings.neurips.cc/paper/2020/file/44feb0096faa8326192570788b38c1d1-Paper.pdf

  40. Zhao T, Zhang X, Wang S (2021) Graphsmote: Imbalanced node classification on graphs with graph neural networks, WSDM 833–841

  41. Abel R, Benami I,Louzoun Y (2019) Topological based classification using graph convolutional networks. arXiv:1911.06892

  42. Lov’asz L (1996) Random walks on graphs: A survey, Combinatorica

  43. Grover A, Leskovec J (2016) node2vec: Scalable feature learning for networks, KDD 855–864

  44. Li Q, Han Z, Wu XM (2018) Deeper insights into graph convolutional networks for semi-supervised learning, AAAI 3538–3545

  45. Liu M, Gao H, Ji S (2020) Towards deeper graph neural networks, KDD ’20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining Virtual Event CA USA July, 338–348

  46. Hamilton LW, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. In: Advances In Neural Information Processing Systems 30 (NIPS 2017) 1024–1034

  47. Zhang M, Cui Z, Neumann M, Chen Y (2018) An end-to-end deep learning architecture for graph classification. In: Thirty-second aaai conference on artificial intelligence / thirtieth innovative applications of artificial intelligence conference / eighth aaai symposium on educational advances in artificial intelligence 4438–4445

  48. McAuley J, Targett C, Shi Q, van den Hengel A (2015) Image-based recommendations on styles and substitutes. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval pp. 43–52

  49. Qu M, Bengio Y, Tang J (2019) Gmnn: Graph markov neural networks. International conference on machine learning

  50. Gao H, Ji S (2019) Graph u-nets, CoRR 2083–2092

  51. Maaten vdL, Hinton G (2008) Visualizing data using t-sne, JOURNAL OF MACHINE LEARNING RESEARCH 2579–2605

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (NSFC), "From Learning Outcome to Proactive Learning: Towards a Human-centered AI Based Approach to Intervention on Learning Motivation" (No.62077027) and the European Union’s Horizon 2020 FET Proactive project "WeNet - The Internet of us (No.823783)"

Author information

Authors and Affiliations

  1. School of Articial Intelligence, Jilin University, 130012, Changchun, China

    Rui Song

  2. Department of Information Engineering and Computer Science, University of Trento, Italy and School of Artificial Intelligence, Jilin University, 130012, Changchun, China

    Fausto Giunchiglia

  3. College of Software, Jilin University, 130012, Changchun, China

    Ke Zhao

  4. College of Computer Science and Technology, Key Laboratory of Symbolic Computing and Knowledge Engineering of Ministry of Education, Jilin University, 130012, Changchun, China

    Hao Xu

Authors
  1. Rui Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fausto Giunchiglia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ke Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hao Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hao Xu.

Ethics declarations

Conflicts of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Song, R., Giunchiglia, F., Zhao, K. et al. Topological enhanced graph neural networks for semi-supervised node classification. Appl Intell 53, 23538–23552 (2023). https://doi.org/10.1007/s10489-023-04739-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04739-6

Keywords

Search

Navigation