Skip to main content
SpringerLink
Log in

Graph alternate learning for robust graph neural networks in node classification

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The real-world graphs are full of noises and perturbation. However, recent studies show that the existing graph neural networks (GNNs) are usually sensitive to the quality of the input graph. In this work, we propose a graph alternate learning (GAL) framework to alternately train dual models, i.e., prediction network to learn the graph structure for node classification tasks and graph regularization network to enhance the robustness of GNNs. The adoption of dual models, which learn and teach each other collaboratively at the entire training process, drives the formation of a better graph structure. Furthermore, a node feature selection method is integrated into the GAL network to reduce the influence of node attack. Lastly, in order to evaluate the anti-attack ability of GAL, we devise a smooth input graph adversarial attack, called Smooth-Attack, which can degrade the node classification performance of graph convolutional networks (GCN) and is considered to be a form of over-smoothing. Experiments show that our proposed GAL model can keep superiority on benchmark datasets under edge and node perturbation, and GAL is highly robust against Smooth-Attack.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Hamilton WL, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. arXiv preprint arXiv:1706.02216

  2. Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907

  3. Sanchez-Gonzalez A, Heess N, Springenberg JT, Merel J, Riedmiller M, Hadsell R, Battaglia P (2018) Graph networks as learnable physics engines for inference and control. In: International conference on machine learning, PMLR, pp 4470–4479

  4. Fout AM (2017) Protein interface prediction using graph convolutional networks. PhD thesis, Colorado State University

  5. Su X, Xue S, Liu F, Wu J, Yang J, Zhou C, Hu W, Paris C, Nepal S, Jin D, et al. (2021) A comprehensive survey on community detection with deep learning. arXiv preprint arXiv:2105.12584

  6. Hamaguchi T, Oiwa H, Shimbo M, Matsumoto Y (2017) Knowledge transfer for out-of-knowledge-base entities: a graph neural network approach. arXiv preprint arXiv:1706.05674

  7. Li Y, Tarlow D, Brockschmidt M, Zemel R (2015) Gated graph sequence neural networks. arXiv preprint arXiv:1511.05493

  8. Monti F, Frasca F, Eynard D, Mannion D, Bronstein MM (2019) Fake news detection on social media using geometric deep learning. arXiv preprint arXiv:1902.06673

  9. Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2017) Graph attention networks. arXiv preprint arXiv:1710.10903

  10. Kipf TN, Welling M (2016) Variational graph auto-encoders. arXiv preprint arXiv:1611.07308

  11. Goodfellow IJ, Shlens J, Szegedy C (2014) Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572

  12. Xu H, Ma Y, Liu HC, Deb D, Liu H, Tang JL, Jain AK (2020) Adversarial attacks and defenses in images, graphs and text: a review. Int J Autom Comput 17(2):151–178

    Article  Google Scholar 

  13. Zügner D, Akbarnejad A, Günnemann S (2018) Adversarial attacks on neural networks for graph data. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 2847–2856

  14. Zügner D, Günnemann S (2019) Adversarial attacks on graph neural networks via meta learning. arXiv preprint arXiv:1902.08412

  15. Tang J, Liu H (2012) Feature selection with linked data in social media. In: Proceedings of the 2012 SIAM international conference on data mining. SIAM, pp 118–128

  16. Dai H, Li H, Tian T, Huang X, Wang L, Zhu J, Song L (2018) Adversarial attack on graph structured data. In: International conference on machine learning, PMLR, pp 1115–1124

  17. Akoglu L, Tong H, Koutra D (2015) Graph based anomaly detection and description: a survey. Data Min Knowl Discov 29(3):626–688

    Article  MathSciNet  Google Scholar 

  18. Zhang Y, Xiang T, Hospedales TM, Lu H (2018) Deep mutual learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4320–4328

  19. Zhan K, Niu C (2021) Mutual teaching for graph convolutional networks. Future Gener Comput Syst 115:837–843

    Article  Google Scholar 

  20. Hinton G, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531

  21. Gori M, Monfardini G, Scarselli F (2005) A new model for learning in graph domains. In: Proceedings. 2005 IEEE international joint conference on neural networks, vol 2, 2005. IEEE, pp 729–734

  22. Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G (2008) The graph neural network model. IEEE Trans Neural Netw 20(1):61–80

    Article  Google Scholar 

  23. Gallicchio C, Micheli A (2010) Graph echo state networks. In: The 2010 International joint conference on neural networks (IJCNN), IEEE, pp 1–8

  24. Bruna J, Zaremba W, Szlam A, LeCun Y (2013) Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203

  25. Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. arXiv preprint arXiv:1606.09375

  26. Wu F, Souza A, Zhang T, Fifty C, Yu T, Weinberger K (2019) Simplifying graph convolutional networks. In: International conference on machine learning, PMLR, pp 6861–6871

  27. Atwood J, Towsley D (2016) Diffusion-convolutional neural networks. In: Advances in neural information processing systems, pp 1993–2001

  28. Luo D, Cheng W, Yu W, Zong B, Ni J, Chen H, Zhang X (2021) Learning to drop: Robust graph neural network via topological denoising. In: Proceedings of the 14th ACM international conference on web search and data mining, pp 779–787

  29. Jin W, Li Y, Xu H, Wang Y, Tang J (2020) Adversarial attacks and defenses on graphs: a review and empirical study. arXiv preprint arXiv:2003.00653

  30. Tang X, Li Y, Sun Y, Yao H, Mitra P, Wang S (2020) Transferring robustness for graph neural network against poisoning attacks. In: Proceedings of the 13th international conference on web search and data mining, pp 600–608

  31. Jin W, Ma Y, Liu X, Tang X, Wang S, Tang J (2020) Graph structure learning for robust graph neural networks. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp 66–74

  32. Zhu D, Zhang Z, Cui P, Zhu W (2019) Robust graph convolutional networks against adversarial attacks. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1399–1407

  33. Cai X, Nie F, Huang H (2013) Exact top-k feature selection via l2, 0-norm constraint. In: Twenty-third international joint conference on artificial intelligence

  34. Jin W, Derr T, Wang Y, Ma Y, Liu Z, Tang J (2021) Node similarity preserving graph convolutional networks. In: Proceedings of the 14th ACM international conference on web search and data mining, pp 148–156

  35. Chen Y, Wu L, Zaki M (2020) Iterative deep graph learning for graph neural networks: better and robust node embeddings. Adv Neural Inf Process Syst 33

  36. Belkin M, Niyogi P (2001) Laplacian eigenmaps and spectral techniques for embedding and clustering. Nips 14:585–591

    Google Scholar 

  37. Zhou K, Zha H, Song L (2013) Learning social infectivity in sparse low-rank networks using multi-dimensional hawkes processes. In: Artificial intelligence and statistics, PMLR, pp 641–649

  38. Kalofolias V (2016) How to learn a graph from smooth signals. In: Artificial intelligence and statistics, PMLR, pp 920–929

  39. Entezari N, Al-Sayouri SA, Darvishzadeh A, Papalexakis EE (2020) All you need is low (rank) defending against adversarial attacks on graphs. In: Proceedings of the 13th international conference on web search and data mining, pp 169–177

  40. Liu M, Gao H, Ji S (2020) Towards deeper graph neural networks. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp 338–348

  41. McCallum AK, Nigam K, Rennie J, Seymore K (2000) Automating the construction of internet portals with machine learning. Inf Retr 3(2):127–163

    Article  Google Scholar 

  42. Sen P, Namata G, Bilgic M, Getoor L, Galligher B, Eliassi-Rad T (2008) Collective classification in network data. AI Mag 29(3):93–93

    Google Scholar 

  43. Adamic LA, Glance N (2005) The political blogosphere and the 2004. US Elec

  44. Hu W, Fey M, Zitnik M, Dong Y, Ren H, Liu B, Catasta M, Leskovec J (2020) Open graph benchmark: datasets for machine learning on graphs. arXiv preprint arXiv:2005.00687

  45. Zhang X, Zitnik M (2020) Gnnguard: Defending graph neural networks against adversarial attacks. arXiv preprint arXiv:2006.08149

  46. Macskassy SA, Provost F (2003) A simple relational classifier. New York Univ NY Stern School of Business, Tech. rep

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China under Grant 82071995 and Natural Science Foundation of Jilin Province under Grant 20200201292JC.

Author information

Authors and Affiliations

  1. Key Laboratory of Symbol Computation and Knowledge Engineering of Ministry of Education, College of Computer Science and Technology, Jilin University, Changchun, 130012, China

    Baoliang Zhang, Xiaoxin Guo, Zhenchuan Tu & Jia Zhang

Authors
  1. Baoliang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoxin Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhenchuan Tu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jia Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoxin Guo.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, B., Guo, X., Tu, Z. et al. Graph alternate learning for robust graph neural networks in node classification. Neural Comput & Applic 34, 8723–8735 (2022). https://doi.org/10.1007/s00521-021-06863-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06863-1

Keywords

Search

Navigation