Skip to main content
SpringerLink
Log in

Node Similarity Preserving Graph Convolutional Network Based on Full-frequency Information for Node Classification

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Recently, graph neural networks have achieved good performance in graph representation learning. However, most graph neural networks only utilize node low-frequency signals and destroy node similarity when aggregating graph structure and node features, which limits their ability to represent graph-structured data. Therefore, we propose a node similarity preserving graph convolutional network based on full-frequency information (FSP-GCN). It extracts relevant information to the greatest extent from graph structure and node features while preserving node similarity for aggregation. Precisely, to better capture full-frequency information, we propose an improved aggregation component called MFGCN that adopts a multi-head attention mechanism to integrate signals in different frequency domains adaptively. Then we design a node similarity aggregation method to aggregate a k-nearest neighbor(kNN) graph constructed from the original graph with the feature graph learned from the MFGCN aggregation component to preserve node similarity. Finally, we employ contrastive learning to preserve node similarity further. We also compare the performance of the FSP-GCN model with that of ten real-world networks, using well-known assortative and disassortative datasets. The results demonstrate that FSP-GCN offers significant performance improvement compared to representative baselines. In addition, extensive comparative experiments are conducted on the robustness of the model and alleviating the over-smoothing problem. The results show that the FSP-GCN model can effectively alleviate the over-smoothing problem and resist the adversarial attack of the graph structure.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Tang J, Sun J, Wang C, Yang, Z (2009) Social influence analysis in large-scale networks. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD ’09, pp 807–816. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/1557019.1557108

  2. Wu Z, Pan S, Chen F, Long G, Zhang C, Yu PS (2021) A comprehensive survey on graph neural networks. IEEE Trans Neural Netw Learn Syst 32(1):4–24. https://doi.org/10.1109/TNNLS.2020.2978386

    Article  MathSciNet  Google Scholar 

  3. Liu M, Wang Z, Ji S (2021) Non-local graph neural networks. IEEE Trans Pattern Anal Mach Intell 1:1. https://doi.org/10.1109/tpami.2021.3134200

    Article  Google Scholar 

  4. Zhang M, Chen Y (2018) Link prediction based on graph neural networks. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. NIPS’18, pp 5171–5181. Curran Associates Inc., Red Hook, NY, USA. https://doi.org/10.48550/arXiv.1802.09691

  5. Knyazev B, Taylor GW, Amer MR (2019) Understanding attention and generalization in graph neural networks. Curran Associates Inc., Red Hook

    Google Scholar 

  6. Yao L, Mao C, Luo Y (2019) Graph convolutional networks for text classification. Proc AAAI Conf Artif Intell 33(01):7370–7377. https://doi.org/10.1609/aaai.v33i01.33017370

    Article  Google Scholar 

  7. He X, Deng K, Wang X, Li Y, Zhang Y, Wang M (2020) LightGCN: simplifying and powering graph convolution network for recommendation, pp 639–648. association for computing machinery, New York, NY, USA. https://doi.org/10.1145/3397271.3401063

  8. Zeng F, Chen N, Yang D (2022) Simplified-boosting ensemble convolutional network for text classification. Neural Process Lett. https://doi.org/10.1007/s11063-022-10843-4

    Article  Google Scholar 

  9. Bongini P, Bianchini M, Scarselli F (2021) Molecular generative graph neural networks for drug discovery. Neurocomputing 450:242–252. https://doi.org/10.1016/j.neucom.2021.04.039

    Article  Google Scholar 

  10. Jin D, Liu Z, Li W, He D, Zhang W (2019) Graph convolutional networks meet markov random fields: Semi-supervised community detection in attribute networks. Proc AAAI Conf Artif Intell 33(01):152–159. https://doi.org/10.1609/aaai.v33i01.3301152

    Article  Google Scholar 

  11. Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. NIPS’16, pp 3844–3852, Red Hook, NY, USA

  12. Hamilton WL, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. NIPS’17, pp 1025–1035, Red Hook, NY, USA

  13. Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. https://doi.org/10.48550/arXiv.1609.02907

  14. Gilmer J, Schoenholz SS, Riley PF, Vinyals O, Dahl GE (2017) Neural message passing for quantum chemistry. In: Proceedings of the 34th International Conference on Machine Learning - Volume 70. ICML’17, pp 1263–1272. JMLR.org, ???

  15. Wu F, Souza A, Zhang T, Fifty C, Yu T, Weinberger K (2019) Simplifying graph convolutional networks. In: International Conference on Machine Learning, pp 6861–6871. PMLR

  16. Li Q, Wu X-M, Liu H, Zhang X, Guan Z (2019) Label efficient semi-supervised learning via graph filtering. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

  17. Bruna J, Zaremba W, Szlam A, Lecun Y (2014) Spectral networks and locally connected networks on graphs. In: International Conference on Learning Representations (ICLR2014), CBLS, April 2014

  18. Xu B, Shen H, Cao Q, Cen K, Cheng X (2020) Graph convolutional networks using heat kernel for semi-supervised learning. https://doi.org/10.48550/ARXIV.2007.16002

  19. Ribeiro LFR, Saverese PHP, Figueiredo DR (2017) struc2vec: Learning node representations from structural identity. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD ’17, pp 385–394. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3097983.3098061

  20. Newman MEJ (2003) Mixing patterns in networks. Phys Rev E 67:026126. https://doi.org/10.1103/PhysRevE.67.026126

    Article  MathSciNet  Google Scholar 

  21. Zhu J, Yan Y, Zhao L, Heimann M, Akoglu L, Koutra D (2020) Beyond homophily in graph neural networks: Current limitations and effective designs. NIPS’20, vol 33, pp 7793–7804. https://doi.org/10.48550/arXiv.2006.11468

  22. Zügner D, Akbarnejad A, Günnemann S (2018) Adversarial attacks on neural networks for graph data. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD ’18, pp 2847–2856. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3219819.3220078

  23. Jin W, Derr T, Wang Y, Ma Y, Liu Z, Tang J (2021) Node similarity preserving graph convolutional networks. In: Proceedings of the 14th ACM International Conference on Web Search and Data Mining. WSDM ’21, pp 148–156. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3437963.3441735

  24. Bo D, Wang X, Shi C, Shen H (2021) Beyond low-frequency information in graph convolutional networks. Proceedings of the AAAI Conference on Artificial Intelligence 35(5):3950–3957. https://doi.org/10.48550/arXiv.2101.00797

  25. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30:541 arXiv:1706.03762

  26. Bruna J, Zaremba W, Szlam A, LeCun Y (2013) Spectral networks and locally connected networks on graphs. https://doi.org/10.48550/arXiv.1312.6203

  27. Xu B, Cen K, Huang J, Shen H, Cheng X (2020) A survey on graph convolutional neural network. Chin J Computers 43(5):26. https://doi.org/10.11897/SP.J.1016.2020.00755

    Article  Google Scholar 

  28. Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2017) Graph attention networks. https://doi.org/10.48550/ARXIV.1710.10903

  29. Wu Z, Lin Y, Wang J, Gregory S (2016) Link prediction with node clustering coefficient. Physica A: Stat Mech Appl 452(C):1–8. https://doi.org/10.5555/3327345.3327423

    Article  Google Scholar 

  30. Cai H, Zheng VW, Chang KC-C (2018) A comprehensive survey of graph embedding: Problems, techniques, and applications. IEEE Trans Knowl Data Eng 30(9):1616–1637. https://doi.org/10.1109/TKDE.2018.2807452

    Article  Google Scholar 

  31. Coşkun M, Koyutürk M (2021) Node similarity-based graph convolution for link prediction in biological networks. Bioinformatics 37(23):4501–4508. https://doi.org/10.1093/bioinformatics/btab464

    Article  Google Scholar 

  32. Mnih V, Heess N, Graves A, kavukcuoglu k (2014) Recurrent models of visual attention. In: Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2. NIPS’14, vol 27, pp 2204–2212. MIT Press, Cambridge, MA, USA. https://doi.org/10.5555/2969033.2969073

  33. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. https://doi.org/10.48550/ARXIV.1409.0473

  34. Yin W, Schütze H, Xiang B, Zhou B (2016) Abcnn: attention-based convolutional neural network for modeling sentence pairs. Trans Assoc Comput Linguistics 4:259–272. https://doi.org/10.48550/arXiv.1512.05193

    Article  Google Scholar 

  35. Zhang L, Song H, Aletras N, Lu H (2022) Node-feature convolution for graph convolutional networks. Pattern Recognit 128:108661. https://doi.org/10.1016/j.patcog.2022.108661

    Article  Google Scholar 

  36. Cui G, Zhou JSGCN, Yang C, Liu Z (2020) Adaptive graph encoder for attributed graph embedding. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD ’20, pp 976–985. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3394486.3403140

  37. Leskovec KXWHJ, Jegelka S (2019) How powerful are graph neural networks. ICLR. Keyulu Xu Weihua Hu Jure Leskovec and Stefanie Jegelka

  38. Dong W, Charikar M, Li K (2011) Efficient k-nearest neighbor graph construction for generic similarity measures, pp 577–586. https://doi.org/10.1145/1963405.1963487

  39. Pei H, Wei B, Chang KC-C, Lei Y, Yang B (2020) Geom-gcn: Geometric graph convolutional networks. In: International Conference on Learning Representations. https://doi.org/10.1109/IJCNN48605.2020.9207639

  40. Zhu X, Ghahramani Z, Lafferty JD (2003) Semi-supervised learning using gaussian fields and harmonic functions. In: Proceedings of the 20th International Conference on Machine Learning (ICML-03), pp. 912–919

  41. Xu K, Li C, Tian Y, Sonobe T, Kawarabayashi K, Jegelka S (2018) Representation learning on graphs with jumping knowledge networks. Proceedings of the 35th International Conference on Machine Learning, 5453–5462. https://doi.org/10.48550/arXiv.1806.03536. PMLR

  42. Chen M, Wei Z, Huang Z, Ding B, Li Y (2020) Simple and deep graph convolutional networks, 1725–1735. https://doi.org/10.48550/arXiv.2007.02133. PMLR

  43. Klicpera J, Bojchevski A, Günnemann S (2018) Predict then propagate: graph neural networks meet personalized pagerank. http://arxiv.org/abs/1810.05997

  44. Chien E, Peng J, Li P, Milenkovic, O (2020) Adaptive universal generalized pagerank graph neural network. http://arxiv.org/abs/2006.07988

  45. Zügner D, Günnemann S (2019) Adversarial attacks on graph neural networks via meta learning. ACM Trans Knowl Discov Data 14:5. https://doi.org/10.1145/3394520

    Article  Google Scholar 

  46. Wu H, Wang C, Tyshetskiy Y, Docherty A, Lu K, Zhu L (2019) Adversarial examples for graph data: deep insights into attack and defense. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp 4816–4823. https://doi.org/10.24963/ijcai.2019/669

Download references

Author information

Author notes

  1. J. Liao and C. Liu are contributed equally to this work.

Authors and Affiliations

  1. Computer Science and Technology, Wuhan University of Technology, Peace Avenue, WuHan, 430063, Hubei, China

    Yuqiang Li, Jing Liao, Chun Liu & Lin Li

  2. School of Mechanical Engineering, Wuhan Polytechnic University, Huanhu Middle Road, WuHan, 430048, Hubei, China

    YingJie Wang

Authors
  1. Yuqiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jing Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. YingJie Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chun Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Liao, J., Liu, C. et al. Node Similarity Preserving Graph Convolutional Network Based on Full-frequency Information for Node Classification. Neural Process Lett 55, 5473–5498 (2023). https://doi.org/10.1007/s11063-022-11094-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-11094-z

Keywords

Search

Navigation