Skip to main content
SpringerLink
Log in

Hybrid deep graph convolutional networks

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Graph neural networks (GNNs) leverage graph convolutions or their approximations to cope with graph-structured data. According to whether convolution is applied to the spectral domain or spatial domain, advances in this direction are generally divided into spectral approaches and spatial approaches. However, current study generally utilizes one of the approaches, while ignoring the combination of the two approaches. In this paper, we propose hybrid deep graph convolutional networks (HDGCNs), novel neural network architectures that combine the spectral approach and the spatial approach to calculate the adjacency matrix, leveraging the deep graph convolutional networks (GCNs) to release the advantages of this combination. In this way, the gap between spectral approaches and spatial approaches is eliminated, and the characteristics of the two approaches are merged together. Extensive experiments on citation networks and web networks offer evidence that the proposed models outperform state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR)

  2. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, Las Vegas, pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  3. Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. In: 2018 IEEE/CVF conference on computer vision and pattern recognition. IEEE, Salt Lake City, pp 7794–7803. https://doi.org/10.1109/CVPR.2018.00813

  4. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: 2018 IEEE/CVF conference on computer vision and pattern recognition. IEEE, Salt Lake City, pp 7132–7141

  5. Bahdanau D, Cho K, Bengio Y (2015) Neural machine translation by jointly learning to align and translate. In: 3th International conference on learning representations

  6. Gehring J, Auli M, Grangier D, Dauphin Y (2017) A convolutional encoder model for neural machine translation. In: Proceedings of the 55th annual meeting of the association for computational linguistics, association for computational linguistics, Vancouver, Canada, pp 123–135. https://doi.org/10.18653/v1/P17-1012

  7. Gori M, Monfardini G, Scarselli F (2005) A new model for learning in graph domains. In: Proceedings. 2005 IEEE international joint conference on neural networks, 2005, vol 2. IEEE, Montreal, pp 729–734. https://doi.org/10.1109/IJCNN.2005.1555942

  8. Scarselli F, Gori M, Tsoi Ah Chung, Hagenbuchner M, Monfardini G (2009) The graph neural network model. IEEE Trans Neural Netw 20(1):61–80. https://doi.org/10.1109/TNN.2008.2005605

    Article  Google Scholar 

  9. Li Y, Tarlow D, Brockschmidt M, Zemel R (2016) Gated graph sequence neural networks. In: 4th International conference on learning representations

  10. Bruna J, Zaremba W, Szlam A, LeCun Y (2014) Spectral networks and locally connected networks on graphs. In: 2th International conference on learning representations

  11. Atwood J, Towsley D (2015) Deep convolutional networks on graph-structured data. arXiv preprint. arXiv:1506.05163

  12. Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. In: Advances in neural information processing systems, pp 3844–3852

  13. Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: 5th International conference on learning representations

  14. Wu F, Zhang T (2019) Simplifying graph convolutional networks. In: Proceedings of the 36th international conference on machine learning, vol 97, pp 6861–6871

  15. Klicpera J, Bojchevski A, Günnemann S (2019) Predict then propagate: graph neural networks meet personalized PageRank. In: 7th International conference on learning representations

  16. Klicpera J, Weißenberger S, Günnemann S (2019) Diffusion improves graph learning. In: Advances in neural information processing systems

  17. Page L, Brin S, Motwani R, Winograd T (1999) The Pagerank citation ranking: bringing order to the web. In: Proceedings of the WWW: the web conference

  18. Duvenaud D, Maclaurin D, Aguilera-Iparraguirre J, Gomez-Bombarelli R, Hirzel T, Aspuru-Guzik A, Adams RP (2015) Convolutional networks on graphs for learning molecular fingerprints. In: Advances in neural information processing systems, pp 2224–2232

  19. Atwood J, Towsley D (2016) Diffusion-convolutional neural networks. In: Advances in neural information processing systems, pp 1993–2001

  20. Niepert M, Ahmed M, Kutzkov K (2016) Learning convolutional neural networks for graphs. In: Proceedings of the 33rd international conference on machine learning, vol 48, pp 2014–2023

  21. Monti F, Boscaini D, Masci J, Rodola E, Svoboda J, Bronstein MM (2017) Geometric deep learning on graphs and manifolds using mixture model CNNs. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, Honolulu, pp 5425–5434. https://doi.org/10.1109/CVPR.2017.576

  22. Hamilton WL, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. In: Advances in neural information processing systems, pp 1024–1034

  23. Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. In: 6th International conference on learning representations

  24. Thekumparampil KK, Wang C, Oh S, Li LJ (2018) Attention-based graph neural network for semi-supervised learning. arXiv preprint. arXiv:1803.03735

  25. Abu-El-Haija S, Perozzi B, Kapoor A, Alipourfard N, Lerman K, Harutyunyan H, Steeg GV, Galstyan A (2019) MixHop: higher-order graph convolutional architectures via sparsified neighborhood mixing. In: Proceedings of the 36th international conference on machine learning, pp 21–29

  26. Zhang K, Zhu Y, Wang J, Zhang J (2020) Adaptive structural fingerprints for graph attention networks. In: 8th International conference on learning representations

  27. Xu K, Li C, Tian Y, Sonobe T, Kawarabayashi Ki, Jegelka S (2018) Representation learning on graphs with jumping knowledge networks. In: Proceedings of the 35th international conference on machine learning

  28. Rong Y, Huang W, Xu T, Huang J (2020) DropEdge: towards deep graph convolutional network on node classification. In: Proceedings of the 37th international conference on machine learning

  29. Chen M, Wei Z, Huang Z, Ding B, Li Y (2020) Simple and deep graph convolutional networks. In: Proceedings of the 37th international conference on machine learning, pp 1725–1735

  30. Chung FR, Graham FC (1997) Spectral graph theory. American Mathematical Society, Providence

    Google Scholar 

  31. Li Q, Han Z, Wu XM (2018) Deeper insights into graph convolutional networks for semi-supervised learning. In: Proceedings of the AAAI conference on artificial intelligence, pp 3538–3545

  32. Ye Z, Ma T, Chen C (2020) Curvature graph network. In: 8th International conference on learning representations

  33. Hardt M, Ma T (2017) Identity matters in deep learning. In: 5th International conference on learning representations

  34. Li R, Wang S, Zhu F, Huang J (2018) Adaptive graph convolutional neural networks. In: Proceedings of the AAAI conference on artificial intelligence

  35. NT H, Maehara T (2019) Revisiting graph neural networks: all we have is low-pass filters. arXiv preprint. arXiv:1905.09550

  36. Vashishth S, Sanyal S, Nitin V, Talukdar P (2020) Composition-based multi-relational graph convolutional networks. In: 8th International conference on learning representations

  37. Gao H, Wang Z, Ji S (2018) Large-scale learnable graph convolutional networks. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. ACM, London, pp 1416–1424. https://doi.org/10.1145/3219819.3219947

  38. Chen J, Ma T, Xiao C (2018) FastGCN: fast learning with graph convolutional networks via importance sampling. In: 6th International conference on learning representations

  39. Huang W, Zhang T, Rong Y, Huang J (2018) Adaptive sampling towards fast graph representation learning. In: Advances in neural information processing systems

  40. Lee J, Lee I, Kang J (2019) Self-attention graph pooling. In: Proceedings of the 36th international conference on machine learning

  41. Chiang WL, Liu X, Si S, Li Y, Bengio S, Hsieh CJ (2019) Cluster-GCN: an efficient algorithm for training deep and large graph convolutional networks. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. ACM, Anchorage, pp 257–266. https://doi.org/10.1145/3292500.3330925

  42. Li G, Muller M, Thabet A, Ghanem B (2019) DeepGCNs: can GCNs go as deep as CNNs? In: 2019 IEEE/CVF international conference on computer vision (ICCV). IEEE, Seoul, pp 9266–9275. https://doi.org/10.1109/ICCV.2019.00936

  43. Loukas A (2020) What graph neural networks cannot learn: depth vs width. In: 8th International conference on learning representations

  44. Oono K, Suzuki T (2020) Graph neural networks exponentially lose expressive power for node classification. In: 8th International conference on learning representations

  45. Zhou K, Huang X, Li Y, Zha D, Chen R, Hu X (2020) Towards deeper graph neural networks with differentiable group normalization. In: Advances in neural information processing systems, vol 33

  46. Chen Y, Wu L, Zaki MJ (2020) Iterative deep graph learning for graph neural networks: better and robust node embeddings. In: Advances in neural information processing systems, vol 33

  47. Zhou J, Cui G, Zhang Z, Yang C, Liu Z, Wang L, Li C, Sun M (2019) Graph neural networks: a review of methods and applications. In: 7th International conference on learning representations

  48. Chen L, Wu L, Hong R, Zhang K, Wang M (2020) Revisiting graph based collaborative filtering: a linear residual graph convolutional network approach. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 27–34. https://doi.org/10.1609/aaai.v34i01.5330

  49. Wu Z, Pan S, Chen F, Long G, Zhang C, Yu PS (2021) A comprehensive survey on graph neural networks. IEEE Trans Neural Netw Learn Syst 32(1):4–24. https://doi.org/10.1109/TNNLS.2020.2978386

    Article  MathSciNet  Google Scholar 

  50. Zhang J, Shi X, Xie J, Ma H, King I, Yeung DY (2018) GaAN: gated attention networks for learning on large and spatiotemporal graphs. In: Proceedings of the 34th conference on uncertainty in artificial intelligence, pp 339–349

  51. Veličković P, Fedus W, Hamilton WL, Liò P, Bengio Y, Hjelm RD (2019) Deep graph infomax. In: 7th International conference on learning representations

  52. Xu B, Shen H, Cao Q, Qiu Y, Cheng X (2019) Graph wavelet neural network. In: 7th International conference on learning representations

  53. Zhu H, Feng F, He X, Wang X, Li Y, Zheng K, Zhang Y (2020) Bilinear graph neural network with neighbor interactions. In: Proceedings of the twenty-ninth international joint conference on artificial intelligence, international joint conferences on Artificial Intelligence Organization, Yokohama, Japan, vol 5, pp 1452–1458. https://doi.org/10.24963/ijcai.2020/202

  54. Gao H, Ji S (2019) Graph U-Nets. In: Proceedings of the 36th international conference on machine learning

  55. Bianchi FM, Grattarola D, Alippi C (2020) Spectral clustering with graph neural networks for graph pooling. In: Proceedings of the 37th international conference on machine learning

  56. Ying R, You J, Morris C, Ren X, Hamilton WL, Leskovec J (2018) Hierarchical graph representation learning with differentiable pooling. In: Advances in neural information processing systems

  57. Cai D, Lam W (2020) Graph transformer for graph-to-sequence learning. In: Proceedings of the AAAI conference on artificial intelligence

  58. Xhonneux LPAC, Qu M, Tang J (2020) Continuous graph neural networks. In: Proceedings of the 37th international conference on machine learning, pp 10432–10411

  59. Avelar PHC, Tavares AR, Gori M, Lamb LC (2021) Discrete and continuous deep residual learning over graphs. In: Proceedings of the 13th international conference on agents and artificial intelligence, vol 2, pp 119–131. https://doi.org/10.5220/0010231501190131

  60. Sen P, Namata G, Bilgic M, Getoor L, Galligher B, Eliassi-Rad T (2008) Collective classification in network data. AI Mag 29(3):93. https://doi.org/10.1609/aimag.v29i3.2157

    Article  Google Scholar 

  61. Rozemberczki B, Allen C, Sarkar R (2019) Multi-scale attributed node embedding. arXiv preprint. arXiv:1909.13021

  62. Pei H, Wei B, Chang KCC, Lei Y, Yang B (2020) Geom-GCN: geometric graph convolutional networks. In: 8th International conference on learning representations

  63. Yang Z, Cohen WW, Salakhutdinov R (2016) Revisiting semi-supervised learning with graph embeddings. In: Proceedings of the 33th international conference on machine learning, pp 40–48

  64. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: 3th International conference on learning representations

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (Grant no. 61772386).

Author information

Authors and Affiliations

  1. School of Computer Science, Wuhan University, Wuhan, China

    Fei Yang, Huyin Zhang & Shiming Tao

Authors
  1. Fei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huyin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shiming Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huyin Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, F., Zhang, H. & Tao, S. Hybrid deep graph convolutional networks. Int. J. Mach. Learn. & Cyber. 13, 2239–2255 (2022). https://doi.org/10.1007/s13042-022-01520-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-022-01520-y

Keywords

Search

Navigation