Skip to main content
Springer Nature Link
Log in

Virtual Nodes based Heterogeneous Graph Convolutional Neural Network for Efficient Long-Range Information Aggregation

  • Conference paper
  • First Online:
Artificial Neural Networks and Machine Learning – ICANN 2024 (ICANN 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15020))

Included in the following conference series:

  • 678 Accesses

Abstract

Heterogeneous Graph Neural Networks (HGNNs) have exhibited powerful performance in heterogeneous graph learning by aggregating information from various types of nodes and edges. However, existing heterogeneous graph models often struggle to capture long-range information or necessitate stacking numerous layers to learn such dependencies, resulting in high computational complexity and encountering over-smoothing issues. In this paper, we propose a Virtual Nodes based Heterogeneous Graph Convolutional Network (VN-HGCN), which leverages virtual nodes to facilitate enhanced information flow within the graph. Virtual nodes are auxiliary nodes interconnected with all nodes of a specific type in the graph, facilitating efficient aggregation of long-range information across different types of nodes and edges. By incorporating virtual nodes into the graph structure, VN-HGCN achieves effective information aggregation with only 4 layers. Additionally, we demonstrate that VN-HGCN can serve as a versatile framework that can be seamlessly applied to other HGNN models, showcasing its generalizability. Empirical evaluations validate the effectiveness of VN-HGCN, and extensive experiments conducted on three real-world heterogeneous graph datasets demonstrate the superiority of our model over several state-of-the-art baselines.

The work described in this paper was supported partially by the National Natural Science Foundation of China (12271111), Guangdong Basic and Applied Basic Research Foundation (2022A1515011726), Special Support Plan for High-Level Talents of Guangdong Province (2019TQ05X571). The corresponding author is Jia Cai.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Dong, Y., Chawla, N.V., Swami, A.: metapath2vec: scalable representation learning for heterogeneous networks. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD, pp. 135–144. ACM (2017)

    Google Scholar 

  2. Fu, X., Zhang, J., Meng, Z., King, I.: MAGNN: metapath aggregated graph neural network for heterogeneous graph embedding. In: The World Wide Web Conference 2020, WWW, pp. 2331–2341. ACM/IW3C2 (2020)

    Google Scholar 

  3. Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O., Dahl, G.E.: Neural message passing for quantum chemistry. In: Proceedings of the 34th International Conference on Machine Learning, ICML. vol. 70, pp. 1263–1272. PMLR (2017)

    Google Scholar 

  4. Han, H., Zhao, T., Yang, C., Zhang, H., Liu, Y., Wang, X., Shi, C.: OpenHGNN: an open source toolkit for heterogeneous graph neural network. In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management, CIKM, pp. 3993–3997. ACM (2022)

    Google Scholar 

  5. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 770–778 (2016)

    Google Scholar 

  6. Hong, H., Guo, H., Lin, Y., Yang, X., Li, Z., Ye, J.: An attention-based graph neural network for heterogeneous structural learning. In: The 34th AAAI Conference on Artificial Intelligence, pp. 4132–4139. AAAI Press (2020)

    Google Scholar 

  7. Hu, Z., Dong, Y., Wang, K., Sun, Y.: Heterogeneous graph transformer. In: The World Wide Web Conference 2020, WWW, pp. 2704–2710. ACM/IW3C2 (2020)

    Google Scholar 

  8. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 2261–2269 (2017)

    Google Scholar 

  9. Hussein, R., Yang, D., Cudré-Mauroux, P.: Are meta-paths necessary?: Revisiting heterogeneous graph embeddings. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, 2018, CIKM, pp. 437–446. ACM (2018)

    Google Scholar 

  10. Ishiguro, K., Maeda, S.I., Koyama, M.: Graph warp module: an auxiliary module for boosting the power of graph neural networks in molecular graph analysis (2019). arXiv preprint arXiv:1902.01020

  11. Ji, H., Wang, X., Shi, C., Wang, B., Philip, S.Y.: Heterogeneous graph propagation network. IEEE Trans. Knowl. Data Eng. 35(1), 521–532 (2021)

    Google Scholar 

  12. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: 3rd International Conference on Learning Representations, ICLR (2015)

    Google Scholar 

  13. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: 5th International Conference on Learning Representations, ICLR (2017)

    Google Scholar 

  14. Li, G., Müller, M., Thabet, A.K., Ghanem, B.: DeepGCNs: can GCNs go as deep as CNNs? In: 2019 IEEE/CVF International Conference on Computer Vision, ICCV, pp. 9266–9275. IEEE (2019)

    Google Scholar 

  15. Li, Q., Han, Z., Wu, X.: Deeper insights into graph convolutional networks for semi-supervised learning. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence, pp. 3538–3545. AAAI Press (2018)

    Google Scholar 

  16. Liu, J., Hooi, B., Kawaguchi, K., Xiao, X.: MGNNI: multiscale graph neural networks with implicit layers. In: Conference on Neural Information Processing Systems 2022, NeurIPS (2022)

    Google Scholar 

  17. Liu, J., Kawaguchi, K., Hooi, B., Wang, Y., Xiao, X.: EIGNN: efficient infinite-depth graph neural networks. In: Conference on Neural Information Processing Systems 2021, NeurIPS, pp. 18762–18773 (2021)

    Google Scholar 

  18. Liu, M., Gao, H., Ji, S.: Towards deeper graph neural networks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD, pp. 338–348. ACM (2020)

    Google Scholar 

  19. Luan, S., Zhao, M., Chang, X., Precup, D.: Break the ceiling: stronger multi-scale deep graph convolutional networks. In: Conference on Neural Information Processing Systems 2019, NeurIPS, pp. 10943–10953 (2019)

    Google Scholar 

  20. Luo, L., Fang, Y., Cao, X., Zhang, X., Zhang, W.: Detecting communities from heterogeneous graphs: a context path-based graph neural network model. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, CIKM, pp. 1170-1180. ACM (2021)

    Google Scholar 

  21. Lv, Q., et al.: Are we really making much progress?: Revisiting, benchmarking and refining heterogeneous graph neural networks. In: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD, pp. 1150–1160. ACM (2021)

    Google Scholar 

  22. Pham, T., Tran, T., Dam, H., Venkatesh, S.: Graph classification via deep learning with virtual nodes (2017). arXiv preprint arXiv:1708.04357

  23. Rong, Y., Huang, W., Xu, T., Huang, J.: DropEdge: towards deep graph convolutional networks on node classification. In: 8th International Conference on Learning Representations, ICLR (2020)

    Google Scholar 

  24. Sun, Y., Han, J.: Mining heterogeneous information networks: a structural analysis approach. ACM SIGKDD Explor. Newsl. 14(2), 20–28 (2013)

    Article  Google Scholar 

  25. Sun, Y., Han, J., Yan, X., Yu, P.S., Wu, T.: PathSim: meta path-based top-k similarity search in heterogeneous information networks. Proc. VLDB Endow. 4(11), 992–1003 (2011)

    Article  Google Scholar 

  26. Vaswani, A., et al.: Attention is all you need. In: Conference on Neural Information Processing Systems 2017, NeurIPS, pp. 5998–6008 (2017)

    Google Scholar 

  27. Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. In: 6th International Conference on Learning Representations, ICLR (2018)

    Google Scholar 

  28. Wang, X., Ji, H., Shi, C., Wang, B., Ye, Y., Cui, P., Yu, P.S.: Heterogeneous graph attention network. In: The World Wide Web Conference 2019, WWW, pp. 2022–2032. ACM (2019)

    Google Scholar 

  29. Xu, K., Li, C., Tian, Y., Sonobe, T., Kawarabayashi, K., Jegelka, S.: Representation learning on graphs with jumping knowledge networks. In: Proceedings of the 35th International Conference on Machine Learning, ICML. vol. 80, pp. 5449–5458. PMLR (2018)

    Google Scholar 

  30. Yang, Y., Guan, Z., Li, J., Zhao, W., Cui, J., Wang, Q.: Interpretable and efficient heterogeneous graph convolutional network. IEEE Trans. Knowl. Data Eng. 35(2), 1637–1650 (2023)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Statistics and Mathematics, Guangdong University of Finance and Economics, Guangzhou, 510320, Guangdong, China

    Ranhui Yan

  2. School of Digital Economics, Guangdong University of Finance and Economics, Guangzhou, 510320, Guangdong, China

    Jia Cai

Authors
  1. Ranhui Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jia Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jia Cai .

Editor information

Editors and Affiliations

  1. IDSIA USI-SUPSI, Lugano, Switzerland

    Michael Wand

  2. Comenius University, Bratislava, Slovakia

    Kristína Malinovská

  3. KAUST Center of Generative AI, Thuwal, Saudi Arabia

    Jürgen Schmidhuber

  4. Helmholtz Zentrum München, Neuherberg, Germany

    Igor V. Tetko

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yan, R., Cai, J. (2024). Virtual Nodes based Heterogeneous Graph Convolutional Neural Network for Efficient Long-Range Information Aggregation. In: Wand, M., Malinovská, K., Schmidhuber, J., Tetko, I.V. (eds) Artificial Neural Networks and Machine Learning – ICANN 2024. ICANN 2024. Lecture Notes in Computer Science, vol 15020. Springer, Cham. https://doi.org/10.1007/978-3-031-72344-5_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-72344-5_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-72343-8

  • Online ISBN: 978-3-031-72344-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics