Skip to main content

Advertisement

HG-search: multi-stage search for heterogeneous graph neural networks

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In recent years, heterogeneous graphs, a complex graph structure that can express multiple types of nodes and edges, have been widely used for modeling various real-world scenarios. As a powerful analysis tool, heterogeneous graph neural networks (HGNNs) can effectively mine the information and knowledge in heterogeneous graphs. However, designing an excellent HGNN architecture requires a lot of domain knowledge and is a time-consuming and laborious task. Inspired by neural architecture search (NAS), some works on homogeneous graph NAS have emerged. However, there are few works on heterogeneous graph NAS. In addition, the hyperparameters related to the HGNN architecture are also important factors affecting its performance in downstream tasks. Manually tuning hyperparameters is also a tedious and inefficient process. To solve the above problems, we propose a novel search (HG-Search for short) algorithm specifically for HGNNs, which achieves fully automatic architecture design and hyperparameter tuning. Specifically, we first design a search space for HG-Search, composed of two parts: HGNN architecture search space and hyperparameter search space. Furthermore, we propose a multi-stage search (MS-Search for short) module and combine it with the policy gradient search (PG-Search for short). Experiments on real-world datasets show that this method can design HGNN architectures comparable to those manually designed by humans and achieve automatic hyperparameter tuning, significantly improving the performance in downstream tasks. The code and related datasets can be found at https://github.com/dawn-creator/HG-Search.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Algorithm 2
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Availability of data and materials

The data and materials have been published on github. https://github.com/dawn-creator/HG-Search/tree/main/Dataset_process

Code availability

The code has been published on github. https://github.com/dawn-creator/HG-Search

References

  1. Gori M, Monfardini G, Scarselli F (2005) A new model for learning in graph domains. In: Proceedings 2005 IEEE international joint conference on neural networks, 2005., vol. 2. pp 729–734

  2. Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G (2009) The graph neural network model. IEEE Trans Neural Netw 20:61–80

    Article  MATH  Google Scholar 

  3. Hamilton WL, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. Curran Associates Inc., Red Hook, NY, USA, pp 1025–1035

  4. Zhang C, Song D, Huang C, Swami A, Chawla NV (2019) Heterogeneous graph neural network. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. pp 793–803

  5. Sun Z, Deng Z, Nie J-Y, Tang J (2018) Rotate: Knowledge graph embedding by relational rotation in complex space. arXiv:1902.10197

  6. Yu L, Sun L, Du B, Liu C, Lv W, Xiong H (2020) Hybrid micro/macro level convolution for heterogeneous graph learning. arXiv:2012.14722

  7. Wang X, Ji H, Shi C, Wang B, Ye Y, Cui P, Yu PS (2019) Heterogeneous graph attention network. The world wide web conference. Association for Computing Machinery, New York, NY, USA, pp 2022–2032

    Chapter  MATH  Google Scholar 

  8. Ji H, Wang X, Shi C, Wang B, Yu PS (2023) Heterogeneous graph propagation network. IEEE Trans Knowl Data Eng 35:521–532

    MATH  Google Scholar 

  9. Gao Y, Yang H, Zhang P, Zhou C, Hu Y (2021) Graph neural architecture search. In: Proceedings of the twenty-ninth international conference on international joint conferences on artificial intelligence. pp 1403–1409

  10. Zhou K, Huang X, Song Q, Chen R, Hu X (2022) Auto-gnn: Neural architecture search of graph neural networks. Front Big Data 5:1029307

    Article  MATH  Google Scholar 

  11. Selvi C, Sivasankar E (2019) A novel adaptive genetic neural network (agnn) model for recommender systems using modified k-means clustering approach. Multim Tools Appl 78:14303–14330

    Article  MATH  Google Scholar 

  12. Shi M, Tang Y, Zhu X, Huang Y, Wilson D, Zhuang Y, Liu J (2022) Genetic-gnn: Evolutionary architecture search for graph neural networks. Knowl-Based Syst 247:108752

    Article  Google Scholar 

  13. Li G, Qian G, Delgadillo IC, Müller M, Thabet AK, Ghanem B (2019) Sgas: Sequential greedy architecture search. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp 1617–1627

  14. Zhao H, Yao Q, Tu W-W (2021) Search to aggregate neighborhood for graph neural network. 2021 IEEE 37th International Conference on Data Engineering (ICDE). pp. 552–563

  15. Wei L, Zhao H, Yao Q, He Z (2021) Pooling architecture search for graph classification. In: Proceedings of the 30th ACM international conference on information & knowledge management. Association for Computing Machinery, New York, NY, USA, pp 2091–2100

  16. Zhu G, Zhu Z-W, Wang W, Xu Z, Yuan C, Huang Y (2023) Autoac: Towards automated attribute completion for heterogeneous graph neural network. 2023 IEEE 39th International Conference on Data Engineering (ICDE). pp 2808–2821

  17. Ning W, Cheng R, Shen J, Haldar NAH, Kao B, Yan X, Huo N, Lam WK, Li T, Tang B (2022) Automatic meta-path discovery for effective graph-based recommendation. In: Proceedings of the 31st ACM international conference on information & knowledge management. Association for Computing Machinery, New York, NY, USA, pp 1563–1572

  18. Ding Y, Yao Q, Zhao H, Zhang T (2021) Diffmg: Differentiable meta graph search for heterogeneous graph neural networks. In: Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining. Association for Computing Machinery, New York, NY, USA, pp 279–288

  19. Li C, Xu H, He K (2023) Differentiable meta multigraph search with partial message propagation on heterogeneous information networks. In: Proceedings of the AAAI conference on artificial intelligence, vol. 37. pp 8518–8526

  20. Gao Y, Zhang P, Li Z, Zhou C, Liu Y, Hu Y (2021) Heterogeneous graph neural architecture search. 2021 IEEE International Conference on Data Mining (ICDM). pp 1066–1071

  21. Gao Y, Zhang P, Zhou C, Yang H, Li Z, Hu Y, Yu PS (2023) Hgnas++: Efficient architecture search for heterogeneous graph neural networks. IEEE Trans Knowl Data Eng 35:9448–9461

    Article  Google Scholar 

  22. Zhang Z, Zhang Z, Wang X, Qin Y, Qin Z, Zhu W (2023) Dynamic heterogeneous graph attention neural architecture search. In: Proceedings of the AAAI conference on artificial intelligence, vol. 37. pp 11307–11315

  23. Yang T, Hu L, Shi C, Ji H, Li X, Nie L (2021) Hgat: Heterogeneous graph attention networks for semi-supervised short text classification. ACM Trans Inform Syst 39(3):1–29

    Article  MATH  Google Scholar 

  24. Hu Z, Dong Y, Wang K, Sun Y (2020) Heterogeneous graph transformer. In: Proceedings of the web conference 2020. Association for Computing Machinery, New York, NY, USA, pp 2704–2710

  25. Lv Q, Ding M, Liu Q, Chen Y, Feng W, He S, Zhou C, Jiang J, Dong Y, Tang J (2021) Are we really making much progress? revisiting, benchmarking and refining heterogeneous graph neural networks. In: Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining. Association for Computing Machinery, New York, NY, USA, pp 1150–1160

  26. Hong H, Guo H, Lin Y, Yang X, Li Z, Ye J (2019) An attention-based graph neural network for heterogeneous structural learning. arXiv:1912.10832

  27. Schlichtkrull M, Kipf TN, Bloem P, Berg R, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: The Semantic Web: 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3–7, 2018, Proceedings. Springer, Berlin, Heidelberg, pp 593–607

  28. Kipf T, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv:1609.02907

  29. Fu X, Zhang J, Meng Z, King I (2020) Magnn: Metapath aggregated graph neural network for heterogeneous graph embedding. In: Proceedings of The Web Conference 2020. Association for Computing Machinery, New York, NY, USA, pp 2331–2341

  30. Wang X, Lu Y, Shi C, Wang R, Cui P, Mou S (2022) Dynamic heterogeneous information network embedding with meta-path based proximity. IEEE Trans Knowl Data Eng 34:1117–1132

    Article  MATH  Google Scholar 

  31. Yang X, Yan M, Pan S, Ye X, Fan D (2023) Simple and efficient heterogeneous graph neural network. In: Proceedings of the AAAI conference on artificial intelligence, vol. 37. pp 10816–10824

  32. Yun S, Jeong M, Kim R, Kang J, Kim HJ (2019) Graph transformer networks. In: Proceedings of the 33rd international conference on neural information processing systems. pp 11983–11993

  33. Zhao Y, Wang D, Gao X, Mullins RD, Lio’ P, Jamnik M (2020) Probabilistic dual network architecture search on graphs. arXiv:2003.09676

  34. Li Y, Wen Z, Wang Y, Xu C (2021) One-shot graph neural architecture search with dynamic search space. In: Proceedings of the AAAI conference on artificial intelligence, vol. 35. pp 8510–8517

  35. Cai S, Li L, Deng J, Zhang B, Zha Z, Su L, Huang Q (2021) Rethinking graph neural architecture search from message-passing. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp 6653–6662

  36. Lai K-H, Zha D, Zhou K, Hu X (2020) Policy-gnn: Aggregation optimization for graph neural networks. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. Association for Computing Machinery, New York, NY, USA, pp 461–471

  37. Han Z, Xu F, Shi J, Shang Y, Ma H, Hui P, Li Y (2020) Genetic meta-structure search for recommendation on heterogeneous information network. In: Proceedings of the 29th ACM international conference on information & knowledge management. Association for Computing Machinery, New York, NY, USA, pp 455–464

  38. Sun Y, Han J (2013) Mining heterogeneous information networks: A structural analysis approach. SIGKDD Explor. Newsl. 14(2):20–28

    Article  MATH  Google Scholar 

  39. Sun Y, Han J, Yan X, Yu PS, Wu T (2011) Pathsim: Meta path-based top-k similarity search in heterogeneous information networks. Proc VLDB Endow 4(11):992–1003

    Article  MATH  Google Scholar 

  40. Velickovic P, Cucurull G, Casanova A, Romero A, Lio’ P, Bengio Y (2017) Graph attention networks. arXiv:1710.10903

  41. Xu K, Hu W, Leskovec J, Jegelka S (2018) How powerful are graph neural networks? arXiv:1810.00826

  42. Mao Q, Liu Z, Liu C, Sun J (2023) Hinormer: Representation learning on heterogeneous information networks with graph transformer. In: Proceedings of the ACM Web Conference 2023. Association for Computing Machinery, New York, NY, USA, pp 599–610

  43. Zhu S, Zhou C, Pan S, Zhu X, Wang B (2019) Relation structure-aware heterogeneous graph neural network. 2019 IEEE International Conference on Data Mining (ICDM). pp 1534–1539

  44. Zhao T, Yang C, Li Y, Gan Q, Wang Z, Liang F, Zhao H, Shao Y, Wang X, Shi C (2022) Space4hgnn: A novel, modularized and reproducible platform to evaluate heterogeneous graph neural network. In: Proceedings of the 45th International ACM SIGIR conference on research and development in information retrieval. Association for Computing Machinery, New York, NY, USA, pp 2776–2789

  45. Shchur O, Mumme M, Bojchevski A, Günnemann S (2018) Pitfalls of graph neural network evaluation. arXiv:1811.05868

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China [grant numbers 62372494]; Natural Science Foundation of Jilin Province [grant number 20240302086GX, 20240101369JC, 20220101117JC].

Author information

Authors and Affiliations

Authors

Contributions

Conceptualization: Hongmin Sun and Wei Du; Methodology: Hongmin Sun and Wei Du; Formal analysis and investigation: Hongmin Sun and Wei Du; Writing - original draft preparation: Hongmin Sun, Wei Du and Jianhao Liu; Writing - review and editing: Wei Du and Ao Kan; Funding acquisition: Wei Du; Resources: Wei Du; Supervision: Wei Du.All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Wei Du.

Ethics declarations

Conflict of interest/Competing interests

The authors declare no conflict of interest/competing interests.

Ethical and informed consent for data used

The data used in this paper were obtained through publicly available sources, and no ethical or informed consent considerations were required.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, H., Kan, A., Liu, J. et al. HG-search: multi-stage search for heterogeneous graph neural networks. Appl Intell 55, 6 (2025). https://doi.org/10.1007/s10489-024-06058-w

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-06058-w

Keywords