Skip to main content
SpringerLink
Log in

Graph neural architecture prediction

  • Review
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Graph neural networks (GNNs) have shown their superiority in the modeling of graph data. Recently, increasing attention has been paid to automatic graph neural architecture search, aiming to overcome the shortcomings of manually constructing GNN architectures that requires a lot of expert experience. However, existing graph neural architecture search (GraphNAS) methods can only select architecture from the partial evaluated GNN architectures. To solve the challenges, we propose a Graph Neural Architecture Prediction (GraphNAP) framework, which can select the optimal GNN architecture from the search space efficiently. To achieve this goal, a neural predictor is designed in GraphNAP. Firstly, the neural predictor is trained by a small number of sampled GNN architectures. Then, the trained neural predictor is used to predict all GNN architectures in the search space. In this way, GraphNAP can efficiently explore the performance of all GNN architectures in the search space and then select the optimal GNN architecture. The experimental results show that GraphNAP outperforms state-of-the-art both handcrafted and GraphNAS-based methods for both graph and node classification tasks. The python implementation of GraphNAP can be found at https://github.com/BeObm/GraphNAP.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Algorithm 2
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Data availability

All datasets used in the paper are publicly available.

Notes

  1. https://github.com/rusty1s/pytorch_geometric.

References

  1. Wu Z, Pan S, Chen F, Long G, Zhang C, Philip SY (2020) A comprehensive survey on graph neural networks. IEEE Trans Neural Netw Learn Syst 32(1):4–24

    Article  MathSciNet  Google Scholar 

  2. Bai L, Cui L, Jiao Y, Rossi L, Hancock ER (2022) Learning backtrackless aligned-spatial graph convolutional networks for graph classification. IEEE Trans Pattern Anal Mach Intell 44(2):783–798

    Article  Google Scholar 

  3. Manipur I, Manzo M, Granata I, Giordano M, Maddalena L, Guarracino MR (2021) Netpro2vec: a graph embedding framework for biomedical applications. IEEE/ACM Trans Comput Biol Bioinf 19(2):729–740

    Article  Google Scholar 

  4. Gao Y, Yang H, Zhang P, Zhou C, Hu Y (2020) Graph neural architecture search. In: Proceedings of the joint conference on artificial intelligence, pp 1403–1409

  5. Pasa L, Navarin N, Sperduti A (2022) Polynomial-based graph convolutional neural networks for graph classification. Mach Learn 111:1205–1237

    Article  MathSciNet  Google Scholar 

  6. Chen J, Gao J, Chen Y, Moctard OB, Lyu T, Li Z (2022) Auto-GNAS: a parallel graph neural architecture search framework. IEEE Trans Parallel Distrib Syst 33(11):3117–3128

    Article  Google Scholar 

  7. Qin Y, Wang X, Zhang Z, Zhu W (2021) Graph differentiable architecture search with structure learning. In: Proceedings of the annual conference on neural information processing systems, pp 16860–16872

  8. Guan C, Wang X, Zhu W (2021) Autoattend: automated attention representation search. In: Proceedings of the international conference on machine learning, pp 3864–3874

  9. Li Y, King I (2020) Autograph: automated graph neural network. In: Proceedings of the international conference on neural information processing. Springer, pp 189–201

  10. Guo Z, Zhang X, Mu H, Heng W, Liu Z, Wei Y, Sun J (2020) Single path one-shot neural architecture search with uniform sampling. In: Proceedings of the European conference on computer vision. Lecture notes in computer science, pp 544–560

  11. Li Y, Wen Z, Wang Y, Xu C (2021) One-shot graph neural architecture search with dynamic search space. In: Proceedings of the AAAI conference on artificial intelligence, pp 8510–8517

  12. Gao Y, Yang H, Zhang P, Zhou C, Hu Y (2020) Graph neural architecture search. In: Proceedings of the international joint conference on artificial intelligence, pp 1403–1409

  13. Zhou K, Song Q, Huang X, Hu X (2019) Auto-gnn: neural architecture search of graph neural networks. CoRR arXiv:1909.03184

  14. You J, Ying Z, Leskovec J (2020) Design space for graph neural networks. In: Proceedings of advances in neural information processing systems, pp 17009–17021

  15. Sperduti A, Starita A (1997) Supervised neural networks for the classification of structures. IEEE Trans Neural Netw 8(3):714–735

    Article  Google Scholar 

  16. Gilmer J, Schoenholz SS, Riley PF, Vinyals O, Dahl GE (2017) Neural message passing for quantum chemistry. In: Proceedings of the international conference on machine learning. PMLR, pp 1263–1272

  17. Kolouri S, Naderializadeh N, Rohde GK, Hoffmann H (2021) Wasserstein embedding for graph learning. In: Proceedings of the international conference on learning representations

  18. Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: Proceedings of the international conference on learning representations

  19. Wang Y, Wang W, Liang Y, Cai Y, Hooi B (2021) Curgraph: curriculum learning for graph classification. In: Proceedings of the web conference, pp 1238–1248

  20. Xu K, Hu W, Leskovec J, Jegelka S (2019) How powerful are graph neural networks? In: Proceedings of the international conference on learning representations

  21. Oloulade BM, Gao J, Chen J, Lyu T, Al-Sabri R (2021) Graph neural architecture search: a survey. Tsinghua Sci Technol 27(4):692–708

    Article  Google Scholar 

  22. Zhou K, Song Q, Huang X, Hu X (2019) Auto-gnn: neural architecture search of graph neural networks. CoRR arXiv:1909.03184

  23. Sutton RS, McAllester DA, Singh SP, Mansour Y (1999) Policy gradient methods for reinforcement learning with function approximation. In: Proceedings of advances in neural information processing systems, pp 1057–1063

  24. Yu K, Sciuto C, Jaggi M, Musat C, Salzmann M (2020) Evaluating the search phase of neural architecture search. In: Proceedings of the international conference on learning representations

  25. Zhang Y, Lin Z, Jiang J, Zhang Q, Wang Y, Xue H, Zhang C, Yang Y (2020) Deeper insights into weight sharing in neural architecture search. CoRR arXiv:2001.01431

  26. Shi M, Wilson DA, Zhu X, Huang Y, Zhuang Y, Liu J, Tang Y (2020) Evolutionary architecture search for graph neural networks. CoRR arXiv:2009.10199

  27. Li Y, King I (2020) Autograph: automated graph neural network. In: Proceedings of the international conference on neural information processing, pp 189–201

  28. Zhao H, Yao Q, Tu W (2021) Search to aggregate neighborhood for graph neural network. In: Proceedings of the IEEE international conference on data engineering, pp 552–563

  29. Lai K-H, Zha D, Zhou K, Hu X (2020) Policy-gnn: Aggregation optimization for graph neural networks. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining, pp 461–471

  30. Shi H, Pi R, Xu H, Li Z, Kwok JT, Zhang T (2020) Bridging the gap between sample-based and one-shot neural architecture search with BONAS. In: Proceedings of annual conference on neural information processing systems, pp 6–12

  31. White C, Zela A, Ru R, Liu Y, Hutter F (2021) How powerful are performance predictors in neural architecture search? In: Proceedings of annual conference on neural information processing systems, pp 28454–28469

  32. White C, Neiswanger W, Savani Y (2021) BANANAS: Bayesian optimization with neural architectures for neural architecture search. In: Proceedings of AAAI conference on artificial intelligence, pp 10293–10301

  33. Hu W, Fey M, Zitnik M, Dong Y, Ren H, Liu B, Catasta M, Leskovec J (2020) Open graph benchmark: datasets for machine learning on graphs. In: Proceedings of advances in neural information processing systems

  34. Wei L, Zhao H, Yao Q, He Z (2021) Pooling architecture search for graph classification. In: Proceedings of the ACM international conference on information and knowledge management, pp 2091–2100

  35. Velickovic P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. In: Proceedings of the international conference on learning representations

  36. Huang W, Zhang T, Rong Y, Huang J (2018) Adaptive sampling towards fast graph representation learning. In: Proceedings of advances in neural information processing systems, pp 4563–4572

  37. Wang Y, Derr T (2021) Tree decomposed graph neural network. In: Demartini G, Zuccon G, Culpepper JS, Huang Z, Tong H (eds) Proceedings of the ACM international conference on information and knowledge management, pp 2040–2049

  38. Chen M, Wei Z, Huang Z, Ding B, Li Y (2020) Simple and deep graph convolutional networks. In: Proceedings of the international conference on machine learning, pp 1725–1735

  39. Fey M, Lenssen JE (2019) Fast graph representation learning with PyTorch geometric. In: Proceedings of THE ICLR workshop on representation learning on graphs and manifolds

Download references

Acknowledgements

The work was supported by the National Natural Science Foundation of China (No. 62272487).

Funding

The work was supported by the National Natural Science Foundation of China (No. 62272487).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Central South University, Yuelu Street, Changsha, 410083, Hunan, China

    Jianliang Gao, Babatounde Moctard Oloulade, Raeed Al-Sabri, Jiamin Chen, Tengfei Lyu & zhenpeng Wu

Authors
  1. Jianliang Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Babatounde Moctard Oloulade
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Raeed Al-Sabri
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiamin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tengfei Lyu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. zhenpeng Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

J.G. did conceptualization, investigation, project administration, supervision, editing, validation, and review. B.M.O. done conceptualization, data duration, formal analysis, investigation, methodology, resources, software, validation, visualization, editing, and review. R.A. performed investigation, resources, editing, validation, and review. J.C. contributed to investigation, resources, editing, validation, and review. T.L. and Z.W. were involved in investigation, validation, and review.

Corresponding author

Correspondence to Jiamin Chen.

Ethics declarations

Conflict of interest

No conflicts of interest or competing interests.

Ethics approval

Not applicable.

Consent to participate

The manuscript is approved by all authors for publication.

Consent for publication

All authors who participated in this study give the publisher permission to publish this work.

Code availability

https://github.com/BeObm/GraphNAP.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

We aim to demonstrate that the CHRS method yields a more representative and diverse set of sampled architectures compared to random and uniform sampling.

1.1 Proof of diversity

We first prove that CHRS generates a more diverse set of architectures compared to uniform sampling. In uniform sampling, each candidate is equally likely to be selected, resulting in a higher chance of repeated architectures. In CHRS, we sample m GNN architectures for each candidate, ensuring a more comprehensive exploration of the search space.

Lemma 1

The CHRS method produces a set T of n GNN architectures with a higher diversity compared to uniform sampling.

Proof

Assume we have k unique architectures in the search space. In uniform sampling, the probability of selecting a specific architecture is 1/k. The probability of selecting m architectures of the same candidate is \((1/k)^m\). Thus, the probability of not selecting any repeated architectures for a candidate is given by \((1 - (1/k)^m)\). For CHRS, the lower limit m is determined as m = round(n/q). Therefore, the probability of not selecting any repeated architectures for a candidate in CHRS is \((1 - (1/k)^m)\), which is greater than or equal to the probability for uniform sampling. Since CHRS ensures a higher probability of not selecting repeated architectures, it generates a more diverse set of architectures compared to uniform sampling. Hence, Lemma 1 is proven.

1.2 Proof of representativeness

Next, we prove that CHRS provides a more representative sampling of the search space than random sampling. Random sampling lacks control over the distribution of sampled architectures, potentially leading to an insufficient representation of the search space. \(\square \)

Lemma 2

The CHRS method yields a set T of n GNN architectures that better represents the characteristics of the search space compared to random sampling.

Proof

In random sampling, each architecture has an equal chance of being selected. The probability of selecting m architectures with the same candidate is \((1/q)^m\). The probability of not selecting any repeated architectures for a candidate is \((1 - (1/q)^m)\).

In CHRS, we sample m architectures for each candidate, ensuring a controlled distribution of architectures. By sampling m architectures, we increase the chance of covering various regions of the search space for each candidate. Therefore, the probability of not selecting any repeated architectures for a candidate in CHRS is \((1 - (1/q)^m)\), which is greater than or equal to the probability for random sampling. As CHRS provides a higher probability of not selecting repeated architectures, it leads to a more representative sampling of the search space compared to random sampling. Hence, Lemma 2 is proven.

Based on Lemmas 1 and 2, we conclude that the CHRS method produces a more diverse and representative set of GNN architectures compared to random and uniform sampling methods. These properties enable CHRS to capture a broader range of architectures, ultimately leading to improved performance of the machine learning model. \(\square \)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, J., Oloulade, B.M., Al-Sabri, R. et al. Graph neural architecture prediction. Knowl Inf Syst 66, 29–58 (2024). https://doi.org/10.1007/s10115-023-01968-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-023-01968-6

Keywords

Search

Navigation