Skip to main content

Advertisement

Springer Nature Link
Log in

Adaptive curvature exploration geometric graph neural network

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Graph Neural networks (GNNs) which are powerful and widely applied models are based on the assumption that graph topologies play key roles in the graph representation learning.However, the existing GNN methods are based on the Euclidean space embedding, which is difficult to represent a variety of graph geometric properties well. Recently, Riemannian geometries have been introduced into GNNs, such as Hyperbolic Graph Neural Networks proposed for the hierarchy-preserving graph representation learning. In Riemannian geometry, the different graph topological structures can be reflected by corresponding curved embedding spaces, such as a hyperbolic space can be understood as a continuous tree-like structure and a spherical space can be understood as a continuous clique. However, most existing non-Euclidean GNNs are based on heuristic, manual statistical, or estimation methods, which is difficult to automatically select the appropriate embedding space for graphs with different topological properties. To deal with this problem, we propose the Adaptive Curvature Exploration Geometric Graph Neural Network to automatically learn high-quality graph representations and explore the embedding space with optimal curvature at the same time. We optimize the multi-objective optimization problem of the graph representation learning and curvature exploration with the multi-agent reinforcement learning and using the Nash Q-learning algorithm to collaboratively train the two agents to reach Nash equilibrium. We construct extensive experiments including synthetic and real-world graph datasets, and the results demonstrate significant and consistent performance improvement and generalization of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. The datasets and the source code are released at https://github.com/RingBDStack/ACE-HGNN.

References

  1. Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: ICLR

  2. Xu K, Hu W, Leskovec J, Jegelka S (2018) How powerful are graph neural networks? In: International conference on learning representations

  3. Zhang M, Cui Z, Neumann M, Chen Y (2018) An end-to-end deep learning architecture for graph classification. In: Thirty-second AAAI conference on artificial intelligence

  4. Hamilton WL, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. In: Proceedings of the 31st international conference on neural information processing systems, pp. 1025–1035

  5. Zhang Z, Cui P, Zhu W (2020) A survey. IEEE Trans Knowl Data Eng Deep Learn Graphs

  6. Li J, Zhu T, Zhou H, Sun Q, Jiang C, Zhang S, Hu C (2022) Aiqoser: building the efficient inference-qos for ai services. In: IWQoS, pp. 1–10. IEEE

  7. Sun Q, Li J, Yuan H, Fu X, Peng H, Ji C, Li Q, Yu PS (2022) Position-aware structure learning for graph topology-imbalance by relieving under-reaching and over-squashing. In: CIKM

  8. Sun Q, Li J, Peng H, Wu J, Fu X, Ji C, Yu PS (2022) Graph structure learning with variational information bottleneck. AAAI 36:4165–4174

    Article  Google Scholar 

  9. Gilmer J, Schoenholz SS, Riley PF, Vinyals O, Dahl GE (2017) Neural message passing for quantum chemistry. In: International conference on machine learning, pp 1263–1272. PMLR

  10. Aaron Clauset, Cristopher Moore, Newman Mark EJ (2008) Hierarchical structure and the prediction of missing links in networks. Nature 453(7191):98–101

    Article  Google Scholar 

  11. Krioukov D, Papadopoulos F, Kitsak M, Vahdat A, Boguná M (2010) Hyperbolic geometry of complex networks. Phys Rev E 82(3):036106

    Article  MathSciNet  Google Scholar 

  12. Fragkiskos P, Maksim K, Ángeles SM, Bogun á Mn, Krioukov D (2012) Popularity versus similarity in growing networks. Nature 489(7417):537–540

  13. Alexander Stephanie, Spivak M (1978) A comprehensive introduction to differential geometry. Bull Am Math Soc 84(1):27–32

    Article  MathSciNet  Google Scholar 

  14. Nickel M, Kiela D (2017) Poincaré embeddings for learning hierarchical representations. Adv Neural Inf Process Syst 30:6338–6347

    Google Scholar 

  15. Nickel M, Kiela D (2018) Learning continuous hierarchies in the lorentz model of hyperbolic geometry. In: International conference on machine learning, pp 3779–3788. PMLR

  16. Chami I, Ying Z, Ré C, Leskovec J (2019) Hyperbolic graph convolutional neural networks. Adv Neural Inf Process Syst 32:4868–4879

    Google Scholar 

  17. Defferrard M, Perraudin N, Kacprzak T, Sgier RR (2019) Deepsphere: towards an equivariant graph-based spherical cnn. arXiv preprint arXiv:1904.05146

  18. Ungar Abraham Albert(2008) A gyrovector space approach to hyperbolic geometry. Synthesis Lectures on Mathematics and Statistics 1(1):1–194

  19. Sala F, De Sa C, Gu A, Ré C (2018) Representation tradeoffs for hyperbolic embeddings. In: International conference on machine learning, pp. 4460–4469. PMLR

  20. Ganea O, Bécigneul G, Hofmann T (2018) Hyperbolic entailment cones for learning hierarchical embeddings. In: International conference on machine learning, pp. 1646–1655. PMLR

  21. Gu A, Sala F, Gunel B, Ré C (2018) Learning mixed-curvature representations in product spaces. In: International conference on learning representations

  22. Davidson TR, Falorsi L, De Cao N, Kipf T, Tomczak JM (2018) Hyperspherical variational auto-encoders. In: 34th Conference on uncertainty in artificial intelligence 2018, UAI 2018, pp 856–865. Association For Uncertainty in Artificial Intelligence (AUAI)

  23. Xu J, Durrett G (2018) Spherical latent spaces for stable variational autoencoders. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 4503–4513

  24. Wilson RC, Hancock ER, Pekalska E, Duin RPW (2014) Spherical and hyperbolic embeddings of data. IEEE Trans Pattern Anal Mach Intell , 36(11):2255–2269

  25. Bachmann G, Bécigneul G, Ganea O (2020) Constant curvature graph convolutional networks. In: International conference on machine learning, pp. 486–496. PMLR

  26. Fu X, Li J, Wu J, Sun Q, Ji C, Wang S, Tan J, Peng H, Yu PS (2021) Ace-hgnn: adaptive curvature exploration hyperbolic graph neural network. arXiv preprint arXiv:2110.07888

  27. Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. In: International conference on learning representations

  28. Li J, Peng H, Cao Y, Dou Y, Zhang H, Yu P, He L (2021) Higher-order attribute-enhancing heterogeneous graph neural networks. IEEE Trans Knowl Data Eng

  29. Jin Yilun, Song Guojie, Shi Chuan (2020) Gralsp: Graph neural networks with local structural patterns. Proc AAAI Conf Artif Intell 34:4361–4368

    Google Scholar 

  30. Monti F, Otness K, Bronstein MM (2018) Motifnet: a motif-based graph convolutional network for directed graphs. In: 2018 IEEE data science workshop (DSW), pp 225–228. IEEE

  31. Dou Y, Liu Z, Sun L, Deng Y, Peng H, Yu PS (2020) Enhancing graph neural network-based fraud detectors against camouflaged fraudsters. In: Proceedings of the 29th ACM international conference on information and knowledge management, pp. 315–324

  32. Sadeghi A, Wang G, Giannakis GB (2019) Deep reinforcement learning for adaptive caching in hierarchical content delivery networks. IEEE Trans Cognitive Commun Netw, 5(4):1024–1033

  33. Peng H, Zhang R, Dou Y, Yang R, Zhang J, Yu PS (2021) Reinforced neighborhood selection guided multi-relational graph neural networks. arXiv preprint arXiv:2104.07886

  34. Sun Q, Li J, Peng H, Wu J, Ning Y, Yu PS, He L (2021) Sugar: subgraph neural network with reinforcement pooling and self-supervised mutual information mechanism. In: Proceedings of the web conference 2021, pp 2081–2091

  35. Peng H, Yang R, Wang Z, Li J, He L, Yu P, Zomaya A, Ranjan R (2021) Lime: low-cost incremental learning for dynamic heterogeneous information networks. IEEE Trans Comput

  36. Ines C, Albert G, Vaggos C, Christopher R. From trees to continuous embeddings and back: Hyperbolic hierarchical clustering. arXiv preprint arXiv:2010.00402, 2020

  37. Sonthalia R, Gilbert A (2020) Tree! i am no tree! i am a low dimensional hyperbolic embedding. Adv Neural Inf Process Syst, 33

  38. Balazevic I, Allen C, Hospedales T (2019) Multi-relational poincaré graph embeddings. Adv Neural Inf Process Syst 32:4463–4473

    Google Scholar 

  39. Sun Z, Chen M, Hu W, Wang C, Dai J, Zhang W (2020) Knowledge association with hyperbolic knowledge graph embeddings. In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pp 5704–5716

  40. Valentin K, Leyla M, Ustinova E, Oseledets I, Lempitsky V (2020) Hyperbolic image embeddings. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6418–6428

  41. Liu S, Chen J, Pan L, Ngo C-W, Chua T-S, Jiang Y-G (2020) Hyperbolic visual embedding learning for zero-shot recognition. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9273–9281

  42. Lamping J, Rao R, Pirolli P (1995) A focus+ context technique based on hyperbolic geometry for visualizing large hierarchies. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 401–408

  43. Bonnabel Silvere (2013) Stochastic gradient descent on riemannian manifolds. IEEE Trans Auto Control 58(9):2217–2229

    Article  MathSciNet  MATH  Google Scholar 

  44. Dhingra B, Shallue CJ, Norouzi M, Dai AM, Dahl GE (2018) Embedding text in hyperbolic spaces. NAACL HLT 2018, p 59

  45. Lin T, Zha H (2008) Riemannian manifold learning. IEEE TPAMI 30(5):796–809

    Article  Google Scholar 

  46. Ungar AA (1999) The hyperbolic pythagorean theorem in the poincaré disc model of hyperbolic geometry. AM MATH MON, 106(8):759–763

  47. Ungar AA (2005) Analytic hyperbolic geometry: mathematical foundations and applications. World Sci

  48. Ungar AA (2014) Analytic hyperbolic geometry in n dimensions: an introduction. CRC Press,

  49. Tifrea A, Bécigneul G, Ganea OE (2019) Hyperbolic word embeddings. ICLR, Poincare glove

  50. Ganea OE, Bécigneul G, Hofmann T (2018) Hyperbolic neural networks. In: NeurIPS, pp 5350–5360

  51. Stadler Wolfram (1979) A survey of multicriteria optimization or the vector maximum problem, part i: 1776–1960. J Optimiz Theory App 29(1):1–52

    Article  MATH  Google Scholar 

  52. Ungar AA (2010) Barycentric calculus in Euclidean and hyperbolic geometry: a comparative introduction. World Sci

  53. Hu J, Wellman MP (2003) Nash q-learning for general-sum stochastic games. J Mach Learn Res, 4(Nov):1039–1069

  54. Holland PW, Blackmond LK, Samuel L (1983) Stochastic blockmodels: first steps. Social Netw 5(2):109–137

    Article  MathSciNet  Google Scholar 

  55. Barabási A-L, Albert R (1999) Emergence of scaling in random networks. Science 286(5439):509–512

    Article  MathSciNet  MATH  Google Scholar 

  56. Watts Duncan J, Strogatz Steven H (1998) Collective dynamics of ‘small-world’networks. Nature 393(6684):440–442

    Article  MATH  Google Scholar 

  57. Sen P, Namata G, Bilgic M, Getoor L, Galligher B, Eliassi-Rad Tina (2008) Collective classification in network data. AI Mag 29(3):93–93

    Google Scholar 

  58. Namata G, London B, Getoor L, Huang B (2012) UMD EDU. Query-driven active surveying for collective classification. In: MLG, 8

  59. Ryan RA, Nesreen K (2015) The network data repository with interactive graph analytics and visualization. In AAAI

  60. Jonckheere Edmond, Lohsoonthorn Poonsuk, Bonahon Francis (2008) Scaled Gromov hyperbolic graphs. J Graph Theory 57(2):157–180

    Article  MathSciNet  MATH  Google Scholar 

  61. Narayan Onuttom, Saniee Iraj (2011) Large-scale curvature of networks. Phys Rev E 84(6):066108

    Article  Google Scholar 

  62. Adcock AB, Sullivan BD, Mahoney MW (2013) Tree-like structure in large social and information networks. In ICDM, pp 1–10. IEEE

  63. Ganea OE, Bécigneul G, Hofmann T (2018) Hyperbolic neural networks. In Bengio S, Wallach HM, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R (eds), NeurIPS, pp 5350–5360

  64. Nickel M, Kiela D (2017) Poincaré embeddings for learning hierarchical representations. In: NeurIPS, pp 6338–6347

  65. Van der Maaten L, Hinton G (2008) Visualizing data using t-sne. J Mach Learn Research, 9(11)

Download references

Acknowledgements

The corresponding author is Jianxin Li. The authors of this paper were supported by the NSFC through grants (No.U20B2053 and 62172443) and the ARC DECRA Project (No. DE200100964). This work is also supported in part by NSF under grants III-1763325, III-1909323, III-2106758, and SaTC-1930941.

Author information

Authors and Affiliations

  1. Beijing Advanced Innovation Center for Big Data and Brain Computing, Beihang University, Beijing, 100191, China

    Xingcheng Fu, Jianxin Li, Jiawen Qin, Qingyun Sun, Cheng Ji & Hao Peng

  2. School of Computer Science and Engineering, Beihang University, Beijing, 100191, China

    Xingcheng Fu, Jianxin Li, Jiawen Qin, Qingyun Sun & Cheng Ji

  3. School of Computing, Macquarie University, Sydney, Australia

    Jia Wu

  4. School of Computer Science and Engineering, Central South University, Changsha, 410083, China

    Senzhang Wang

  5. Department of Computer Science, University of Illinois at Chicago, Chicago, IL, 60607, USA

    Philip S. Yu

Authors
  1. Xingcheng Fu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jianxin Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Jia Wu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Jiawen Qin
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Qingyun Sun
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Cheng Ji
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Senzhang Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  8. Hao Peng
    View author publications

    You can also search for this author inPubMed Google Scholar

  9. Philip S. Yu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Jianxin Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fu, X., Li, J., Wu, J. et al. Adaptive curvature exploration geometric graph neural network. Knowl Inf Syst 65, 2281–2304 (2023). https://doi.org/10.1007/s10115-022-01811-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-022-01811-4

Keywords