Skip to main content

Curvature-Based Pooling Within Graph Neural Networks

  • Conference paper
  • First Online:
Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2023)

Abstract

Over-squashing and over-smoothing are two critical issues, that limit the capabilities of graph neural networks (GNNs). While over-smoothing eliminates the differences between nodes making them indistinguishable, over-squashing refers to the inability of GNNs to propagate information over long distances, as exponentially many node states are squashed into fixed-size representations. Both phenomena share similar causes, as both are largely induced by the graph topology. To mitigate these problems in graph classification tasks, we propose CurvPool, a novel pooling method. CurvPool exploits the notion of curvature of a graph to adaptively identify structures responsible for both over-smoothing and over-squashing. By clustering nodes based on the Balanced Forman curvature, CurvPool constructs a graph with a more suitable structure, allowing deeper models and the combination of distant information. We compare it to other state-of-the-art pooling approaches and establish its competitiveness in terms of classification accuracy, computational complexity, and flexibility. CurvPool outperforms several comparable methods across all considered tasks. The most consistent results are achieved by pooling densely connected clusters using the sum aggregation, as this allows additional information about the size of each pool.

C. Sanders and A. Roth—Equal contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://gitlab.com/Cedric_Sanders/masterarbeit.

References

  1. Alon, U., Yahav, E.: On the bottleneck of graph neural networks and its practical implications. arXiv preprint arXiv:2006.05205 (2020)

  2. Bianchi, F.M., Grattarola, D., Alippi, C.: Spectral clustering with graph neural networks for graph pooling. In: International Conference on Machine Learning, pp. 874–883. PMLR (2020)

    Google Scholar 

  3. Bianchi, F.M., Lachi, V.: The expressive power of pooling in graph neural networks. arXiv preprint arXiv:2304.01575 (2023)

  4. Blakely, D., Lanchantin, J., Qi, Y.: Time and space complexity of graph convolutional networks. Accessed 31 Dec 2021

    Google Scholar 

  5. Borgwardt, K.M., Ong, C.S., Schönauer, S., Vishwanathan, S.V.N., Smola, A.J., Kriegel, H.P.: Protein function prediction via graph kernels. Bioinformatics 21, i47–i56 (2005). https://doi.org/10.1093/bioinformatics/bti1007

  6. Bron, C., Kerbosch, J.: Algorithm 457: finding all cliques of an undirected graph. Commun. ACM 16(9), 575–577 (1973)

    Article  MATH  Google Scholar 

  7. Cai, C., Wang, Y.: A note on over-smoothing for graph neural networks. arXiv preprint arXiv:2006.13318 (2020)

  8. Chen, D., Lin, Y., Li, W., Li, P., Zhou, J., Sun, X.: Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 3438–3445 (2020)

    Google Scholar 

  9. Diehl, F., Brunner, T., Le, M.T., Knoll, A.: Towards graph pooling by edge contraction. In: ICML 2019 Workshop on Learning and Reasoning with Graph-Structured Data (2019)

    Google Scholar 

  10. Du, J., Wang, S., Miao, H., Zhang, J.: Multi-channel pooling graph neural networks. In: IJCAI, pp. 1442–1448 (2021)

    Google Scholar 

  11. Eppstein, D., Löffler, M., Strash, D.: Listing all maximal cliques in large sparse real-world graphs. J. Exp. Algorithmics (JEA) 18, 3–1 (2013)

    MATH  Google Scholar 

  12. Fey, M., Yuen, J.G., Weichert, F.: Hierarchical inter-message passing for learning on molecular graphs. arXiv preprint arXiv:2006.12179 (2020)

  13. Forman, R.: Bochner’s method for cell complexes and combinatorial Ricci curvature. Discrete Comput. Geom. 29, 323–374 (2003)

    Google Scholar 

  14. Gao, H., Chen, Y., Ji, S.: Learning graph pooling and hybrid convolutional operations for text representations. In: The World Wide Web Conference, pp. 2743–2749 (2019)

    Google Scholar 

  15. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456. PMLR (2015)

    Google Scholar 

  16. Khasahmadi, A.H., Hassani, K., Moradi, P., Lee, L., Morris, Q.: Memory-based graph networks. arXiv preprint arXiv:2002.09518 (2020)

  17. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)

  18. Lee, J., Lee, I., Kang, J.: Self-attention graph pooling. In: International Conference on Machine Learning, pp. 3734–3743. PMLR (2019)

    Google Scholar 

  19. Lei, F., Liu, X., Jiang, J., Liao, L., Cai, J., Zhao, H.: Graph convolutional networks with higher-order pooling for semisupervised node classification. Concur. Comput. Pract. Experience 34(16), e5695 (2022)

    Article  Google Scholar 

  20. Li, J., Ma, Y., Wang, Y., Aggarwal, C., Wang, C.D., Tang, J.: Graph pooling with representativeness. In: 2020 IEEE International Conference on Data Mining (ICDM), pp. 302–311. IEEE (2020)

    Google Scholar 

  21. Li, Q., Han, Z., Wu, X.M.: Deeper insights into graph convolutional networks for semi-supervised learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)

    Google Scholar 

  22. Liu, N., Jian, S., Li, D., Zhang, Y., Lai, Z., Xu, H.: Hierarchical adaptive pooling by capturing high-order dependency for graph representation learning. IEEE Trans. Knowl. Data Eng. (2021)

    Google Scholar 

  23. Luzhnica, E., Day, B., Lio, P.: Clique pooling for graph classification. arXiv preprint arXiv:1904.00374 (2019)

  24. Morris, C., et al.: Weisfeiler and Leman go neural: higher-order graph neural networks. Proc. AAAI Conf. Artif. Intell. 33, 4602–4609 (2019). https://doi.org/10.1609/aaai.v33i01.33014602

    Article  MATH  Google Scholar 

  25. Nguyen, T., Grishman, R.: Graph convolutional networks with argument-aware pooling for event detection. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)

    Google Scholar 

  26. Noutahi, E., Beaini, D., Horwood, J., Giguère, S., Tossou, P.: Towards interpretable sparse graph representation learning with Laplacian pooling. arXiv preprint arXiv:1905.11577 (2019)

  27. Nt, H., Maehara, T.: Revisiting graph neural networks: all we have is low-pass filters. arXiv preprint arXiv:1905.09550 (2019)

  28. Ollivier, Y.: Ricci curvature of metric spaces. C.R. Math. 345(11), 643–646 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  29. Ollivier, Y.: Ricci curvature of Markov chains on metric spaces. J. Funct. Anal. 256(3), 810–864 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  30. Oono, K., Suzuki, T.: Graph neural networks exponentially lose expressive power for node classification. arXiv preprint arXiv:1905.10947 (2019)

  31. Pan, S., Hu, R., Long, G., Jiang, J., Yao, L., Zhang, C.: Adversarially regularized graph autoencoder for graph embedding. arXiv preprint arXiv:1802.04407 (2018)

  32. Ranjan, E., Sanyal, S., Talukdar, P.: ASAP: adaptive structure aware pooling for learning hierarchical graph representations. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 5470–5477 (2020)

    Google Scholar 

  33. Roth, A., Liebig, T.: Forecasting unobserved node states with spatio-temporal graph neural networks. In: 2022 IEEE International Conference on Data Mining Workshops (ICDMW), pp. 740–747. IEEE (2022)

    Google Scholar 

  34. Roth, A., Liebig, T.: Transforming PageRank into an infinite-depth graph neural network. In: Amini, M.R., Canu, S., Fischer, A., Guns, T., Kralj Novak, P., Tsoumakas, G. (eds.) Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 469–484. Springer, Cham (2022)

    Google Scholar 

  35. Tomita, E., Tanaka, A., Takahashi, H.: The worst-case time complexity for generating all maximal cliques and computational experiments. Theoret. Comput. Sci. 363(1), 28–42 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  36. Topping, J., Di Giovanni, F., Chamberlain, B.P., Dong, X., Bronstein, M.M.: Understanding over-squashing and bottlenecks on graphs via curvature. arXiv preprint arXiv:2111.14522 (2021)

  37. Wang, Y.G., Li, M., Ma, Z., Montufar, G., Zhuang, X., Fan, Y.: Haar graph pooling. In: International Conference on Machine Learning, pp. 9952–9962. PMLR (2020)

    Google Scholar 

  38. Watts, D.J.: Networks, dynamics, and the small-world phenomenon. Am. J. Sociol. 105(2), 493–527 (1999)

    Article  MATH  Google Scholar 

  39. Wu, Z., et al.: MoleculeNet: a benchmark for molecular machine learning. Chem. Sci. 9(2), 513–530 (2018)

    Article  PubMed  MATH  Google Scholar 

  40. Xu, K., Hu, W., Leskovec, J., Jegelka, S.: How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 (2018)

  41. Yan, Y., Hashemi, M., Swersky, K., Yang, Y., Koutra, D.: Two sides of the same coin: heterophily and oversmoothing in graph convolutional neural networks. In: 2022 IEEE International Conference on Data Mining (ICDM), pp. 1287–1292. IEEE (2022)

    Google Scholar 

  42. Yanardag, P., Vishwanathan, S.: Deep graph kernels. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1365–1374 (2015)

    Google Scholar 

  43. Ying, Z., You, J., Morris, C., Ren, X., Hamilton, W., Leskovec, J.: Hierarchical graph representation learning with differentiable pooling. In: Advances in Neural Information Processing Systems, vol. 31 (2018)

    Google Scholar 

  44. Yuan, H., Ji, S.: StructPool: structured graph pooling via conditional random fields. In: Proceedings of the 8th International Conference on Learning Representations (2020)

    Google Scholar 

  45. Zhang, M., Cui, Z., Neumann, M., Chen, Y.: An end-to-end deep learning architecture for graph classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)

    Google Scholar 

  46. Zhang, Y.D., Satapathy, S.C., Guttery, D.S., Górriz, J.M., Wang, S.H.: Improved breast cancer classification through combining graph convolutional network and convolutional neural network. Inf. Process. Manage. 58(2), 102439 (2021)

    Article  Google Scholar 

  47. Zhao, H., Xie, J., Wang, H.: Graph convolutional network based on multi-head pooling for short text classification. IEEE Access 10, 11947–11956 (2022)

    Article  MATH  Google Scholar 

Download references

Acknowledgements

This research has been funded by the Federal Ministry of Education and Research of Germany and the state of North-Rhine Westphalia as part of the Lamarr-Institute for Machine Learning and Artificial Intelligence and by the Federal Ministry of Education and Research of Germany under grant no. 01IS22094E WEST-AI.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Andreas Roth .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sanders, C., Roth, A., Liebig, T. (2025). Curvature-Based Pooling Within Graph Neural Networks. In: Meo, R., Silvestri, F. (eds) Machine Learning and Principles and Practice of Knowledge Discovery in Databases. ECML PKDD 2023. Communications in Computer and Information Science, vol 2135. Springer, Cham. https://doi.org/10.1007/978-3-031-74633-8_35

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-74633-8_35

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-74632-1

  • Online ISBN: 978-3-031-74633-8

  • eBook Packages: Artificial Intelligence (R0)

Publish with us

Policies and ethics