Skip to main content

Advertisement

LocalDGP: local degree-balanced graph partitioning for lightweight GNNs

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Graph neural networks (GNNs) have been widely employed in various fields including knowledge graphs and social networks. When dealing with large-scale graphs, traditional full-batch training methods suffer from excessive GPU memory consumption. To solve this problem, subgraph sampling methods divide the graph into multiple subgraphs and then train the GNN on each subgraph sequentially, which can reduce GPU memory consumption. However, the existing graph partitioning algorithms (e.g., METIS) require global graph information before partitioning, and consume a significant amount of memory to store this information, which is detrimental for large-scale graph partitioning. Moreover, the GNN parameters in the subgraph sampling methods are shared among all the subgraphs. The structural differences between the subgraphs and the global graph (e.g., differences in node degree distributions) will produce a gradient bias on the subgraphs, resulting in a degradation of GNN accuracy. Therefore, a local degree-balanced graph partitioning algorithm named LocalDGP is proposed in this paper. First, in LocalDGP, only the local graph information is acquired during the partitioning process, which can reduce memory consumption. Second, the nodes are balancedly partitioned into subgraphs based on degree to ensure that the subgraph structure is consistent with the global graph. Extensive experimental results on four graph datasets demonstrate that LocalDGP can improve the accuracy of the GNNs while reducing memory consumption. The code is publicly available at https://github.com/li143yf/LocalDGP.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Algorithm 1
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The data source has been described in the manuscript.

Materials Availability

The datasets and materials used during this study are available by following the links in the text.

Code Availability

The code can be obtained by contacting the author (Shengjie Li).

References

  1. Gori M, Monfardini G, Scarselli F (2005) A new model for learning in graph domains. In: Proceedings. IEEE international joint conference on neural networks (IJCNN), vol 2, pp 729–734

  2. Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv:1609.02907

  3. Velickovic P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y et al (2017) Graph attention networks. Stat 1050(20):10–48550

    Google Scholar 

  4. Jain L, Katarya R, Sachdeva S (2023) Opinion leaders for information diffusion using graph neural network in online social networks. ACM Transactions on the Web 17(2):1–37

    Article  MATH  Google Scholar 

  5. Xia L, Huang C, Xu Y, Dai P, Bo L (2024) Multi-behavior graph neural networks for recommender system. IEEE Trans Neural Netw Learn Syst 35(4):5473–5487

    Article  MATH  Google Scholar 

  6. Bongini P, Bianchini M, Scarselli F (2021) Molecular generative graph neural networks for drug discovery. Neurocomputing 450:242–252

    Article  MATH  Google Scholar 

  7. Chiang W-L, Liu X, Si S, Li Y, Bengio S, Hsieh C-J (2019) Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks. In: International conference on knowledge discovery and data mining (SIGKDD), pp 257–266

  8. Bai J, Ren Y, Zhang J (2021) Ripple walk training: a subgraph-based training framework for large and deep graph neural network. In: International joint conference on neural networks (IJCNN), pp 1–8

  9. Fey M, Lenssen JE, Weichert F, Leskovec J (2021) Gnnautoscale: Scalable and expressive graph neural networks via historical embeddings. In: Proceedings of the 38th international conference on machine learning (ICML), vol 139, pp 3294–3304

  10. Yu H, Wang L, Wang B, Liu M, Yang T, Ji S (2022) Graphfm: improving large-scale gnn training via feature momentum. In: International conference on machine learning (ICML), vol 162, pp 25684–25701

  11. Shi Z, Liang X, Wang J (2023) Lmc: fast training of gnns via subgraph sampling with provable convergence. In: The Eleventh international conference on learning representations (ICLR)

  12. Zhang Q, Sun Y, Hu Y, Wang S, Yin B (2023) A subgraph sampling method for training large-scale graph convolutional network. Inf Sci 649:119661

    Article  Google Scholar 

  13. Xie C, Yan L, Li WJ, Zhang Z (2014) Distributed power-law graph computing: theoretical and empirical analysis. In: Advances in neural information processing systems (NIPS), pp 1673–1681

  14. Lin J, Wan Y, Xu J, Qi X (2023) Long-tailed graph neural networks via graph structure learning for node classification. Appl Intell 53(17):20206–20222

    Article  Google Scholar 

  15. Chen J, Wang X, Xu X (2022) Gc-lstm: graph convolution embedded lstm for dynamic network link prediction. Appl Intell 52(7):7513–7528

    Article  MATH  Google Scholar 

  16. Wang X, Xin J, Wang Z, Chen Q, Wang Z (2023) An evolving graph convolutional network for dynamic functional brain network. Appl Intell 53(11):13261–13274

    Article  MATH  Google Scholar 

  17. Wu C, Wang X, Lian D, Xie X, Chen E (2023) A causality inspired framework for model interpretation. In: International conference on knowledge discovery and data mining (SIGKDD), pp 2731–2741

  18. Chen L, Wu L, Zhang K, Hong R, Lian D, Zhang Z, Zhou J, Wang M (2023) Improving recommendation fairness via data augmentation. In: Proceedings of the ACM web conference 2023 (WWW’23), pp 1012–1020

  19. Gao C, Wang S, Li S, Chen J, He X, Lei W, Li B, Zhang Y, Jiang P (2023) Cirs: bursting filter bubbles by counterfactual interactive recommender system. ACM Trans Inf Syst 42(1):1–27

    Article  MATH  Google Scholar 

  20. Liu M, Meng F, Liang Y (2022) Generalized pose decoupled network for unsupervised 3d skeleton sequence-based action representation learning. Cyborg and Bionic Systems 2022:0002

    Article  Google Scholar 

  21. Liu J, Wang X, Wang C, Gao Y, Liu M (2024) Temporal decoupling graph convolutional network for skeleton-based gesture recognition. IEEE Trans Multimed 26:811–823

    Article  MATH  Google Scholar 

  22. Wang X, Zhang W, Wang C, Gao Y, Liu M (2024) Dynamic dense graph convolutional network for skeleton-based human motion prediction. IEEE Trans Image Process 33:1–15

    Article  MATH  Google Scholar 

  23. Gao Q, Deng Z, Ju Z, Zhang T (2023) Dual-hand motion capture by using biological inspiration for bionic bimanual robot teleoperation. Cyborg and Bionic Systems 4:0052

    Article  Google Scholar 

  24. Zhang Y, Xu X, Zhao Y, Wen Y, Tang Z, Liu M (2024) Facial prior guided micro-expression generation. IEEE Trans Image Proces 33:525–540

    Article  Google Scholar 

  25. Dey RK, Das AK (2023) Modified term frequency-inverse document frequency based deep hybrid framework for sentiment analysis. Multimed Tools Appl 82(21):32967–32990

    Article  MATH  Google Scholar 

  26. Dey RK, Das AK (2024) Neighbour adjusted dispersive flies optimization based deep hybrid sentiment analysis framework. Multimed Tools Appl 83(24):64393–64416

    Article  MATH  Google Scholar 

  27. Xiao Z, Tong H, Qu R, Xing H, Luo S, Zhu Z, Song F, Feng L (2023) Capmatch: Semi-supervised contrastive transformer capsule with feature-based knowledge distillation for human activity recognition. IEEE Trans Neural Netw Learn Syst, pp 1–15

  28. Xiao Z, Xing H, Qu R, Feng L, Luo S, Dai P, Zhao B, Dai Y (2024) Densely knowledge-aware network for multivariate time series classification. IEEE Transactions on Systems, Man, and Cybernetics: Systems 54(4):2192–2204

    Article  MATH  Google Scholar 

  29. Xiao Z, Xu X, Xing H, Zhao B, Wang X, Song F, Qu R, Feng L (2024) Dtcm: Deep transformer capsule mutual distillation for multivariate time series classification. IEEE Transactions on Cognitive and Developmental Systems 16(4):1445–1461

    Article  Google Scholar 

  30. Xiao Z, Xing H, Qu R, Li H, Feng L, Zhao B, Yang J (2024) Self-bidirectional decoupled distillation for time series classification. IEEE Trans Artif Intell 5(8):4101–4110

    Article  MATH  Google Scholar 

  31. Stanton I, Kliot G (2012) Streaming graph partitioning for large distributed graphs. In: International conference on knowledge discovery and data mining (SIGKDD), pp 1222–1230

  32. Tsourakakis C, Gkantsidis C, Radunovic B, Vojnovic M (2014) Fennel: Streaming graph partitioning for massive scale graphs. In: ACM international conference on web search and data mining (WSDM). New York, pp 333–342

  33. Ji S, Bu C, Li L, Wu X (2023) Localtgep: a lightweight edge partitioner for time-varying graph. IEEE Trans Emerg Top Comput 12(2):455–466

    Article  MATH  Google Scholar 

  34. Kernighan BW, Lin S (1970) An efficient heuristic procedure for partitioning graphs. The Bell System Technical Journal 49(2):291–307

    Article  MATH  Google Scholar 

  35. Zhang C, Wei F, Liu Q, Tang ZG, Li Z (2017) Graph edge partitioning via neighborhood heuristic. In: International conference on knowledge discovery and data mining (SIGKDD), pp 605–614

  36. Gasteiger J, Qian C, Günnemann S (2022) Influence-based mini-batching for graph neural networks. In: Proceedings of the first learning on graphs conference, vol 198, pp 9–1. PMLR

  37. Shchur O, Mumme M, Bojchevski A, Günnemann S (2018) Pitfalls of graph neural network evaluation. arXiv:1811.05868

  38. Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. In: Proceedings of advances in neural information processing systems 30: annual conference on neural information processing systems (NIPS), vol 30, pp 1024–1034

  39. Hu W, Fey M, Zitnik M, Dong Y, Ren H, Liu B, Catasta M, Leskovec J (2020) Open graph benchmark: datasets for machine learning on graphs. In: Proceedings of advances in neural information processing systems (NIPS), vol 33, pp 22118–22133

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under grants 62306100 and 62406096, the Anhui University Natural Science Key Research Project under grant No. KJ2021A0994, the Program for Scientific Research Innovation Team in Colleges and Universities of Anhui Province under grant No. 2022AH010095, the Natural Science Research Project of Anhui Educational Committee under grant 2023AH052180, and the Talent Research Project of Hefei University under grant 23RC12.

Author information

Authors and Affiliations

Authors

Contributions

Shengwei Ji: Conceptualization, Methodology, Supervision. Shengjie Li: Software, Conducting experiments, Writing - Original draft preparation. Fei Liu and Qiang Xu: Reviewing, Investigation and Editing.

Corresponding authors

Correspondence to Fei Liu or Qiang Xu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ji, S., Li, S., Liu, F. et al. LocalDGP: local degree-balanced graph partitioning for lightweight GNNs. Appl Intell 55, 109 (2025). https://doi.org/10.1007/s10489-024-05964-3

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-05964-3

Keywords