Abstract
Graph neural networks (GNNs) have been widely employed in various fields including knowledge graphs and social networks. When dealing with large-scale graphs, traditional full-batch training methods suffer from excessive GPU memory consumption. To solve this problem, subgraph sampling methods divide the graph into multiple subgraphs and then train the GNN on each subgraph sequentially, which can reduce GPU memory consumption. However, the existing graph partitioning algorithms (e.g., METIS) require global graph information before partitioning, and consume a significant amount of memory to store this information, which is detrimental for large-scale graph partitioning. Moreover, the GNN parameters in the subgraph sampling methods are shared among all the subgraphs. The structural differences between the subgraphs and the global graph (e.g., differences in node degree distributions) will produce a gradient bias on the subgraphs, resulting in a degradation of GNN accuracy. Therefore, a local degree-balanced graph partitioning algorithm named LocalDGP is proposed in this paper. First, in LocalDGP, only the local graph information is acquired during the partitioning process, which can reduce memory consumption. Second, the nodes are balancedly partitioned into subgraphs based on degree to ensure that the subgraph structure is consistent with the global graph. Extensive experimental results on four graph datasets demonstrate that LocalDGP can improve the accuracy of the GNNs while reducing memory consumption. The code is publicly available at https://github.com/li143yf/LocalDGP.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
The data source has been described in the manuscript.
Materials Availability
The datasets and materials used during this study are available by following the links in the text.
Code Availability
The code can be obtained by contacting the author (Shengjie Li).
References
Gori M, Monfardini G, Scarselli F (2005) A new model for learning in graph domains. In: Proceedings. IEEE international joint conference on neural networks (IJCNN), vol 2, pp 729–734
Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv:1609.02907
Velickovic P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y et al (2017) Graph attention networks. Stat 1050(20):10–48550
Jain L, Katarya R, Sachdeva S (2023) Opinion leaders for information diffusion using graph neural network in online social networks. ACM Transactions on the Web 17(2):1–37
Xia L, Huang C, Xu Y, Dai P, Bo L (2024) Multi-behavior graph neural networks for recommender system. IEEE Trans Neural Netw Learn Syst 35(4):5473–5487
Bongini P, Bianchini M, Scarselli F (2021) Molecular generative graph neural networks for drug discovery. Neurocomputing 450:242–252
Chiang W-L, Liu X, Si S, Li Y, Bengio S, Hsieh C-J (2019) Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks. In: International conference on knowledge discovery and data mining (SIGKDD), pp 257–266
Bai J, Ren Y, Zhang J (2021) Ripple walk training: a subgraph-based training framework for large and deep graph neural network. In: International joint conference on neural networks (IJCNN), pp 1–8
Fey M, Lenssen JE, Weichert F, Leskovec J (2021) Gnnautoscale: Scalable and expressive graph neural networks via historical embeddings. In: Proceedings of the 38th international conference on machine learning (ICML), vol 139, pp 3294–3304
Yu H, Wang L, Wang B, Liu M, Yang T, Ji S (2022) Graphfm: improving large-scale gnn training via feature momentum. In: International conference on machine learning (ICML), vol 162, pp 25684–25701
Shi Z, Liang X, Wang J (2023) Lmc: fast training of gnns via subgraph sampling with provable convergence. In: The Eleventh international conference on learning representations (ICLR)
Zhang Q, Sun Y, Hu Y, Wang S, Yin B (2023) A subgraph sampling method for training large-scale graph convolutional network. Inf Sci 649:119661
Xie C, Yan L, Li WJ, Zhang Z (2014) Distributed power-law graph computing: theoretical and empirical analysis. In: Advances in neural information processing systems (NIPS), pp 1673–1681
Lin J, Wan Y, Xu J, Qi X (2023) Long-tailed graph neural networks via graph structure learning for node classification. Appl Intell 53(17):20206–20222
Chen J, Wang X, Xu X (2022) Gc-lstm: graph convolution embedded lstm for dynamic network link prediction. Appl Intell 52(7):7513–7528
Wang X, Xin J, Wang Z, Chen Q, Wang Z (2023) An evolving graph convolutional network for dynamic functional brain network. Appl Intell 53(11):13261–13274
Wu C, Wang X, Lian D, Xie X, Chen E (2023) A causality inspired framework for model interpretation. In: International conference on knowledge discovery and data mining (SIGKDD), pp 2731–2741
Chen L, Wu L, Zhang K, Hong R, Lian D, Zhang Z, Zhou J, Wang M (2023) Improving recommendation fairness via data augmentation. In: Proceedings of the ACM web conference 2023 (WWW’23), pp 1012–1020
Gao C, Wang S, Li S, Chen J, He X, Lei W, Li B, Zhang Y, Jiang P (2023) Cirs: bursting filter bubbles by counterfactual interactive recommender system. ACM Trans Inf Syst 42(1):1–27
Liu M, Meng F, Liang Y (2022) Generalized pose decoupled network for unsupervised 3d skeleton sequence-based action representation learning. Cyborg and Bionic Systems 2022:0002
Liu J, Wang X, Wang C, Gao Y, Liu M (2024) Temporal decoupling graph convolutional network for skeleton-based gesture recognition. IEEE Trans Multimed 26:811–823
Wang X, Zhang W, Wang C, Gao Y, Liu M (2024) Dynamic dense graph convolutional network for skeleton-based human motion prediction. IEEE Trans Image Process 33:1–15
Gao Q, Deng Z, Ju Z, Zhang T (2023) Dual-hand motion capture by using biological inspiration for bionic bimanual robot teleoperation. Cyborg and Bionic Systems 4:0052
Zhang Y, Xu X, Zhao Y, Wen Y, Tang Z, Liu M (2024) Facial prior guided micro-expression generation. IEEE Trans Image Proces 33:525–540
Dey RK, Das AK (2023) Modified term frequency-inverse document frequency based deep hybrid framework for sentiment analysis. Multimed Tools Appl 82(21):32967–32990
Dey RK, Das AK (2024) Neighbour adjusted dispersive flies optimization based deep hybrid sentiment analysis framework. Multimed Tools Appl 83(24):64393–64416
Xiao Z, Tong H, Qu R, Xing H, Luo S, Zhu Z, Song F, Feng L (2023) Capmatch: Semi-supervised contrastive transformer capsule with feature-based knowledge distillation for human activity recognition. IEEE Trans Neural Netw Learn Syst, pp 1–15
Xiao Z, Xing H, Qu R, Feng L, Luo S, Dai P, Zhao B, Dai Y (2024) Densely knowledge-aware network for multivariate time series classification. IEEE Transactions on Systems, Man, and Cybernetics: Systems 54(4):2192–2204
Xiao Z, Xu X, Xing H, Zhao B, Wang X, Song F, Qu R, Feng L (2024) Dtcm: Deep transformer capsule mutual distillation for multivariate time series classification. IEEE Transactions on Cognitive and Developmental Systems 16(4):1445–1461
Xiao Z, Xing H, Qu R, Li H, Feng L, Zhao B, Yang J (2024) Self-bidirectional decoupled distillation for time series classification. IEEE Trans Artif Intell 5(8):4101–4110
Stanton I, Kliot G (2012) Streaming graph partitioning for large distributed graphs. In: International conference on knowledge discovery and data mining (SIGKDD), pp 1222–1230
Tsourakakis C, Gkantsidis C, Radunovic B, Vojnovic M (2014) Fennel: Streaming graph partitioning for massive scale graphs. In: ACM international conference on web search and data mining (WSDM). New York, pp 333–342
Ji S, Bu C, Li L, Wu X (2023) Localtgep: a lightweight edge partitioner for time-varying graph. IEEE Trans Emerg Top Comput 12(2):455–466
Kernighan BW, Lin S (1970) An efficient heuristic procedure for partitioning graphs. The Bell System Technical Journal 49(2):291–307
Zhang C, Wei F, Liu Q, Tang ZG, Li Z (2017) Graph edge partitioning via neighborhood heuristic. In: International conference on knowledge discovery and data mining (SIGKDD), pp 605–614
Gasteiger J, Qian C, Günnemann S (2022) Influence-based mini-batching for graph neural networks. In: Proceedings of the first learning on graphs conference, vol 198, pp 9–1. PMLR
Shchur O, Mumme M, Bojchevski A, Günnemann S (2018) Pitfalls of graph neural network evaluation. arXiv:1811.05868
Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. In: Proceedings of advances in neural information processing systems 30: annual conference on neural information processing systems (NIPS), vol 30, pp 1024–1034
Hu W, Fey M, Zitnik M, Dong Y, Ren H, Liu B, Catasta M, Leskovec J (2020) Open graph benchmark: datasets for machine learning on graphs. In: Proceedings of advances in neural information processing systems (NIPS), vol 33, pp 22118–22133
Acknowledgements
This work was supported by the National Natural Science Foundation of China under grants 62306100 and 62406096, the Anhui University Natural Science Key Research Project under grant No. KJ2021A0994, the Program for Scientific Research Innovation Team in Colleges and Universities of Anhui Province under grant No. 2022AH010095, the Natural Science Research Project of Anhui Educational Committee under grant 2023AH052180, and the Talent Research Project of Hefei University under grant 23RC12.
Author information
Authors and Affiliations
Contributions
Shengwei Ji: Conceptualization, Methodology, Supervision. Shengjie Li: Software, Conducting experiments, Writing - Original draft preparation. Fei Liu and Qiang Xu: Reviewing, Investigation and Editing.
Corresponding authors
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ji, S., Li, S., Liu, F. et al. LocalDGP: local degree-balanced graph partitioning for lightweight GNNs. Appl Intell 55, 109 (2025). https://doi.org/10.1007/s10489-024-05964-3
Accepted:
Published:
DOI: https://doi.org/10.1007/s10489-024-05964-3