Abstract
Distributed graph representation learning refers to the process of learning graph data representation in a distributed computing environment. In the process of distributed graph representation learning, nodes need to exchange data frequently, making data transmission crucial in this context. The content of data transmission, including plaintext data, ciphertext data, and model parameters, affects the performance, computational and communication costs, and privacy protection of distributed graph representation learning. However, there is currently a lack of comprehensive investigations into distributed graph representation learning. This paper fills this gap by conducting a detailed study on distributed graph representation learning and summarizing various methods for transmitting different types of content. We review the applications and evaluation methods of distributed graph representation learning and, through an analysis of the strengths and limitations of existing research, provide insights into the future development directions of distributed graph representation learning.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chen, C., et al.: Vertically federated graph neural network for privacy-preserving node classification. In: IJCAI, pp. 1959–1965. ijcai.org (2022)
Chen, F., Li, P., Miyazaki, T., Wu, C.: Fedgraph: federated graph learning with intelligent sampling. IEEE Trans. Parallel Distributed Syst. 33(8), 1775–1786 (2022)
Chen, J., Huang, G., Zheng, H., Yu, S., Jiang, W., Cui, C.: Graph-fraudster: adversarial attacks on graph neural network-based vertical federated learning. IEEE Trans. Comput. Soc. Syst. 10(2), 492–506 (2023)
Cheung, T.H., Dai, W., Li, S.: Fedsgc: federated simple graph convolution for node classification. In: IJCAI Workshops (2021)
Fu, X., Zhang, J., Meng, Z., King, I.: MAGNN: metapath aggregated graph neural network for heterogeneous graph embedding. In: WWW, pp. 2331–2341. ACM / IW3C2 (2020)
Guo, Y., Zhao, R., Lai, S., Fan, L., Lei, X., Karagiannidis, G.K.: Distributed machine learning for multiuser mobile edge computing systems. IEEE J. Sel. Top. Signal Process. 16(3), 460–473 (2022)
Hamilton, W.L., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: NIPS, pp. 1024–1034 (2017)
Harper, F.M., Konstan, J.A.: The movielens datasets: history and context. ACM Trans. Interact. Intell. Syst. 5(4), 19:1–19:19 (2016)
Hu, W., et al.: Open graph benchmark: datasets for machine learning on graphs. In: NeurIPS (2020)
Jia, Z., Lin, S., Gao, M., Zaharia, M., Aiken, A.: Improving the accuracy, scalability, and performance of graph neural networks with roc. In: MLSys. mlsys.org (2020)
Kaler, T., et al.: Accelerating training and inference of graph neural networks with fast sampling and pipelining. In: MLSys. mlsys.org (2022)
Klauck, H., Nanongkai, D., Pandurangan, G., Robinson, P.: Distributed computation of large-scale graph problems. In: SODA, pp. 391–410. SIAM (2015)
Li, Q., Coutino, M., Leus, G., Christensen, M.G.: Privacy-preserving distributed graph filtering. In: EUSIPCO, pp. 2155–2159. IEEE (2020)
Li, T., Sahu, A.K., Talwalkar, A., Smith, V.: Federated learning: challenges, methods, and future directions. IEEE Signal Process. Mag. 37(3), 50–60 (2020)
Liu, Y., Fang, S., Wang, L., Huan, C., Wang, R.: Neural graph collaborative filtering for privacy preservation basedon federated transfer learning. Electron. Libr. 40(6), 729–742 (2022)
Ma, L., et al.: Neugraph: Parallel deep neural network computation on large graphs. In: USENIX Annual Technical Conference, pp. 443–458. USENIX Association (2019)
Malewicz, G., et al.: Pregel: a system for large-scale graph processing. In: SIGMOD Conference, pp. 135–146. ACM (2010)
McAuley, J.J., Targett, C., Shi, Q., van den Hengel, A.: Image-based recommendations on styles and substitutes. In: SIGIR, pp. 43–52. ACM (2015)
Md, V., et al.: Distgnn: scalable distributed training for large-scale graph neural networks. In: SC, p. 76. ACM (2021)
Miao, X., et al.: P\({}^{\text{2 }}\)cg: a privacy preserving collaborative graph neural network training framework. VLDB J. 32(4), 717–736 (2023)
Namata, G., London, B., Getoor, L., Huang, B., Edu, U.: query-driven active surveying for collective classification. In: 10th International Workshop on Mining And Learning with Graphs. vol. 8, p. 1 (2012)
Ni, X., Xu, X., Lyu, L., Meng, C., Wang, W.: A vertical federated learning framework for graph convolutional network. CoRR abs/2106.11593 (2021)
Peng, L., Wang, N., Dvornek, N., Zhu, X., Li, X.: Fedni: Federated graph learning with network inpainting for population-based disease prediction. IEEE Transactions on Medical Imaging (2022)
Qiu, P., et al.: Your labels are selling you out: Relation leaks in vertical federated learning. IEEE Transactions on Dependable and Secure Computing (2022)
Ren, Y., Jie, Y., Wang, Q., Zhang, B., Zhang, C., Wei, L.: A hybrid secure computation framework for graph neural networks. In: PST, pp. 1–6. IEEE (2021)
RodrÃguez, E., Otero, B., Canal, R.: A survey of machine and deep learning methods for privacy protection in the internet of things. Sensors 23(3), 1252 (2023)
Rossi, R.A., Ahmed, N.K.: The network data repository with interactive graph analytics and visualization. In: AAAI, pp. 4292–4293. AAAI Press (2015)
Sen, P., Namata, G., Bilgic, M., Getoor, L., Gallagher, B., Eliassi-Rad, T.: Collective classification in network data. AI Mag. 29(3), 93–106 (2008)
Shao, Y., et al.: Distributed graph neural network training: a survey. CoRR abs/2211.00216 (2022)
Wang, L., et al.: Flexgraph: a flexible and efficient distributed framework for GNN training. In: EuroSys, pp. 67–82. ACM (2021)
Wang, S., Xie, J., Lu, M., Xiong, N.N.: Fedgraph-kd: an effective federated graph learning scheme based on knowledge distillation. In: BigDataSecurity/HPSC/IDS, pp. 130–134. IEEE (2023)
Wang, S., Zheng, Y., Jia, X.: Secgnn: privacy-preserving graph neural network training and inference as a cloud service. IEEE Transactions on Services Computing (2023)
Wu, B., et al.: A survey of trustworthy graph learning: Reliability, explainability, and privacy protection. arXiv preprint arXiv:2205.10014 (2022)
Wu, C., Wu, F., Cao, Y., Huang, Y., Xie, X.: Fedgnn: federated graph neural network for privacy-preserving recommendation. CoRR abs/2102.04925 (2021)
Wu, C., Wu, F., Lyu, L., Qi, T., Huang, Y., Xie, X.: A federated graph neural network framework for privacy-preserving personalization. Nat. Commun. 13(1), 3091 (2022)
Wu, N., Yu, L., Yang, X., Cheng, K.T., Yan, Z.: Federated learning with imbalanced and agglomerated data distribution for medical image classification. arXiv preprint arXiv:2206.13803 (2022)
Yang, S., Chen, W., Zhang, X., Liang, C., Wang, H., Cui, W.: A graph-based model for transmission network vulnerability analysis. IEEE Syst. J. 14(1), 1447–1456 (2020)
Yao, Y., Jin, W., Ravi, S., Joe-Wong, C.: Fedgcn: convergence and communication tradeoffs in federated training of graph convolutional networks. arXiv preprint arXiv:2201.12433 (2022)
Zhang, D., et al.: AGL: a scalable system for industrial-purpose graph machine learning. Proc. VLDB Endow. 13(12), 3125–3137 (2020)
Zhang, K., Yang, C., Li, X., Sun, L., Yiu, S.: Subgraph federated learning with missing neighbor generation. In: NeurIPS, pp. 6671–6682 (2021)
Zheng, C., et al.: Bytegnn: efficient graph neural network training at large scale. Proc. VLDB Endow. 15(6), 1228–1242 (2022)
Zhou, A.C., Qiu, R., Lambert, T., Allard, T., Ibrahim, S., Abbadi, A.E.: Pgpregel: an end-to-end system for privacy-preserving graph processing in geo-distributed data centers. In: SoCC, pp. 386–402. ACM (2022)
Acknowledgments
This work was supported by the National Key R &D Program of China No.2021YFF0900800, the NSFC No.62202279, the Shandong Provincial Key Research and Development Program (Major Scientific and Technological Innovation Project) No.2021CXGC010108, the Shandong Provincial Natural Science Foundation No.ZR2022QF018, the Shandong Provincial Outstanding Youth Science Foundation No.2023HWYQ-039, the Fundamental Research Funds of Shandong University.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zhang, H., Zhang, Y., He, W., Xu, Y., Cui, L. (2024). Towards Distributed Graph Representation Learning. In: Sun, Y., Lu, T., Wang, T., Fan, H., Liu, D., Du, B. (eds) Computer Supported Cooperative Work and Social Computing. ChineseCSCW 2023. Communications in Computer and Information Science, vol 2012. Springer, Singapore. https://doi.org/10.1007/978-981-99-9637-7_41
Download citation
DOI: https://doi.org/10.1007/978-981-99-9637-7_41
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-9636-0
Online ISBN: 978-981-99-9637-7
eBook Packages: Computer ScienceComputer Science (R0)
