Abstract
Graph Neural Networks (GNNs) are one of the primary methods for molecular property prediction due to their ability to learn state-of-the-art level representations from graph-structured molecular data. In addition, the Federated Learning (FL) paradigm, which allows multiple ends to collaborate on machine learning training without sharing local data, is being considered for introduction to improve the performance of multiple ends. However, in FL, the molecular graph data among clients are not only Non-Independent Identically Distribution (Non-IID) but also skewed in quantity distribution. In this paper, we propose the GFedKRL framework to perform knowledge distillation and re-learning during the interaction between clients and servers in each cluster after clustering the graph embeddings uploaded. We also analyze the risk of privacy leakage in the GFedKRL and propose personalized local differential privacy to protect privacy while better controlling the amount of noise input and improving model performance. In addition, to resist the impact of noise data on the clients’ model, graph representation learning is enhanced by knowledge contrast learning at the local clients. Finally, our approach achieves better results in three experimental datasets compared with four public benchmark methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ma, Y., Wang, S., Aggarwal, C.C., Tang, J.: Graph convolutional networks with EigenPooling. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (2019)
Xie, H., Ma, J., Xiong, L., Yang, C.: Federated graph classification over non-IID graphs. ArXiv, abs/2106.13423 (2021)
Wu, B., Yang, X., Pan, S., Yuan, X.: Adapting membership inference attacks to GNN for graph classification: approaches and implications. In: 2021 IEEE International Conference on Data Mining (ICDM), pp. 1421–1426 (2021)
Kang, J., Xiong, Z., Niyato, D.T., Zou, Y., Zhang, Y., Guizani, M.: Reliable federated learning for mobile networks. IEEE Wirel. Commun. 27, 72–80 (2019)
Lim, W.Y.B., et al.: Hierarchical incentive mechanism design for federated machine learning in mobile networks. IEEE Internet Things J. 7, 9575–9588 (2020)
Geiping, J., Bauermeister, H., Dröge, H., Moeller, M.: Inverting gradients - how easy is it to break privacy in federated learning? ArXiv, abs/2003.14053 (2020)
Dwork, C., Roth, A.: The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci. 9, 211–407 (2014)
Li, D., Wang, J., Li, Q., Yuhang, H., Li, X.: A privacy preservation framework for feedforward-designed convolutional neural networks. Neural Netw. Off. J. Int. Neural Netw. Soc. 155, 14–27 (2022)
Sajadmanesh, S., Gática-Pérez, D.: Locally private graph neural networks. In: Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security (2020)
Lin, W., Li, B., Wang, C.: Towards private learning on decentralized graphs with local differential privacy. IEEE Trans. Inf. Forensics Secur. 17, 2936–2946 (2022)
Qiu, Y., Huang, C., Wang, J., Huang, Z., Xiao, J.: A privacy-preserving subgraph-level federated graph neural network via differential privacy. In: Knowledge Science, Engineering and Management (2022)
Wei, Y., et al.: Heterogeneous graph neural network for privacy-preserving recommendation. In: 2022 IEEE International Conference on Data Mining (ICDM), pp. 528–537 (2022)
He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.B.: Momentum contrast for unsupervised visual representation learning. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9726–9735 (2019)
Shokri, R., Stronati, M., Song, C., Shmatikov, V.: Membership inference attacks against machine learning models. In: 2017 IEEE Symposium on Security and Privacy (SP), pp. 3–18 (2016)
Wang, T., Blocki, J., Li, N., Jha, S.: Locally differentially private protocols for frequency estimation. In: USENIX Security Symposium (2017)
Mangat, N.S.: An improved randomized response strategy. J. Royal Stat. Soc. Ser. B Methodol. 56(1), 93–95 (1994)
McMahan, H.B., Moore, E., Ramage, D., Hampson, S., y Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: International Conference on Artificial Intelligence and Statistics (2016)
Sahu, A.K., Li, T., Sanjabi, M., Zaheer, M., Talwalkar, A.S., Smith, V.: Federated optimization in heterogeneous networks. arXiv: Learning (2018)
Acknowledgment
This paper was supported by the National Natural Science Foundation of China (Nos. 62162005 and U21A20474), Guangxi Science and Technology Project (GuikeAA22067070), Innovation Project of Guangxi Graduate Education (YCBZ2023058), Guangxi “Bagui Scholar” Teams for Innovation and Research Project, and Guangxi Collaborative Innovation Center of Multisource Information Integration and Intelligent Processing.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ning, Y., Wang, J., Li, D., Yan, D., Li, X. (2023). GFedKRL: Graph Federated Knowledge Re-Learning for Effective Molecular Property Prediction via Privacy Protection. In: Iliadis, L., Papaleonidas, A., Angelov, P., Jayne, C. (eds) Artificial Neural Networks and Machine Learning – ICANN 2023. ICANN 2023. Lecture Notes in Computer Science, vol 14256. Springer, Cham. https://doi.org/10.1007/978-3-031-44213-1_36
Download citation
DOI: https://doi.org/10.1007/978-3-031-44213-1_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44212-4
Online ISBN: 978-3-031-44213-1
eBook Packages: Computer ScienceComputer Science (R0)