Abstract
Federated learning has become an emerging technology to protect data privacy in the distributed learning area, by keeping each client user’s data locally. However, recent work shows that client users’ data might still be stolen (or reconstructed) directly from gradient updates. After exploring the attack and defense techniques of these data reconstruction methods, we discover that the attacker cannot steal the victim’s data unless it has prior knowledge about the victim’s data size. Thus, the attacker can hardly reconstruct any useful information without these prior knowledge. In this paper, we provide a novel data reconstruction method to obtain a high-dimensional compressed data from the gradient updates, without these prior knowledge. Experiment results show that our reconstructed data can be used to attack the model, with high attack accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Albaseer, A., Ciftler, B.S., Abdallah, M.M., Al-Fuqaha, A.I.: Exploiting unlabeled data in smart cities using federated edge learning. In: 16th International Wireless Communications and Mobile Computing Conference, IWCMC 2020, Limassol, Cyprus, pp. 1666–1671. IEEE (2020)
Hamer, J., Mohri, M., Suresh, A.T.: FedBoost: a communication-efficient algorithm for federated learning. In: III, H.D., Singh, A. (eds.) Proceedings of the 37th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 119, pp. 3973–3983. PMLR (2020)
Hanzely, F., Richtárik, P.: Federated learning of a mixture of global and local models. CoRR abs/2002.05516 (2020)
Kim, H., Park, J., Bennis, M., Kim, S.L.: Blockchained on-device federated learning. IEEE Commun. Lett. 24(6), 1279–1283 (2019)
Lin, T., Kong, L., Stich, S.U., Jaggi, M.: Ensemble distillation for robust model fusion in federated learning. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H. (eds.) Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, virtual (2020)
Reddi, S.J., et al.: Adaptive federated optimization. CoRR abs/2003.00295 (2020)
So, J., Güler, B., Avestimehr, S.: A scalable approach for privacy-preserving collaborative machine learning. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H. (eds.) Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, virtual (2020)
Sui, D., Chen, Y., Zhao, J., Jia, Y., Sun, W.: Feded: federated learning via ensemble distillation for medical relation extraction. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (2020)
Wang, H., Yurochkin, M., Sun, Y., Papailiopoulos, D.S., Khazaeni, Y.: Federated learning with matched averaging. CoRR abs/2002.06440 (2020)
Wang, T., Zhu, J., Torralba, A., Efros, A.A.: Dataset distillation. CoRR abs/1811.10959 (2018)
Wang, Z., Song, M., Zhang, Z., Song, Y., Wang, Q., Qi, H.: Beyond inferring class representatives: user-level privacy leakage from federated learning. CoRR abs/1812.00535 (2018)
Wei, K., et al.: Federated learning with differential privacy: algorithms and performance analysis. IEEE Trans. Inf. Forensics Secur. 15, 3454–3469 (2020)
Xie, M., Long, G., Shen, T., Zhou, T., Wang, X., Jiang, J.: Multi-center federated learning. CoRR abs/2005.01026 (2020)
Zhu, L., Han, S.: Deep leakage from gradients. In: Yang, Q., Fan, L., Yu, H. (eds.) Federated Learning. LNCS (LNAI), vol. 12500, pp. 17–31. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-63076-8_2
Acknowledgments
This work is supported by National Natural Science Foundation of China (No. 61802383), Research Project of Pazhou Lab for Excellent Young Scholars (No. PZL2021KF0024), Guangzhou Basic and Applied Basic Research Foundation (No. 202201010330, No. 202201020162), Guangdong Philosophy and Social Science Planning Project (No. GD19YYJ02), Research on the Supporting Technologies of the Metaverse in Cultural Media (No. PT252022039), Jiangsu Key Laboratory of Media Design and Software Technology (No. 21ST0202).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, X., Li, J., Zhang, J., Yan, J., Zhu, E., Chen, K. (2023). Data Reconstruction from Gradient Updates in Federated Learning. In: Xu, Y., Yan, H., Teng, H., Cai, J., Li, J. (eds) Machine Learning for Cyber Security. ML4CS 2022. Lecture Notes in Computer Science, vol 13655. Springer, Cham. https://doi.org/10.1007/978-3-031-20096-0_44
Download citation
DOI: https://doi.org/10.1007/978-3-031-20096-0_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20095-3
Online ISBN: 978-3-031-20096-0
eBook Packages: Computer ScienceComputer Science (R0)