Abstract
Nowadays, artificial intelligence is limited by privacy and security problems. Compared with the ordinary machine learning, federated learning (FL) enables multiple participants to collaboratively learn a shared machine learning model while keeping all the training data on local devices. However, most of the current secured federated learning systems (FLSs) are built up with high computational and communication costs. On the other hand, optimizing the network structure of federated learning systems can reduce communication complexity by considering the correlation of the transmission channels.
In this paper, we propose Network Coding Federated Learning Systems (NC-FLSs). Specifically, it considers the whole communication network by connecting all the clients and the server. Applying a linear NC scheme to construct a linear combination of the original messages, which is transmitted over the network instead of the messages themselves. Based on NC-FLSs, the communication cost is halved and both data privacy and security are improved with the imperceptibly higher computational cost. Moreover, considering that the network coding structure is independent of the FL model, any FLSs can also be upgraded to its corresponding NC-FLSs. We also implement differential privacy on an NC-FLS to train an image classifier while keeping clients’ local data secure and private, which achieves superior performance and efficiency.
This paper is supported by National Key Research and Development Program of China under grant No. 2018YFB1003500, No. 2018YFB0204400 and No. 2017YFB1401202.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abadi, M., et al.: Deep learning with differential privacy. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, CCS ’16, ACM, New York, NY, USA, pp. 308–318 (2016). https://doi.org/10.1145/2976749.2978318
Atayero, A.A., Feyisetan, O.: Security issues in cloud computing: the potentials of homomorphic encryption. J. Emerg. Trend Comput. Inf. Sci. 2(10), 546–552 (2011)
Bhattad, K., et al.: Weakly secure network coding. NetCod, Apr 104 (2005)
Dean, J., et al.: Large scale distributed deep networks. In: Advances in Neural Information Processing Systems, pp. 1223–1231 (2012)
Dimakis, A.G., Godfrey, P.B., Wu, Y., Wainwright, M.J., Ramchandran, K.: Network coding for distributed storage systems. IEEE Trans. Inf. Theory 56(9), 4539–4551 (2010)
Dwork, C.: Differential privacy. In: Encyclopedia of Cryptography and Security, pp. 338–340 (2011)
Dwork, C., Kenthapadi, K., McSherry, F., Mironov, I., Naor, M.: Our data, ourselves: privacy via distributed noise generation. In: Vaudenay, S. (ed.) EUROCRYPT 2006. LNCS, vol. 4004, pp. 486–503. Springer, Heidelberg (2006). https://doi.org/10.1007/11761679_29
Fragouli, C., Markopoulou, A.: A network coding approach to overlay network monitoring. In: Allerton Conference (2005)
Han, C., Yin, J., Ye, L., Ke, Y., Yang, Y.: An integrated fast data transmission scheme based on network coding. IEEE Access 7, 112216–112228 (2019)
Kairouz, P., et al.: Advances and open problems in federated learning. arXiv preprint arXiv:1912.04977 (2019)
KoneÄŤnĂ˝, J., McMahan, H.B., Ramage, D., Richtarik, P.: Federated optimization: distributed machine learning for on-device intelligence. arXiv preprint arXiv:1610.02527 (2016)
Li, S.Y., Yeung, R.W., Cai, N.: Linear network coding. IEEE Trans. Inf. Theory 49(2), 371–381 (2003)
MartĂnez-Peñas, U., Kschischang, F.R.: Reliable and secure multishot network coding using linearized reed-solomon codes. IEEE Trans. Inf. Theory 65(8), 4785–4803 (2019)
McMahan, B., Ramage, D.: Federated learning: collaborative machine learning without centralized training data. Google Research Blog 3, (2017)
Naeem, A., Rehmani, M.H., Saleem, Y., Rashid, I., Crespi, N.: Network coding in cognitive radio networks: a comprehensive survey. IEEE Commun. Surv. Tutorials 19(3), 1945–1973 (2017)
Phong, L.T., Aono, Y., Hayashi, T., Wang, L., Moriai, S.: Privacy-preserving deep learning via additively homomorphic encryption. IEEE Trans. Inf. Forensics Secur. 13(5), 1333–1345 (2018). https://doi.org/10.1109/TIFS.2017.2787987
Recht, B., et al.: Hogwild: a lock-free approach to parallelizing stochastic gradient descent. In: Advances in Neural Information Processing Systems, pp. 693–701 (2011)
Shokri, R., Shmatikov, V.: Privacy-preserving deep learning. In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 1310–1321. ACM (2015)
Yang, Q., Liu, Y., Chen, T., Tong, Y.: Federated machine learning: concept and applications. ACM Trans. Intell. Syst. Technol. (TIST) 10(2), 12 (2019)
Acknowledgement
This paper is supported by National Key Research and Development Program of China under grant No. 2018YFB1003500, No. 2018YFB0204400 and No. 2017YFB1401202.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Kong, L., Tao, H., Wang, J., Huang, Z., Xiao, J. (2020). Network Coding for Federated Learning Systems. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Lecture Notes in Computer Science(), vol 12533. Springer, Cham. https://doi.org/10.1007/978-3-030-63833-7_46
Download citation
DOI: https://doi.org/10.1007/978-3-030-63833-7_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63832-0
Online ISBN: 978-3-030-63833-7
eBook Packages: Computer ScienceComputer Science (R0)