Abstract
Distributed learning has emerged as a useful tool for analyzing data stored in multiple geographic locations, especially when the distributed data sets are large and hard to move around, or the data owner is reluctant to put data into the Cloud due to privacy concerns. In distributed learning, only the locally computed models are uploaded to the fusion server, which however may still cause privacy issues since the fusion server could implement various inference attacks from its observations. To address this problem, we propose a secure distributed learning system that aims to utilize the additive property of partial homomorphic encryption to prevent direct exposure of the computed models to the fusion server. Furthermore, we propose two optimization mechanisms for applying partial homomorphic encryption to model parameters in order to improve the overall efficiency. Through experimental analysis, we demonstrate the effectiveness of our proposed mechanisms in practical distributed learning systems. Furthermore, we analyze the relationship between the computational time in the training process and several important system parameters, which can serve as a useful guide for selecting proper parameters for balancing the trade-off among model accuracy, model security and system overhead.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
McMahan, H.B., Moore, E., Ramage, D., Hampson, S.: Communication-efficient learning of deep networks from decentralized data. arXiv preprint arXiv:1602.05629 (2016)
Bonawitz, K., et al.: Practical secure aggregation for federated learning on user-held data. arXiv preprint arXiv:1611.04482 (2016)
Verma, D., Julier, S., Cirincione, G.: Federated AI for building AI solutions across multiple agencies. In: AAAI FSS-18: Artificial Intelligence in Government and Public Sector, Arlington, VA, USA (2018)
Wang, S., et al.: When edge meets learning: adaptive control for resource-constrained distributed machine learning. In: IEEE International Conference on Computer Communications (2018)
Verma, D., Chakraborty, S., Calo, S., Julier, S., Pasteris, S.: An algorithm for model fusion for distributed learning. In: Ground/Air Multisensor Interoperability, Integration, and Networking for Persistent ISR IX, vol. 10635, p. 106350O. International Society for Optics and Photonics (2018)
Li, M., et al.: Scaling distributed machine learning with the parameter server. In: USENIX Symposium on Operating Systems Design and Implementation (OSDI), vol. 14, pp. 583–598 (2014)
Kraska, T., Talwalkar, A., Duchi, J.: MLbase: a distributed machine-learning system. In: 6th Biennial Conference on Innovative Data Systems Research (CIDR 2013) (2013)
Dean, J., et al.: Large scale distributed deep networks. In: Advances in Neural Information Processing Systems, pp. 1223–1231 (2012)
Shokri, R., Stronati, M., Song, C., Shmatikov, V.: Membership inference attacks against machine learning models. In: 2017 IEEE Symposium on Security and Privacy (SP), pp. 3–18. IEEE (2017)
Long, Y., et al.: Understanding membership inferences on well-generalized learning models. arXiv preprint arXiv:1802.04889 (2018)
Gentry, C.: A fully homomorphic encryption scheme. Stanford University (2009)
Paillier, P.: Public-key cryptosystems based on composite degree residuosity classes. In: Stern, J. (ed.) EUROCRYPT 1999. LNCS, vol. 1592, pp. 223–238. Springer, Heidelberg (1999). https://doi.org/10.1007/3-540-48910-X_16
ElGamal, T.: A public key cryptosystem and a signature scheme based on discrete logarithms. IEEE Trans. Inf. Theory 31(4), 469–472 (1985)
Nakano, K., Olariu, S.: A survey on leader election protocols for radio networks. In: Proceedings. International Symposium on Parallel Architectures, Algorithms and Networks, I-SPAN 2002, pp. 71–76. IEEE (2002)
Gupta, S., Agrawal, A., Gopalakrishnan, K., Narayanan, P.: Deep learning with limited numerical precision. In: International Conference on Machine Learning, pp. 1737–1746 (2015)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Snoek, J., Larochelle, H., Adams, R.P.: Practical Bayesian optimization of machine learning algorithms. In: Advances in Neural Information Processing Systems, pp. 2951–2959 (2012)
https://blog.n1analytics.com/distributed-machine-learning-and-partially-homomorphic-encryption-1/
Fredrikson, M., Jha, S., Ristenpart, T.: Model inversion attacks that exploit confidence information and basic countermeasures. In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 1322–1333. ACM (2015)
Goldreich, O.: Secure multi-party computation. Manuscript. Preliminary version 78 (1998)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Liu, C., Chakraborty, S., Verma, D. (2019). Secure Model Fusion for Distributed Learning Using Partial Homomorphic Encryption. In: Calo, S., Bertino, E., Verma, D. (eds) Policy-Based Autonomic Data Governance. Lecture Notes in Computer Science(), vol 11550. Springer, Cham. https://doi.org/10.1007/978-3-030-17277-0_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-17277-0_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-17276-3
Online ISBN: 978-3-030-17277-0
eBook Packages: Computer ScienceComputer Science (R0)