Skip to main content
SpringerLink
Log in

CECMLP: New Cipher-Based Evaluating Collaborative Multi-layer Perceptron Scheme in Federated Learning

  • Conference paper
  • First Online:
Applied Cryptography and Network Security (ACNS 2021)

Part of the book series: Lecture Notes in Computer Science ((LNSC,volume 12726))

Included in the following conference series:

  • 1161 Accesses

Abstract

Due to the large volume of available datasets and powerful computing infrastructures, federated learning has been widely explored in many scenarios, e.g. medical screening, and image processing. It refers to all participants to jointly learn shared models under the orchestration of the server without exposing their datasets. In federated learning, since the data qualities of the participants are extremely diverse, reliability is used to measure the data qualities of the participants. To make the learning task liberally and non-discriminative, participants’ reliability privacy related to their data quality should be well preserved. However, the existing work assumed that the reliability of participants is transparent for the server provider, resulting in a severe challenge in practical applications. To thwart this challenge, we propose a novel federated learning scheme, which prevents each participant’s training set privacy and reliability privacy from being revealed to the public. Moreover, to further reduce the impact of unreliable participants and improve training efficiency, we design a cipher-based reliability weighted method to differentiate and intensify different contributions of the (un)reliable participants for joint model training. Security analysis shows that our proposed scheme can achieve the desired security requirements. Moreover, extensive performance evaluations demonstrate that our design achieves higher accuracy and is more robust against unreliable participants than conventional federated learning.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Crypto++ Homepage. https://www.cryptopp.com/

  2. Choi, E., Bahadori, M.T., Song, L., Stewart, W.F., Sun, J.: GRAM: graph-based attention model for healthcare representation learning. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), Halifax, pp. 787–795. ACM (2017). https://doi.org/10.1145/3097983.3098126

  3. Choi, E., Bahadori, M.T., Schuetz, A., Stewart, W.F., Sun, J.: Doctor AI: predicting clinical events via recurrent neural networks. In: Proceedings of the 1st Machine Learning in Health Care (MLHC), Los Angeles, pp. 301–318 (2016). JMLR.org

  4. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  5. Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning. In: Proceedings of NIPS Workshop on Deep Learning and Unsupervised Feature Learning (2011)

    Google Scholar 

  6. Phong, L.T., Aono, Y., Hayashi, T., Wang, L., Moriai, S.: Privacy-preserving deep learning via additively homomorphic encryption. IEEE Trans. Inf. Forensics Secur. 13(5), 1333–1345 (2018)

    Article  Google Scholar 

  7. Bonawitz, K., et al.: Practical secure aggregation for privacy-preserving machine learning. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS), pp. 1175–1191. ACM, New York (2017). https://doi.org/10.1145/3133956.3133982

  8. McMahan, B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.y.: Communication-efficient learning of deep networks from decentralized data. In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), Cambridge, MA, pp. 1273–1282. PMLR (2017)

    Google Scholar 

  9. Dwork, C.: Differential privacy: a survey of results. In: Agrawal, M., Du, D., Duan, Z., Li, A. (eds.) TAMC 2008. LNCS, vol. 4978, pp. 1–19. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-79228-4_1

    Chapter  MATH  Google Scholar 

  10. Geyer, R.C., Klein, T., Nabi, M.: Differentially private federated learning: a client level perspective. arXiv preprint (2018). arXiv:1712.07557

  11. Gilad-Bachrach, R., Dowlin, N., Laine, K.E., Naehrig, M., Wernsing, J.: CryptoNets: applying neural networks to encrypted data with high throughput and accuracy. In: Proceedings of the 33rd International Conference on Machine Learning (ICML), Cambridge, MA, pp. 201–210. PLMR (2016)

    Google Scholar 

  12. Goldreich, O.: Secure multi-party computation. Manuscript, Preliminary version (1998)

    Google Scholar 

  13. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Piscataway, NJ, pp. 770–778. IEEE (2016). https://doi.org/10.1109/CVPR.2016.90

  14. Huang, G., Liu, Z., Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Piscataway, NJ, pp. 2261–2269. IEEE(2017). https://doi.org/10.1109/CVPR.2017.243

  15. Juvekar, C., Vaikuntanathan, V., Chandrakasan, A.: GAZELLE: a low latency framework for secure neural network inference. In: Proceedings of the 27th USENIX Security Symposium (USENIX), Berkeley, pp. 1651–1669. USENIX (2018)

    Google Scholar 

  16. Hitaj, B., Ateniese, G., Perez-Cruz, F.: Deep models under the GAN: information leakage from collaborative deep learning. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS), pp. 603–618. ACM, New York (2017). https://doi.org/10.1145/3133956.3134012

  17. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of International Conference on Learning Representations (ICLR), pp. 1–14. ICLR (2015)

    Google Scholar 

  18. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017)

    Article  Google Scholar 

  19. Liu, J., Juuti, M., Lu, Y., Asokan, N.: Oblivious neural network predictions via MiniONN transformations. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS), pp. 619–631. ACM, New York (2017). https://doi.org/10.1145/3133956.3134056

  20. Mohassel, P., Zhang, Y.: SecureML: A system for scalable privacy-preserving machine learning. In: Proceedings of the 2017 IEEE Symposium on Security and Privacy (SP), Piscataway, NJ, pp. 19–38. IEEE (2017). https://doi.org/10.1109/SP.2017.12

  21. Mohassel, P., Rindal, P.: ABY3: A mixed protocol framework for machine learning. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security (CCS), pp. 35–52. ACM, New York (2018). https://doi.org/10.1145/3243734.3243760

  22. Moravčík, M., et al.: DeepStack: expert-level artificial intelligence in heads-up no-limit poker. Science 356(6337), 508–513 (2017)

    Article  MathSciNet  Google Scholar 

  23. Ma, X., et al.: Secure multiparty learning from the aggregation of locally trained models. J. Netw. Comput. Appl. 167, 102754102754 (2020)

    Google Scholar 

  24. Li, J., Kuang, X., Lin, S., Ma, X., Tang, Y.: Privacy preservation for machine learning training and classification based on homomorphic encryption schemes. Inf. Sci. 526, 166–179 (2020)

    Article  MathSciNet  Google Scholar 

  25. Paillier, P.: Public-key cryptosystems based on composite degree residuosity classes. In: Stern, J. (ed.) EUROCRYPT 1999. LNCS, vol. 1592, pp. 223–238. Springer, Heidelberg (1999). https://doi.org/10.1007/3-540-48910-X_16

    Chapter  Google Scholar 

  26. Paparrizos, J., White, R.W., Horvitz, E.: Screening for pancreatic adenocarcinoma using signals from web search logs: feasibility study and results. J. Oncol. Pract. 12(8), 737–744 (2016)

    Article  Google Scholar 

  27. Pathak, M., Rane, S., Raj, B.: Multiparty differential privacy via aggregation of locally trained classifiers. In: Proceedings of Advances in Neural Information Processing Systems 23 (NIPS), pp. 1876–1884. Curran Associates, New York (2010)

    Google Scholar 

  28. Riazi, M.S., Weinert, C., Tkachenko, O., Songhori, E.M., Schneider, T., Koushanfar, F.: Chameleon: a hybrid secure computation framework for machine learning applications. In: Proceedings of the 2018 on Asia Conference on Computer and Communications Security (ASIACCS), pp. 707–721. ACM, New York (2018). https://doi.org/10.1145/3196494.3196522

  29. Shokri, R., Shmatikov, V.: Privacy-preserving deep learning. In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (CCS), pp. 1310–1321. ACM, New York (2015). https://doi.org/10.1145/2810103.2813687

  30. Shokri, R., Stronati, M., Song, C., Shmatikov, V.: Membership inference attacks against machine learning models. In: Proceedings of the 2017 IEEE Symposium on Security and Privacy (SP), Piscataway, NJ, pp. 3–18. IEEE (2017). https://doi.org/10.1109/SP.2017.41

  31. Silver, D., et al.: Mastering the game of go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)

    Article  Google Scholar 

  32. Yang, Z., Chang, E.C., Liang, Z.: Adversarial neural network inversion via auxiliary knowledge alignment. arXiv preprint (2019). arXiv:1902.08552

  33. Zhang, X., Chen, X., Liu, J.K., Xiang, Y.: DeepPAR and DeepDPA: privacy-preserving and asynchronous deep learning for industrial IoT. IEEE Trans. Industr. Inf. 16(3), 2081–2090 (2020)

    Article  Google Scholar 

  34. Zhang, J., Zhang, Z., Xiao, X., Yang, Y., Winslett, M.: Functional mechanism: regression analysis under differential privacy. In: Proceedings of the VLDB Endowment, vol. 5, no. 11, pp. 1364–1375. ACM, New York (2012). https://doi.org/10.14778/2350229.2350253

  35. Zhao, L., Wang, Q., Zou, Q., Zhang, Q., Chen, Y.: Privacy-preserving collaborative deep learning with unreliable participants. IEEE Trans. Inf. Forensics Secur. 15, 1486–1500 (2020)

    Article  Google Scholar 

Download references

Acknowledgement

This work is supported by National Natural Science Foundation of China (No. 61902315).

Author information

Authors and Affiliations

  1. State Key Laboratory of Integrated Service Networks (ISN), Xidian University, Xi’an, 710071, China

    Yuqi Chen, Xiaoyu Zhang & Yi Xie

  2. State Key Laboratory of Cryptology, P. O. Box 5159, Beijing, 100878, China

    Yuqi Chen & Xiaoyu Zhang

  3. School of Cyberspace Security, Xi’an University of Posts and Telecommunications, Xi’an, 710121, China

    Meixia Miao

  4. School of Cyber Science and Engineering, Qufu Normal University, Qufu, 273165, China

    Xu Ma

Authors
  1. Yuqi Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoyu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yi Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Meixia Miao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xu Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoyu Zhang .

Editor information

Editors and Affiliations

  1. Waseda University, Tokyo, Japan

    Kazue Sako

  2. CISPA Helmholtz Center for Information Security, Saarbrücken, Germany

    Nils Ole Tippenhauer

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chen, Y., Zhang, X., Xie, Y., Miao, M., Ma, X. (2021). CECMLP: New Cipher-Based Evaluating Collaborative Multi-layer Perceptron Scheme in Federated Learning. In: Sako, K., Tippenhauer, N.O. (eds) Applied Cryptography and Network Security. ACNS 2021. Lecture Notes in Computer Science(), vol 12726. Springer, Cham. https://doi.org/10.1007/978-3-030-78372-3_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-78372-3_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-78371-6

  • Online ISBN: 978-3-030-78372-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation