Skip to main content
SpringerLink
Log in

Threats to Federated Learning

  • Chapter
  • First Online:
Federated Learning

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12500))

Abstract

As data are increasingly being stored in different silos and societies becoming more aware of data privacy issues, the traditional centralized approach of training artificial intelligence (AI) models is facing strong challenges. Federated learning (FL) has recently emerged as a promising solution under this new reality. Existing FL protocol design has been shown to exhibit vulnerabilities which can be exploited by adversaries both within and outside of the system to compromise data privacy. It is thus of paramount importance to make FL system designers aware of the implications of future FL algorithm design on privacy-preservation. Currently, there is no survey on this topic. In this chapter, we bridge this important gap in FL literature. By providing a concise introduction to the concept of FL, and a unique taxonomy covering threat models and two major attacks on FL: 1) poisoning attacks and 2) inference attacks, we provide an accessible review of this important topic. We highlight the intuitions, key techniques as well as fundamental assumptions adopted by various attacks, and discuss promising future research directions towards more robust privacy preservation in FL.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 16.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.ntu.edu.sg/home/han.yu/FL.html.

References

  1. Abadi, M., et al.: Deep learning with differential privacy. In: CCS, pp. 308–318 (2016)

    Google Scholar 

  2. Agarwal, N., Suresh, A.T., Yu, F.X.X., Kumar, S., McMahan, B.: cpSGD: communication-efficient and differentially-private distributed SGD. In: NeurIPS, pp. 7564–7575 (2018)

    Google Scholar 

  3. Aono, Y., Hayashi, T., Wang, L., Moriai, S., et al.: Privacy-preserving deep learning via additively homomorphic encryption. IEEE Trans. Inf. Forensics Secur. 13(5), 1333–1345 (2018)

    Article  Google Scholar 

  4. Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., Shmatikov, V.: How to backdoor federated learning. CoRR, arXiv:1807.00459 (2018)

  5. Barreno, M., Nelson, B., Sears, R., Joseph, A.D., Tygar, J.D.: Can machine learning be secure? In: ICCS, pp. 16–25 (2006)

    Google Scholar 

  6. Bernstein, J., Zhao, J., Azizzadenesheli, K., Anandkumar, A.: signSGD with majority vote is communication efficient and fault tolerant. CoRR, arXiv:1810.05291 (2018)

  7. Bhagoji, A.N., Chakraborty, S., Mittal, P., Calo, S.: Analyzing federated learning through an adversarial lens. CoRR, arXiv:1811.12470 (2018)

  8. Bhowmick, A., Duchi, J., Freudiger, J., Kapoor, G., Rogers, R.: Protection against reconstruction and its applications in private federated learning. CoRR, arXiv:1812.00984 (2018)

  9. Biggio, B., Nelson, B., Laskov, P.: Support vector machines under adversarial label noise. In: ACML, pp. 97–112 (2011)

    Google Scholar 

  10. Biggio, B., Nelson, B., Laskov, P.: Poisoning attacks against support vector machines. CoRR, arXiv:1206.6389 (2012)

  11. Blanchard, P., Guerraoui, R., Stainer, J., et al.: Machine learning with adversaries: Byzantine tolerant gradient descent. In: NeurIPS, pp. 119–129 (2017)

    Google Scholar 

  12. Bonawitz, K., et al.: Practical secure aggregation for privacy-preserving machine learning. In: CCS, pp. 1175–1191 (2017)

    Google Scholar 

  13. Chang, H., Shejwalkar, V., Shokri, R., Houmansadr, A.: Cronus: robust and heterogeneous collaborative learning with black-box knowledge transfer. CoRR, arXiv:1912.11279 (2019)

  14. Chen, L., Wang, H., Charles, Z., Papailiopoulos, D.: Draco: Byzantine-resilient distributed training via redundant gradients. CoRR, arXiv:1803.09877 (2018)

  15. Chen, Y., Su, L., Xu, J.: Distributed statistical machine learning in adversarial settings: Byzantine gradient descent. Proc. ACM Meas. Anal. Comput. Syst. 1(2), 44 (2017)

    Google Scholar 

  16. Fredrikson, M., Jha, S., Ristenpart, T.: Model inversion attacks that exploit confidence information and basic countermeasures. In: CCS, pp. 1322–1333 (2015)

    Google Scholar 

  17. Fung, C., Yoon, C.J., Beschastnikh, I.: Mitigating sybils in federated learning poisoning. CoRR, arXiv:1808.04866 (2018)

  18. Gao, D., Liu, Y., Huang, A., Ju, C., Yu, H., Yang, Q.: Privacy-preserving heterogeneous federated transfer learning. In: IEEE BigData (2019)

    Google Scholar 

  19. Gu, T., Dolan-Gavitt, B., Garg, S.: BadNets: identifying vulnerabilities in the machine learning model supply chain. CoRR, arXiv:1708.06733 (2017)

  20. Hardy, S., et al.: Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption. CoRR, arXiv:1711.10677 (2017)

  21. Hitaj, B., Ateniese, G., Pérez-Cruz, F.: Deep models under the GAN: information leakage from collaborative deep learning. In: CSS, pp. 603–618 (2017)

    Google Scholar 

  22. Huang, L., Joseph, A.D., Nelson, B., Rubinstein, B.I., Tygar, J.D.: Adversarial machine learning. In: Proceedings of the 4th ACM Workshop on Security and Artificial Intelligence, pp. 43–58 (2011)

    Google Scholar 

  23. Kairouz, P., et al.: Advances and open problems in federated learning. CoRR, arXiv:1912.04977 (2019)

  24. Kantarcioglu, M., Clifton, C.: Privacy-preserving distributed mining of association rules on horizontally partitioned data. IEEE Trans. Knowl. Data Eng. 16(9), 1026–1037 (2004)

    Article  Google Scholar 

  25. Li, D., Wang, J.: FedMD: heterogenous federated learning via model distillation. arXiv preprint arXiv:1910.03581 (2019)

  26. Li, H., Ota, K., Dong, M.: Learning IoT in edge: deep learning for the Internet of Things with edge computing. IEEE Netw. 32(1), 96–101 (2018)

    Article  Google Scholar 

  27. Li, T., Sahu, A.K., Talwalkar, A., Smith, V.: Federated learning: challenges, methods, and future directions. CoRR, arXiv:1908.07873 (2019)

  28. Lingjuan Lyu, X.X., Wang, Q.: Collaborative fairness in federated learning. arxiv.org/abs/2008.12161v1 (2020)

  29. Liu, Y., et al.: Fedvision: an online visual object detection platform powered by federated learning. In: IAAI (2020)

    Google Scholar 

  30. Lyu, L., Bezdek, J.C., He, X., Jin, J.: Fog-embedded deep learning for the Internet of Things. IEEE Trans. Ind. Inform. 15(7), 4206–4215 (2019)

    Article  Google Scholar 

  31. Lyu, L., Bezdek, J.C., Jin, J., Yang, Y.: FORESEEN: towards differentially private deep inference for intelligent Internet of Things. IEEE J. Sel. Areas Commun. 38, 2418–2429 (2020)

    Article  Google Scholar 

  32. Lyu, L., Li, Y., Nandakumar, K., Yu, J., Ma, X.: How to democratise and protect AI: fair and differentially private decentralised deep learning. IEEE Trans. Dependable Secur. Comput

    Google Scholar 

  33. Lyu, L., et al.: Towards fair and privacy-preserving federated deep models. IEEE Trans. Parallel Distrib. Syst. 31(11), 2524–2541 (2020)

    Article  Google Scholar 

  34. McMahan, B., Moore, E., Ramage, D., Hampson, S., y Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp. 1273–1282 (2017)

    Google Scholar 

  35. McMahan, H.B., Moore, E., Ramage, D., y Arcas, B.A.: Federated learning of deep networks using model averaging. CoRR, arXiv:1602.05629 (2016)

  36. McMahan, H.B., Ramage, D., Talwar, K., Zhang, L.: Learning differentially private recurrent language models. In: ICLR (2018)

    Google Scholar 

  37. Melis, L., Song, C., De Cristofaro, E., Shmatikov, V.: Exploiting unintended feature leakage in collaborative learning. In: SP, pp. 691–706 (2019)

    Google Scholar 

  38. Nasr, M., Shokri, R., Houmansadr, A.: Comprehensive privacy analysis of deep learning: passive and active white-box inference attacks against centralized and federated learning. In: SP, pp. 739–753 (2019)

    Google Scholar 

  39. Pan, X., Zhang, M., Ji, S., Yang, M.: Privacy risks of general-purpose language models. In: 2020 IEEE Symposium on Security and Privacy (SP), pp. 1314–1331. IEEE (2020)

    Google Scholar 

  40. Pan, X., Zhang, M., Wu, D., Xiao, Q., Ji, S., Yang, M.: Justinian’s GAAvernor: robust distributed learning with gradient aggregation agent. In: USENIX Security Symposium (2020)

    Google Scholar 

  41. Phong, L.T., Aono, Y., Hayashi, T., Wang, L., Moriai, S.: Privacy-preserving deep learning via additively homomorphic encryption. IEEE Trans. Inf. Forensics Secur. 13(5), 1333–1345 (2018)

    Article  Google Scholar 

  42. Shafahi, A., et al.: Poison frogs! Targeted clean-label poisoning attacks on neural networks. In: NeurIPS, pp. 6103–6113 (2018)

    Google Scholar 

  43. Shokri, R., Stronati, M., Song, C., Shmatikov, V.: Membership inference attacks against machine learning models. In: SP, pp. 3–18 (2017)

    Google Scholar 

  44. Su, L., Xu, J.: Securing distributed machine learning in high dimensions. CoRR, arXiv:1804.10140 (2018)

  45. Szegedy, C., et al.: Intriguing properties of neural networks. CoRR, arXiv:1312.6199 (2013)

  46. Vaidya, J., Clifton, C.: Privacy preserving association rule mining in vertically partitioned data. In: KDD, pp. 639–644 (2002)

    Google Scholar 

  47. Yang, Q., Liu, Y., Chen, T., Tong, Y.: Federated machine learning: concept and applications. ACM Trans. Intell. Syst. Technol. (TIST) 10(2), 1–19 (2019)

    Article  Google Scholar 

  48. Yang, Q., Liu, Y., Cheng, Y., Kang, Y., Chen, T., Yu, H.: Federated Learning. Morgan & Claypool Publishers, San Rafael (2019)

    Book  Google Scholar 

  49. Yin, D., Chen, Y., Ramchandran, K., Bartlett, P.: Byzantine-robust distributed learning: towards optimal statistical rates. CoRR, arXiv:1803.01498 (2018)

  50. Zhao, B., Mopuri, K.R., Bilen, H.: iDLG: improved deep leakage from gradients. CoRR, arXiv:2001.02610 (2020)

  51. Zhao, Y., et al.: Local differential privacy based federated learning for Internet of Things. arXiv preprint arXiv:2004.08856 (2020)

  52. Zhu, L., Liu, Z., Han, S.: Deep leakage from gradients. In: NeurIPS, pp. 14747–14756 (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National University of Singapore, Singapore, Singapore

    Lingjuan Lyu

  2. School of Computer Science and Engineering, Nanyang Technological University, Singapore, Singapore

    Han Yu & Jun Zhao

  3. Department of AI, WeBank, Shenzhen, China

    Qiang Yang

  4. Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Kowloon, Hong Kong

    Qiang Yang

Authors
  1. Lingjuan Lyu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Han Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jun Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qiang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lingjuan Lyu .

Editor information

Editors and Affiliations

  1. Hong Kong University of Science and Technology, Hong Kong, Hong Kong

    Qiang Yang

  2. WeBank, Shenzhen, China

    Lixin Fan

  3. Nanyang Technological University, Singapore, Singapore

    Han Yu

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Lyu, L., Yu, H., Zhao, J., Yang, Q. (2020). Threats to Federated Learning. In: Yang, Q., Fan, L., Yu, H. (eds) Federated Learning. Lecture Notes in Computer Science(), vol 12500. Springer, Cham. https://doi.org/10.1007/978-3-030-63076-8_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-63076-8_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-63075-1

  • Online ISBN: 978-3-030-63076-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation