Skip to main content
SpringerLink
Log in
Book cover

European Symposium on Research in Computer Security

ESORICS 2021: Computer Security – ESORICS 2021 pp 455–475Cite as

CONTRA: Defending Against Poisoning Attacks in Federated Learning

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNSC,volume 12972))

Abstract

Federated learning (FL) is an emerging machine learning paradigm. With FL, distributed data owners aggregate their model updates to train a shared deep neural network collaboratively, while keeping the training data locally. However, FL has little control over the local data and the training process. Therefore, it is susceptible to poisoning attacks, in which malicious or compromised clients use malicious training data or local updates as the attack vector to poison the trained global model. Moreover, the performance of existing detection and defense mechanisms drops significantly in a scaled-up FL system with non-iid data distributions. In this paper, we propose a defense scheme named CONTRA to defend against poisoning attacks, e.g., label-flipping and backdoor attacks, in FL systems. CONTRA implements a cosine-similarity-based measure to determine the credibility of local model parameters in each round and a reputation scheme to dynamically promote or penalize individual clients based on their per-round and historical contributions to the global model. With extensive experiments, we show that CONTRA significantly reduces the attack success rate while achieving high accuracy with the global model. Compared with a state-of-the-art (SOTA) defense, CONTRA reduces the attack success rate by 70% and reduces the global model performance degradation by 50%.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Arnold, S., Yesilbas, D.: Demystifying the effects of non-independence in federated learning. arXiv preprint arXiv:2103.11226 (2021)

  2. Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., Shmatikov, V.: How to backdoor federated learning. In: International Conference on Artificial Intelligence and Statistics, pp. 2938–2948. PMLR (2020)

    Google Scholar 

  3. Barreno, M., Nelson, B., Sears, R., Joseph, A.D., Tygar, J.D.: Can machine learning be secure? In: ACM Symposium on Information Computer and Communication security (2006)

    Google Scholar 

  4. Barreno, M., Nelson, B., Joseph, A.D., Tygar, J.D.: The security of machine learning. Mach. Learn. 81(2), 121–148 (2010). https://doi.org/10.1007/s10994-010-5188-5

    Article  MathSciNet  Google Scholar 

  5. Baruch, G., Baruch, M., Goldberg, Y.: A little is enough: circumventing defenses for distributed learning. Adv. Neural Inf. Process. Syst. 32, 8635–8645 (2019)

    Google Scholar 

  6. Beaufays, F., Rao, K., Mathews, R., Ramaswamy, S.: Federated learning for emoji prediction in a mobile keyboard (2019). https://arxiv.org/abs/1906.04329

  7. Bhagoji, A.N., Chakraborty, S., Mittal, P., Calo, S.: Model poisoning attacks in federated learning. In: Workshop on Security in Machine Learning (SecML) (2018)

    Google Scholar 

  8. Bhagoji, A.N., Chakraborty, S., Mittal, P., Calo, S.: Analyzing federated learning through an adversarial lens. In: the 36th International Conference on Machine Learning (2019)

    Google Scholar 

  9. Biggio, B., Nelson, B., Laskov, P.: Poisoning attacks against support vector machines. In: Proceedings of the 29th International Conference on International Conference on Machine Learning, pp. 1467–1474 (2012)

    Google Scholar 

  10. Blanchard, P., El Mhamdi, E.M., Guerraoui, R., Stainer, J.: Machine learning with adversaries: Byzantine tolerant gradient descent. In: the 31st International Conference on Neural Information Processing Systems, pp. 118–128 (2017)

    Google Scholar 

  11. Cao, X., Fang, M., Liu, J., Gong, N.Z.: FLTrust: byzantine-robust federated learning via trust bootstrapping (2020)

    Google Scholar 

  12. Chen, Y., Su, L., Xu, J.: Distributed statistical machine learning in adversarial settings: byzantine gradient descent. POMACS 1, 44:1-44:25 (2017)

    Google Scholar 

  13. Fang, M., Cao, X., Jia, J., Gong, N.: Local model poisoning attacks to byzantine-robust federated learning. In: 29th USENIX Security Symposium (2020)

    Google Scholar 

  14. Fung, C., Yoon, C.J.M., Beschastnikh, I.: The limitations of federated learning in sybil settings. In: 23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID), pp. 301–316 (2020)

    Google Scholar 

  15. Ganju, K., Wang, Q., Yang, W., Gunter, C.A., Borisov, N.: Property inference attacks on fully connected neural networks using permutation invariant representations. In: ACM Conference on Computer and Communication Security, pp. 619–633 (2018)

    Google Scholar 

  16. George, N.: Lending club loan data (version 3) (2019). https://www.kaggle.com/wordsforthewise/lending-club

  17. Gu, T., Dolan-Gavitt, B., Garg, S.: Badnets: identifying vulnerabilities in the machine learning model supply chain (2019)

    Google Scholar 

  18. Hsu, T.M.H., Qi, H., Brown, M.: Measuring the effects of non-identical data distribution for federated visual classification. arXiv preprint arXiv:1909.06335 (2019)

  19. Jagielski, M., Oprea, A., Biggio, B., Liu, C., Nita-Rotaru, C., Li, B.: Manipulating machine learning: poisoning attacks and countermeasures for regression learning. In: the 39th IEEE Symposium on Security and Privacy (2018)

    Google Scholar 

  20. Kaissis, G.A., Makowski, M.R., Rückert, D., Braren, R.F.: Secure, privacy-preserving and federated machine learning in medical imaging. Nat. Mach. Intell. 2(6), 305–311 (2020)

    Article  Google Scholar 

  21. Konečnỳ, J., McMahan, H.B., Yu, F.X., Richtárik, P., Suresh, A.T., Bacon, D.: Federated learning: strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492 (2016)

  22. Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)

    Google Scholar 

  23. Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  24. McMahan, B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp. 1273–1282. PMLR (2017)

    Google Scholar 

  25. McMahan, B., Ramage, D.: Federated learning: collaborative machine learning without centralized training data (2017). https://ai.googleblog.com/2017/04/federated-learning-collaborative.html

  26. Muñoz-González, L., et al.: Towards poisoning of deep learning algorithms with back-gradient optimization. In: the 10th ACM Workshop on Artificial Intelligence and Security, pp. 27–38 (2017)

    Google Scholar 

  27. Muñoz-González, L. Co, K.T., Lupu, E.C.: Byzantine-robust federated machine learning through adaptive model averaging. arXiv preprint arXiv:1909.05125 (2019)

  28. Nguyen, T.D., et al.: FLGUARD: secure and private federated learning. arXiv preprint arXiv:2101.02281 (2021)

  29. Shafahi, A., et al.: Poison frogs! targeted clean-label poisoning attacks on neural networks. In: 32nd International Conference on Neural Information Processing Systems, pp. 6106–6116 (2018)

    Google Scholar 

  30. Shejwalkar, V., Houmansadr, A.: Manipulating the byzantine: optimizing model poisoning attacks and defenses for federated learning. In: Network and Distributed Systems Security (NDSS) Symposium 2021 (2021)

    Google Scholar 

  31. Shen, S., Tople, S., Saxena, P.: Auror: defending against poisoning attacks in collaborative deep learning systems. In: In Proceedings of the 32nd Annual Conference on Computer Security Applications, ACSAC 2016, Los Angeles, CA, USA, 5–9 December 2016, pp. 508–519 (2016)

    Google Scholar 

  32. Shokri, R., Stronati, M., Song, C., Shmatikov, V.: Membership inference attacks against machine learning models. In: 2017 IEEE Symposium on Security and Privacy (SP), pp. 3–18 (2017)

    Google Scholar 

  33. Steinhardt, J., Koh, P.W., Liang, P.: Certified defenses for data poisoning attacks. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, pp. 3520–3532 (2017)

    Google Scholar 

  34. Tolpegin, V., Truex, S., Gursoy, M.E., Liu, L.: Data poisoning attacks against federated learning systems. In: Chen, L., Li, N., Liang, K., Schneider, S. (eds.) ESORICS 2020. LNCS, vol. 12308, pp. 480–501. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58951-6_24

    Chapter  Google Scholar 

  35. Wu, C., Yang, X., Zhu, S., Mitra, P.: Mitigating backdoor attacks in federated learning. arXiv preprint arXiv:2011.01767 (2020)

  36. Xie, C., Huang, K., Chen, P.Y., Li, B.: Dba: distributed backdoor attacks against federated learning. In: International Conference on Learning Representations (2020)

    Google Scholar 

  37. Yin, D., Chen, Y., Kannan, R., Bartlett, P.: Byzantine-robust distributed learning: towards optimal statistical rates. In: 35th International Conference on Machine Learning (2018)

    Google Scholar 

  38. Yurochkin, M., Agarwal, M., Ghosh, S., Greenewald, K., Hoang, N., Khazaeni, Y.: Bayesian nonparametric federated learning of neural networks. In: International Conference on Machine Learning, pp. 7252–7261. PMLR (2019)

    Google Scholar 

Download references

Acknowledgements

The authors were sponsored in part by NSF IIS-2014552, DGE-1565570, DGE-1922649, and the Ripple University Blockchain Research Initiative. The authors would like to thank the anonymous reviewers for their valuable comments and suggestions.

Author information

Authors and Affiliations

  1. The University of Kansas, Lawrence, KS, USA

    Sana Awan, Bo Luo & Fengjun Li

Authors
  1. Sana Awan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bo Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fengjun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fengjun Li .

Editor information

Editors and Affiliations

  1. Purdue University, West Lafayette, IN, USA

    Elisa Bertino

  2. National Research Center for Applied Cybersecurity ATHENE, Fraunhofer Institute for Secure Information Technology SIT, Darmstadt, Germany

    Haya Shulman

  3. National Research Center for Applied Cybersecurity ATHENE , Technische Universität Darmstadt, Fraunhofer Institute for Secure Information Technology SIT, Darmstadt, Germany

    Michael Waidner

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Awan, S., Luo, B., Li, F. (2021). CONTRA: Defending Against Poisoning Attacks in Federated Learning. In: Bertino, E., Shulman, H., Waidner, M. (eds) Computer Security – ESORICS 2021. ESORICS 2021. Lecture Notes in Computer Science(), vol 12972. Springer, Cham. https://doi.org/10.1007/978-3-030-88418-5_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-88418-5_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-88417-8

  • Online ISBN: 978-3-030-88418-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation