Skip to main content
SpringerLink
Log in
Book cover

International Conference on Advanced Data Mining and Applications

ADMA 2022: Advanced Data Mining and Applications pp 238–252Cite as

Personalized Federated Learning with Robust Clustering Against Model Poisoning

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13726))

Abstract

Recently, federated Learning (FL) has been widely used to protect clients’ data privacy in distributed applications, and heterogeneous data and model poisoning are two critical challenges to attack. To tackle the first challenge that data of each client is usually not independent or identically distributed, personalized FL (PFL) or clustered FL, which can be seen as a cluster-wise PFL method to learn multiple models across clients or clusters. To detect the anomaly clients or outliers, local outlier factor is a popular method based on the density of data points. Therefore, a nested bi-level optimization objective is constructed, and an algorithm of PFL with robust clustering called FedPRC is proposed to detect outliers and maintain state-of-the-art performance. The breakdown point of FedPRC can be at least 0.5. Our experimental analysis has demonstrated effectiveness and superior performance in comparison with baselines in multiple benchmark datasets.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    http://www.nist.gov/itl/products-and-services/emnist-dataset.

  2. 2.

    http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html.

References

  1. Ana, L.F., Jain, A.K.: Robust data clustering. In: Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, p. II. IEEE (2003)

    Google Scholar 

  2. Arthur, D., Vassilvitskii, S.: K-means++: the advantages of careful seeding. Technical report, Stanford (2006)

    Google Scholar 

  3. Bhagoji, A.N., Chakraborty, S., Mittal, P., Calo, S.: Analyzing federated learning through an adversarial lens. In: International Conference on Machine Learning, pp. 634–643. PMLR (2019)

    Google Scholar 

  4. Blanchard, P., El Mhamdi, E.M., Guerraoui, R., Stainer, J.: Machine learning with adversaries: Byzantine tolerant gradient descent. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  5. Breunig, M.M., Kriegel, H.P., Ng, R.T., Sander, J.: LOF: identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp. 93–104 (2000)

    Google Scholar 

  6. Brunet-Saumard, C., Genetay, E., Saumard, A.: K-bMOM: a robust Lloyd-type clustering algorithm based on bootstrap median-of-means. Comput. Stat. Data Anal. 167, 107370 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  7. Caldas, S., et al.: Leaf: a benchmark for federated settings. arXiv preprint arXiv:1812.01097 (2018)

  8. Chen, F., Luo, M., Dong, Z., Li, Z., He, X.: Federated meta-learning with fast convergence and efficient communication. arXiv preprint arXiv:1802.07876 (2018)

  9. Chen, X., Liu, C., Li, B., Lu, K., Song, D.: Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526 (2017)

  10. Cheng, G., Chadha, K., Duchi, J.: Fine-tuning is fine in federated learning. arXiv preprint arXiv:2108.07313 (2021)

  11. Cohen, G., Afshar, S., Tapson, J., Van Schaik, A.: EMNIST: extending MNIST to handwritten letters. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 2921–2926. IEEE (2017)

    Google Scholar 

  12. Collins, L., Hassani, H., Mokhtari, A., Shakkottai, S.: Exploiting shared representations for personalized federated learning. In: International Conference on Machine Learning, pp. 2089–2099. PMLR (2021)

    Google Scholar 

  13. Damaskinos, G., El-Mhamdi, E.M., Guerraoui, R., Guirguis, A., Rouault, S.: Aggregathor: Byzantine machine learning via robust gradient aggregation. Proc. Mach. Learn. Syst. 1, 81–106 (2019)

    Google Scholar 

  14. Davé, R.N., Krishnapuram, R.: Robust clustering methods: a unified view. IEEE Trans. Fuzzy Syst. 5(2), 270–293 (1997)

    Article  Google Scholar 

  15. Deshpande, A., Kacham, P., Pratap, R.: Robust \(k\)-means++. In: Conference on Uncertainty in Artificial Intelligence, pp. 799–808. PMLR (2020)

    Google Scholar 

  16. Apple Differential Privacy Team: Learning with privacy at scale (2017). https://machinelearning.apple.com/research/learning-with-privacy-at-scale

  17. Fallah, A., Mokhtari, A., Ozdaglar, A.: Personalized federated learning with theoretical guarantees: a model-agnostic meta-learning approach. In: Advances in Neural Information Processing Systems, vol. 33, pp. 3557–3568 (2020)

    Google Scholar 

  18. García-Escudero, L.A., Gordaliza, A., Matrán, C., Mayo-Iscar, A.: A general trimming approach to robust cluster analysis. Ann. Stat. 36(3), 1324–1345 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  19. García-Escudero, L.A., Gordaliza, A., Matrán, C., Mayo-Iscar, A.: A review of robust clustering methods. Adv. Data Anal. Classif. 4(2), 89–109 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  20. Ghosh, A., Chung, J., Yin, D., Ramchandran, K.: An efficient framework for clustered federated learning. arXiv preprint arXiv:2006.04088 (2020)

  21. Ghosh, A., Hong, J., Yin, D., Ramchandran, K.: Robust federated learning in a heterogeneous environment. arXiv preprint arXiv:1906.06629 (2019)

  22. Guha, S., Rastogi, R., Shim, K.: Rock: a robust clustering algorithm for categorical attributes. Inf. Syst. 25(5), 345–366 (2000)

    Article  Google Scholar 

  23. He, C., et al.: FedCV: a federated learning framework for diverse computer vision tasks. arXiv preprint arXiv:2111.11066 (2021)

  24. Jallepalli, D., Ravikumar, N.C., Badarinath, P.V., Uchil, S., Suresh, M.A.: Federated learning for object detection in autonomous vehicles. In: 2021 IEEE Seventh International Conference on Big Data Computing Service and Applications (BigDataService), pp. 107–114. IEEE (2021)

    Google Scholar 

  25. Li, T., Hu, S., Beirami, A., Smith, V.: Ditto: fair and robust federated learning through personalization. In: International Conference on Machine Learning, pp. 6357–6368. PMLR (2021)

    Google Scholar 

  26. Li, T., Sahu, A.K., Zaheer, M., Sanjabi, M., Talwalkar, A., Smith, V.: Federated optimization in heterogeneous networks. arXiv preprint arXiv:1812.06127 (2018)

  27. Li, T., Sahu, A.K., Zaheer, M., Sanjabi, M., Talwalkar, A., Smithy, V.: FedDANE: a federated newton-type method. In: 2019 53rd Asilomar Conference on Signals, Systems, and Computers, pp. 1227–1231. IEEE (2019)

    Google Scholar 

  28. Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of the IEEE ICCV, pp. 3730–3738 (2015)

    Google Scholar 

  29. Long, G., Shen, T., Tan, Y., Gerrard, L., Clarke, A., Jiang, J.: Federated learning for privacy-preserving open innovation future on digital health. In: Chen, F., Zhou, J. (eds.) Humanity Driven AI, pp. 113–133. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-72188-6_6

    Chapter  Google Scholar 

  30. Luo, J., et al.: Real-world image datasets for federated learning. arXiv preprint arXiv:1910.11089 (2019)

  31. Ma, J., Long, G., Zhou, T., Jiang, J., Zhang, C.: On the convergence of clustered federated learning. arXiv preprint arXiv:2202.06187 (2022)

  32. Mansour, Y., Mohri, M., Ro, J., Suresh, A.T.: Three approaches for personalization with applications to federated learning. arXiv preprint arXiv:2002.10619 (2020)

  33. McMahan, B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp. 1273–1282. PMLR (2017)

    Google Scholar 

  34. Muniyandi, A.P., Rajeswari, R., Rajaram, R.: Network anomaly detection by cascading K-means clustering and C4.5 decision tree algorithm. Procedia Eng. 30, 174–182 (2012)

    Article  Google Scholar 

  35. Nichol, A., Schulman, J.: Reptile: a scalable metalearning algorithm. arXiv preprint arXiv:1803.02999 (2018)

  36. Paul, D., Chakraborty, S., Das, S.: Robust principal component analysis: a median of means approach. arXiv preprint arXiv:2102.03403 (2021)

  37. Rieke, N., et al.: The future of digital health with federated learning. NPJ Digit. Med. 3(1), 1–7 (2020)

    Article  Google Scholar 

  38. Sattler, F., Müller, K.R., Samek, W.: Clustered federated learning: model-agnostic distributed multi-task optimization under privacy constraints. arXiv preprint arXiv:1910.01991 (2019)

  39. Sattler, F., Müller, K.R., Samek, W.: Clustered federated learning: model-agnostic distributed multitask optimization under privacy constraints. IEEE Trans. Neural Netw. Learn. Syst. 32, 3710–3722 (2020)

    Article  MathSciNet  Google Scholar 

  40. Shamsian, A., Navon, A., Fetaya, E., Chechik, G.: Personalized federated learning using hypernetworks. In: International Conference on Machine Learning, pp. 9489–9502. PMLR (2021)

    Google Scholar 

  41. Smith, V., Chiang, C.K., Sanjabi, M., Talwalkar, A.S.: Federated multi-task learning. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  42. Tan, Y., et al.: FedProto: federated prototype learning over heterogeneous devices. arXiv preprint arXiv:2105.00243 (2021)

  43. Wang, H., Yurochkin, M., Sun, Y., Papailiopoulos, D., Khazaeni, Y.: Federated learning with matched averaging. In: International Conference on Learning Representations (2020). https://openreview.net/forum?id=BkluqlSFDS

  44. Xie, M., et al.: Multi-center federated learning. arXiv preprint arXiv:2108.08647 (2021)

  45. Xu, J., Glicksberg, B.S., Su, C., Walker, P., Bian, J., Wang, F.: Federated learning for healthcare informatics. J. Healthcare Inform. Res. 5(1), 1–19 (2021)

    Article  Google Scholar 

  46. Yang, M.S., Lai, C.Y., Lin, C.Y.: A robust EM clustering algorithm for gaussian mixture models. Pattern Recogn. 45(11), 3950–3961 (2012)

    Article  MATH  Google Scholar 

  47. Yang, M.S., Wu, K.L.: A similarity-based robust clustering method. IEEE Trans. Pattern Anal. Mach. Intell. 26(4), 434–448 (2004)

    Article  Google Scholar 

  48. Zheng, Z., Zhou, Y., Sun, Y., Wang, Z., Liu, B., Li, K.: Applications of federated learning in smart cities: recent advances, taxonomy, and open challenges. Connect. Sci. 34(1), 1–28 (2022)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Australian Artificial Intelligence Institute, Faculty of Engineering and IT, University of Technology Sydney, Ultimo, NSW, Australia

    Jie Ma, Ming Xie & Guodong Long

Authors
  1. Jie Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ming Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guodong Long
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guodong Long .

Editor information

Editors and Affiliations

  1. The University of Adelaide, Adelaide, SA, Australia

    Weitong Chen

  2. The University of New South Wales, Sydney, NSW, Australia

    Lina Yao

  3. Macquarie University, Sydney, NSW, Australia

    Taotao Cai

  4. Griffith University, Brisbane, QLD, Australia

    Shirui Pan

  5. Microsoft, Beijing, China

    Tao Shen

  6. The University of Queensland, Brisbane, QLD, Australia

    Xue Li

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ma, J., Xie, M., Long, G. (2022). Personalized Federated Learning with Robust Clustering Against Model Poisoning. In: Chen, W., Yao, L., Cai, T., Pan, S., Shen, T., Li, X. (eds) Advanced Data Mining and Applications. ADMA 2022. Lecture Notes in Computer Science(), vol 13726. Springer, Cham. https://doi.org/10.1007/978-3-031-22137-8_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-22137-8_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-22136-1

  • Online ISBN: 978-3-031-22137-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation