Abstract
Federated learning (FL) is a distributed learning method that offers medical institutes the prospect of collaboration in a global model while preserving the privacy of their patients. Although most medical centers conduct similar medical imaging tasks, their differences, such as specializations, number of patients, and devices, lead to distinctive data distributions. Data heterogeneity poses a challenge for FL and the personalization of the local models. In this work, we investigate an adaptive hierarchical clustering method for FL to produce intermediate semi-global models, so clients with similar data distribution have the chance of forming a more specialized model. Our method forms several clusters consisting of clients with the most similar data distributions; then, each cluster continues to train separately. Inside the cluster, we use meta-learning to improve the personalization of the participants’ models. We compare the clustering approach with classical FedAvg and centralized training by evaluating our proposed methods on the HAM10k dataset for skin lesion classification with extreme heterogeneous data distribution. Our experiments demonstrate significant performance gain in heterogeneous distribution compared to standard FL methods in classification accuracy. Moreover, we show that the models converge faster if applied in clusters and outperform centralized training while using only a small subset of data.
Y. Yeganeh, A. Farshad, J. Boschmann, R. Gaus and M. Frantzen—Equal Contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arivazhagan, M.G., Aggarwal, V., Singh, A.K., Choudhary, S.: Federated learning with personalization layers. arXiv:1912.00818 (2019)
Bdair, T., Navab, N., Albarqouni, S.: FedPerl: semi-supervised peer learning for skin lesion classification. In: de Bruijne, M. (ed.) MICCAI 2021. LNCS, vol. 12903, pp. 336–346. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87199-4_32
Bdair, T., Navab, N., Albarqouni, S.: Semi-supervised federated peer learning for skin lesion classification. Machine Learning for Biomedical Imaging 1(April 2022 issue), 1–10 (2022)
Briggs, C., Fan, Z., Andras, P.: Federated learning with hierarchical clustering of local updates to improve training on non-IID data (2020)
Chen, F., Luo, M., Dong, Z., Li, Z., He, X.: Federated meta-learning with fast convergence and efficient communication (2019)
Fallah, A., Mokhtari, A., Ozdaglar, A.: Personalized federated learning: a meta-learning approach (2020)
Fernández, A., García, S., Galar, M., Prati, R.C., Krawczyk, B., Herrera, F.: Learning from Imbalanced Data Sets. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-98074-4
Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 1126–1135. PMLR, International Convention Centre, Sydney, Australia (2017). http://proceedings.mlr.press/v70/finn17a.html
Jiang, Y., Konecny, J., Rush, K., Kannan, S.: Improving federated learning personalization via model agnostic meta learning (2019)
Khodak, M., Balcan, M.F., Talwalkar, A.: Adaptive gradient-based meta-learning methods (2019)
Li, D., Wang, J.: FedMD: heterogenous federated learning via model distillation. arXiv:1910.03581 (2019)
Li, T., Sahu, A.K., Talwalkar, A., Smith, V.: Federated learning: challenges, methods, and future directions. IEEE Signal Proc. Mag. 37(3), 50–60 (2020)
Li, X., Huang, K., Yang, W., Wang, S., Zhang, Z.: On the convergence of FedAvg on non-IID data. arXiv:1907.02189 (2020)
Madabhushi, A., Lee, G.: Image analysis and machine learning in digital pathology: challenges and opportunities. Medical Image Analysis 33, 170–175 (2016). https://doi.org/10.1016/j.media.2016.06.037, https://www.sciencedirect.com/science/article/pii/S1361841516301141, 20th anniversary of the Medical Image Analysis journal (MedIA)
McMahan, B., Moore, E., Ramage, D., Hampson, S., y Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Singh, A., Zhu, J. (eds.) Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. Proceedings of Machine Learning Research, vol. 54, pp. 1273–1282. PMLR, Fort Lauderdale, FL, USA (2017). http://proceedings.mlr.press/v54/mcmahan17a.html
Nichol, A., Achiam, J., Schulman, J.: On first-order meta-learning algorithms (2018)
Reddi, S., et al.: Adaptive federated optimization. arXiv:2003.00295 (2020)
Rieke, N., et al.: The future of digital health with federated learning (2021)
Sae-Lim, W., Wettayaprasit, W., Aiyarak, P.: Convolutional neural networks using MobileNet for skin lesion classification. In: 2019 16th International Joint Conference on Computer Science and Software Engineering (JCSSE), pp. 242–247. IEEE (2019)
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4510–4520 (2018)
Sattler, F., Wiedemann, S., Müller, K.R., Samek, W.: Robust and communication-efficient federated learning from non-IID data. arXiv:1903.02891 (2019)
Sheller, M.J., et al.: Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data. Sci. Report. 10(1), 1–12 (2020)
Smith, V., Chiang, C.K., Sanjabi, M., Talwalkar, A.: Federated multi-task learning. arXiv:1705.10467 (2018)
Tschandl, P., Rosendahl, C., Kittler, H.: The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci. Data 5(1), 1–9 (2018)
Vanschoren, J.: Meta-learning: a survey. arXiv preprint arXiv:1810.03548 (2018)
Wang, K., Mathews, R., Kiddon, C., Eichner, H., Beaufays, F., Ramage, D.: Federated evaluation of on-device personalization. arXiv:1910.10252 (2019)
Wynants, L., Riley, R., Timmerman, D., Van Calster, B.: Random-effects meta-analysis of the clinical utility of tests and prediction models. Stat. Med. 37(12), 2034–2052 (2018)
Yeganeh, Y., Farshad, A., Navab, N., Albarqouni, S.: Inverse distance aggregation for federated learning with non-IID data. arXiv:2008.07665 (2020)
Yue, L., Tian, D., Chen, W., Han, X., Yin, M.: Deep learning for heterogeneous medical data analysis. World Wide Web 23(5), 2715–2737 (2020). https://doi.org/10.1007/s11280-019-00764-z
Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., Chandra, V.: Federated learning with non-IID data. CoRR arxiv:abs/1806.00582 (2018)
Acknowledgements
We gratefully acknowledge the Munich Center for Machine Learning (MCML) with funding from the Bundesministerium für Bildung und Forschung (BMBF) under the project 01IS18036B.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yeganeh, Y., Farshad, A., Boschmann, J., Gaus, R., Frantzen, M., Navab, N. (2022). FedAP: Adaptive Personalization in Federated Learning for Non-IID Data. In: Albarqouni, S., et al. Distributed, Collaborative, and Federated Learning, and Affordable AI and Healthcare for Resource Diverse Global Health. DeCaF FAIR 2022 2022. Lecture Notes in Computer Science, vol 13573. Springer, Cham. https://doi.org/10.1007/978-3-031-18523-6_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-18523-6_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-18522-9
Online ISBN: 978-3-031-18523-6
eBook Packages: Computer ScienceComputer Science (R0)
