Abstract
With the increasingly strengthened data privacy acts and the difficult data centralization, Federated Learning (FL) has become an effective solution to collaboratively train the model while preserving each client’s privacy. FedAvg is a standard aggregation algorithm that makes the proportion of the dataset size of each client an aggregation weight. However, it can’t deal with non-independent and identically distributed (non-IID) data well because of its fixed aggregation weights and the neglect of data distribution. The paper presents a new aggregation strategy called FedGrav, which is designed to handle non-IID datasets and is inspired by the law of universal gravitation in physics. FedGrav can dynamically adjust the aggregation weights based on the training condition of local models throughout the entire training process, making it an effective solution for non-IID data. The model affinity is creatively proposed by considering both the differences of sample size on the client and the discrepancies among local models. It considers the client sample size as the mass of the local model and defines the model graph distance based on neural network topology. By calculating the affinity among local models, FedGrav can explore internal correlations of them and improve the aggregation weights. The proposed FedGrav has been applied to the CIFAR-10 and the MICCAI Federated Tumor Segmentation (FeTS) Challenge 2021 datasets, and the validation results show that our method outperforms the previous state-of-the-art by 1.54 mean DSC and 2.89 mean HD95. The source code will be available on Github.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
McMahan, B., Moore, E., Ramage, D., Hampson, S., y Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp. 1273–1282. PMLR (2017)
Yang, Q., Liu, Y., Chen, T., Tong, Y.: Federated machine learning: concept and applications. ACM Trans. Intell. Syst. Technol. (TIST) 10(2), 1–19 (2019)
Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., Chandra, V.: Federated learning with non-IID data. arXiv preprint arXiv:1806.00582 (2018)
Li, X., Jiang, M., Zhang, X., et al.: FedBN: federated learning on non-IID features via local batch normalization. In: International Conference on Learning Representations (2020)
Li, T., Sahu, A.K., Zaheer, M., et al.: Federated optimization in heterogeneous networks. Proc. Mach. Learn. Syst. 2, 429–450 (2020)
Sattler, F., Wiedemann, S., Maluller, K.-R., Samek, W.: Robust and communication-efficient federated learning from non-IID data. IEEE Trans. Neural Networks Learn. Syst. 31, 3400–3413 (2019)
Karimireddy, S.P., Kale, S., Mohri, M., Reddi, S.J., Stich, S.U., Suresh, A.T.: Scaffold: stochastic controlled averaging for federated learning. ICML 2020 (2020)
Chen, X., Chen, T., Sun, H., Wu, Z.S., Hong, M.: Distributed training with heterogeneous data: bridging median- and mean-based algorithms. In: NeurIPS 2020 (2020)
Sheller, M.J., Reina, G.A., Edwards, B., Martin, J., Bakas, S.: Multi-institutional deep learning modeling without sharing patient data: a feasibility study on brain tumor segmentation. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds.) BrainLes 2018. LNCS, vol. 11383, pp. 92–104. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11723-8_9
Li, W., et al.: Privacy-preserving federated brain tumour segmentation. In: Suk, H.-I., Liu, M., Yan, P., Lian, C. (eds.) MLMI 2019. LNCS, vol. 11861, pp. 133–141. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32692-0_16
Liu, Q., Chen, C., Qin, J., et al.: Feddg: federated domain generalization on medical image segmentation via episodic learning in continuous frequency space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1013–1023 (2021)
Guo, P., Wang, P., Zhou, J., Jiang, S., Patel, V.M.: Multi-institutional collaborations for improving deep learning-based magnetic resonance image reconstruction using federated learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2423–2432 (2021)
Guo, P., et al.: Auto-FedRL: federated hyperparameter optimization for multi-institutional medical image segmentation. arXiv preprint arXiv:2203.06338 (2022)
Xia, Y., Yang, D., Li, W., et al.: Auto-FedAvg: learnable federated averaging for multi-institutional medical image segmentation. arXiv preprint arXiv:2104.10195 (2021)
Yeganeh, Y., Farshad, A., Navab, N., Albarqouni, S.: Inverse distance aggregation for federated learning with non-IID data. In: Albarqouni, S., et al. (eds.) DART/DCL -2020. LNCS, vol. 12444, pp. 150–159. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60548-3_15
Palihawadana, C., Wiratunga, N., Wijekoon, A., et al.: FedSim: similarity guided model aggregation for Federated Learning. Neurocomputing 483, 432–445 (2022)
Chen, H.Y., Chao, W.L.: FedBE: making Bayesian model ensemble applicable to federated learning. In: International Conference on Learning Representations
Chen, Z., Zhu, M., Yang, C., Yuan, Y.: Personalized retrogress-resilient framework for real-world medical federated learning. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12903, pp. 347–356. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87199-4_33
Dong, N., Voiculescu, I.: Federated contrastive learning for decentralized unlabeled medical images. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12903, pp. 378–387. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87199-4_36
Pati, S., et al.: The federated tumor segmentation (fets) challenge. arXiv preprint arXiv:2105.05874 (2021)
Bakas, S., et al.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci. Data 4(1), 1–13 (2017)
Reina, G.A., et al.: Open: an open-source framework for federated learning. arXiv preprint arXiv:2105.06413 (2021)
Sheller, M.J., et al.: Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data. Sci. Rep. 10(1), 1–12 (2020)
Koer, F., et al.: Brats toolkit: translating brats brain tumor segmentation algorithms into clinical and scientific practice. Front. Neurosci. 14, 125 (2020)
Mächler, L., Ezhov, I., Kofler, F., et al.: FedCostWAvg: a new averaging for better Federated Learning. In: Crimi, A., Bakas, S. (eds.) BrainLes 2021, Part II. LNCS, vol. 12963, pp. 383–391. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-09002-8_34
Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_49
Gabrielsson, R.B.: Topological Data Analysis of Convolutional Neural Networks’ Weights on Images
Nikolentzos, G., Meladianos, P., Vazirgiannis, M.: Matching node embeddings for graph similarity. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence, pp. 2429–2435 (2017)
Wang, J., Liu, Q., Liang, H., Joshi, G., Poor, H.V.: Tackling the objective inconsistency problem in heterogeneous federated optimization. In: Advances in Neural Information Processing Systems, vol. 33 (2020)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Acknowledgements
This work was supported by the Fund for Innovation and Transformation of Haidian District, Beijing, China(No. HDCXZHKC2021201)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Deng, Z. et al. (2023). FedGrav: An Adaptive Federated Aggregation Algorithm for Multi-institutional Medical Image Segmentation. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14221. Springer, Cham. https://doi.org/10.1007/978-3-031-43895-0_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-43895-0_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43894-3
Online ISBN: 978-3-031-43895-0
eBook Packages: Computer ScienceComputer Science (R0)
