Skip to main content
SpringerLink
Log in

A novel federated learning approach based on the confidence of federated Kalman filters

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Federated learning (FL) is an emerging distributed artificial intelligence (AI) algorithm. It can train a global model with multiple participants and at the same time ensure the privacy of the participants’ data. Thus, FL provides a solution for the problems faced by data silos. Existing federated learning algorithms face two significant challenges when dealing with (1) non-independent and identically distributed (non-IID) data, and (2) data with noise or without preprocessing. To address these challenges, a novel federated learning approach based on the confidence of federated Kalman filters is proposed and is referred to as FedCK in this paper. Firstly, this paper proposes a deep Generative Adversarial Network with an advanced auxiliary classifier as a pre-training module. The Non-IID increases the discreteness of the parameters of local models, it is difficult for FL to aggregate an excellent global model. The pre-training module proposed in this paper can deeply mine hidden features and increase the correlation between local model parameters. Secondly, a federated learning framework based on Federated Kalman Filter (FKF) is proposed in this paper. Because the general federation average aggregation algorithm cannot identify the model parameters with noise. This paper uses the idea of FKF to propose a set of adaptive confidence to improve the fault tolerance of FL. Experiments carried out on the MNIST, CIFAR-10 and SVHN datasets demonstrate that FedCK has better robustness and accuracy than classical federated learning methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Data Availability Statement

The data and code used to support the findings of this study are available from the corresponding author upon request (001600@nuist.edu.cn).

References

  1. Yang Q, Liu Y, Chen T, Tong Y (2019) Federated machine learning: concept and applications. ACM Trans Intell Syst Technol 10(2):1–19

    Article  Google Scholar 

  2. Li T, Sahu AK, Talwalkar A, Smith V (2020) Federated learning: challenges, methods, and future directions. IEEE Signal Process Mag 37(3):50–60

    Article  Google Scholar 

  3. Kairouz P, McMahan HB, Avent B, Bellet A, Bennis, Bhagoji AN, Bonawitz K, Charles Z, Cormode G, Cummings R et al Advances and open problems in federated learning, arXiv preprint: arXiv:1912.04977

  4. Bonawitz K, Eichner H, Grieskamp W, Huba D, Ingerman A, Ivanov V, Kiddon C, Kone£n`y J, Mazzocchi S, McMahan HB et al Towards federated learning at scale: System design, arXiv preprint: arXiv:1902.01046

  5. J. Kone£n`y HB, McMahan DR, Richtárik P Federated optimization: Distributed machine learning for on-device intelligence, arXiv preprint: arXiv:1610.02527

  6. Kone£n`y J, McMahan HB, Yu FX, Richtárik P, Suresh AT, Bacon D Federated learning: Strategies for improving communication efficiency, arXiv preprint: arXiv:1610.05492

  7. McMahan HB, Moore E, Ramage D, yArcas BA, Federated learning of deep networks using model averaging

  8. Xia M, Wang K, Zhang X, Xu Y et al (2019) Non-intrusive load disaggregation based on deep dilated residual network. Electr Power Syst Res 170:277–285

    Article  Google Scholar 

  9. Xia M, Qian J, Zhang X, Liu J, Xu Y (2019) River segmentation based on separable attention residual network. J Appl Remote Sens 14(3):032602

    Article  Google Scholar 

  10. Xia M, Zhang X, Weng L, Xu Y et al (2020) Multi-stage feature constraints learning for age estimation. IEEE Trans Inf Forensics Secur 15:2417–2428

    Article  Google Scholar 

  11. Yang Y, Xie R, Jia W, Chen Z, Yang Y, Xie L, Jiang B Accurate and automatic tooth image segmentation model with deep convolutional neural networks and level set method. Neurocomputing

  12. Fu Q, Dong H An ensemble unsupervised spiking neural network for objective recognition. Neurocomputing

  13. Zhao Y, Li M, Lai L, Suda N, Civin D, Chandra V Federated learning with non-iid data, arXiv preprint: arXiv:1806.00582.

  14. Yoshida N, Nishio T, Morikura M, Yamamoto K, Yonetani R (2020) Hybrid-fl for wireless networks: Cooperative learning mechanism using non-iid data. In: ICC 2020–2020 IEEE International Conference on Communications (ICC). IEEE, pp 1–7

  15. Wang S, Tuor T, Salonidis T, Leung KK, Makaya C, He T, Chan K (2019) Adaptive federated learning in resource constrained edge computing systems. IEEE J Sel Areas Commun 37(6):1205–1221

    Article  Google Scholar 

  16. Sattler F, Wiedemann S, Müller K-R, Samek W Robust and communication-efficient federated learning from non-iid data. IEEE Trans Neural Netw Learn Syst

  17. Blanchard P, Guerraoui R, Stainer J et al (2017) Machine learning with adversaries: byzantine tolerant gradient descent. In: Advances in neural information processing systems, pp 119–129

  18. Mhamdi EME, Guerraoui R, Rouault S The hidden vulnerability of distributed learning in Byzantium. arXiv preprint: arXiv:1802.07927

  19. Bishop CM Regularization and complexity control in feed-forward networks

  20. Graves A (2011) Practical variational inference for neural networks. In: Advances in neural information processing systems, pp 2348–2356

  21. Ang F, Chen L, Wang W (2020) Robust federated learning under worst-case model. In: 2020 IEEE wireless communications and networking conference (WCNC). IEEE, pp 1–6

  22. Radford A, Metz L, Chintala S Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint: arXiv:1511.06434

  23. Li C, Wang Z, Qi H (2018) Fast-converging conditional generative adversarial networks for image synthesis. In: 2018 25th IEEE international conference on image processing (ICIP), IEEE, pp 2132–2136

  24. Carlson NA (1988) Federated filter for fault-tolerant integrated navigation systems. In: IEEE PLANS’88., Position Location and Navigation Symposium, Record.’ Navigation into the 21st Century’., IEEE, pp 110–119

  25. Carlson NA, Berarducci MP (1994) Federated kalman filter simulation results. Navigation 41(3):297–322

    Article  Google Scholar 

  26. Ushaq M, Cheng FJ (2014) A fault tolerant integrated navigation scheme realized through online tuning of weighting factors for federated Kalman filter. In: Applied Mechanics and Materials, Vol. 446, Trans Tech Publ, pp 1078–1085

  27. Xia M, Song W, Sun X, Liu J, Ye T, Xu Y (2020) Weighted densely connected convolutional networks for reinforcement learning. Int J Pattern Recognit Artif Intell 34(04):2052001

    Article  Google Scholar 

  28. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, CourvilleA, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems, pp 2672–2680

  29. Wang H, Wang J, Wang J et al. Graph representation learning with generative adversarial nets. AAAI, Graphgan

  30. Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. European conference on computer vision. Springer, New York, pp 694–711

    Google Scholar 

  31. Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1125–1134

  32. Zhu J-Y, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision, pp 2223–2232

  33. Odena A, Olah C, Shlens J (2017) Conditional image synthesis with auxiliary classifier gans. In: International conference on machine learning, pp 2642–2651

  34. Qin Y, Zhang H, Wang S Kalman filtering and combined navigation principle [m], Xi’an: Northwestern Polytechnical University Press

  35. Qi-tai G, Song W Study on the theory of federated filter. J Chin Inertial Technol 6

  36. Zhao L (2008) Federated adaptive Kalman filtering and its application. In: 2008 7th World Congress on Intelligent Control and Automation. IEEE, pp 1369–1372

  37. Ma X, Zhang T, Liu X (2018) Application of adaptive federated filter based on innovation covariance in underwater integrated navigation system. In: 2018 IEEE International Conference on Manipulation, Manufacturing and Measurement on the Nanoscale (3M-NANO), IEEE

Download references

Acknowledgements

Research in this article is supported by the National Natural Science Foundation of China (42075130, 61773219, 61701244), the key special project of the National Key R&D Program (2018YFC1405703), and I would like to express my heartfelt thanks. I would like to express my heartfelt thanks to the reviewers who submitted valuable revisions to this article.

Author information

Authors and Affiliations

  1. School of Automation, Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Kai Hu, Jiasheng Wu, Liguo Weng, Yanwen Zhang, Fei Zheng, Zichao Pang & Min Xia

  2. Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology, (CICAEET), Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Kai Hu, Jiasheng Wu, Liguo Weng, Yanwen Zhang, Fei Zheng, Zichao Pang & Min Xia

Authors
  1. Kai Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiasheng Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liguo Weng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yanwen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fei Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zichao Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Min Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors drafted the manuscript, read, and approved the final manuscript.

Corresponding author

Correspondence to Min Xia.

Ethics declarations

Conflict of interest

No potential conflicts of interest were reported by the author.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, K., Wu, J., Weng, L. et al. A novel federated learning approach based on the confidence of federated Kalman filters. Int. J. Mach. Learn. & Cyber. 12, 3607–3627 (2021). https://doi.org/10.1007/s13042-021-01410-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-021-01410-9

Keywords

Search

Navigation