Skip to main content
Springer Nature Link
Log in

An optimal scheme for speeding the training process of the asynchronous federated learning based on model partition

  • Regular Paper
  • Published:
Computing Aims and scope Submit manuscript

Abstract

Federated learning (FL) in an edge computing environment holds significant promise for enabling artificial intelligence at the network’s edge. Typically, FL requires clients to run complete Deep Neural Networks (DNNs), which can be a substantial burden for resource-constrained edge clients, potentially preventing them from completing tasks within the required timeframe. To address this, using model partition technique to execute parts of the DNN on clients can greatly enhance the applicability of FL. In this paper, we propose an optimal algorithm designed to reduce time latency through model partition technique for asynchronous FL. By utilizing the model partition technique, the DNN model is divided into two segments: one deployed on the client and the other on the edge server for model training. However, varying model partition points across different devices with differing transmission bandwidths can result in significant variations in time latency. So the most difficult part of our algorithm is to determine the suitable partition points for all devices. Initially, we establish a metric that correlates learning accuracy with iteration frequency. Using this metric, we construct the original mathematical model. Given the vast solution space of this model, which makes direct resolution impractical, we introduce an Optimal Bandwidth Allocation (OBA) algorithm to minimize total training time. The OBA algorithm operates by first filtering potential partition points based on network characteristics. It then selects suitable partition points tailored to different clients and bandwidth allocations, thereby achieving reduced training times. Simulation results demonstrate that our algorithm can decrease time latency by 18 to 64% compared to seven other methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Algorithm 2
Algorithm 3
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Jing L, Tian Y (2021) Self-supervised visual feature learning with deep neural networks: a survey. IEEE Trans Pattern Anal Mach Intell 43(11):4037–4058. https://doi.org/10.1109/TPAMI.2020.2992393

    Article  MATH  Google Scholar 

  2. Janiesch Christian Z, Patrick H Kai (2021) Machine learning and deep learning. Electron Mark 31(3):685–695

    Article  MATH  Google Scholar 

  3. Deng Jiankang G, Jia Y, Jing X, Niannan et al (2022) Arcface: additive angular margin loss for deep face recognition. IEEE Trans Pattern Anal Mach Intell 44(10):5962–5979

  4. Kriman S, BS, B G, et al (2020) Deep automatic speech recognition with 1d time-channel separable convolutionsp. ICASSP 2020 - 2020 IEEE International conference on acoustics, speech and signal processing, 6124–6128

  5. Yu T JH, Shen Y (2019) A visual dialog augmented interactive recommender system. Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, 157–165

  6. Chettri L, Bera R (2020) A comprehensive survey on internet of things (IOT) toward 5g wireless systems. IEEE Internet Things J 7(1):16–32

    Article  MATH  Google Scholar 

  7. McMahan B, Moore E, Ramage D, Hampson S, Arcas B.A.y (2017) Communication-efficient learning of deep networks from decentralized data. Proceedings of the 20th international conference on artificial intelligence and statistics 54:1273–1282

  8. Chen J, Ran X (2019) Deep learning with edge computing: a review. Proc IEEE 107(8):1655–1674. https://doi.org/10.1109/JPROC.2019.2921977

    Article  MATH  Google Scholar 

  9. Zhang H, Bosch J, Olsson HH (2021) End-to-end federated learning for autonomous driving vehicles. 2021 International joint conference on neural networks (IJCNN), 1–8

  10. Hu Z, Tarakji AB, Raheja V, Phillips C, Wang T, Mohomed I ( 2019) Deephome: distributed inference with heterogeneous devices in the edge. In: EMDL ’19

  11. Jiang M, Jung T, Karl R, Zhao T (2022) Federated dynamic graph neural networks with secure aggregation for video-based distributed surveillance. ACM Trans Intell Syst Technol (TIST) 13:1–23

    MATH  Google Scholar 

  12. Xiao H, Xu C, Ma Y, Yang S, Zhong L, Muntean G-M (2022) Edge intelligence: a computational task offloading scheme for dependent IOT application. IEEE Trans Wireless Commun 21:7222–7237

    Article  Google Scholar 

  13. Hard A, Rao K, Mathews R, Beaufays F, Augenstein S, Eichner H, Kiddon C, Ramage D (2018) Federated learning for mobile keyboard prediction. CoRR arXiv: abs/1811.03604

  14. Mills J, Hu J, Min G (2020) Communication-efficient federated learning for wireless edge intelligence in IOT. IEEE Internet Things J 7(7):5986–5994

    Article  MATH  Google Scholar 

  15. Zhang W, Gupta S, Lian X, Liu J (2016) Staleness-aware async-sgd for distributed deep learning. Proceedings of the twenty-fifth international joint conference on artificial intelligence, 2350–2356

  16. Zhu G, Liu D, Du Y, You C, Zhang J, Huang K (2020) Toward an intelligent edge: wireless communication meets machine learning. IEEE Commun Mag 58(1):19–25

    Article  MATH  Google Scholar 

  17. Chen M, Semiari O, Saad W, Liu X, Yin C (2020) Federated echo state learning for minimizing breaks in presence in wireless virtual reality networks. IEEE Trans Wireless Commun 19(1):177–191

    Article  Google Scholar 

  18. Cai Z, Shi T (2021) Distributed query processing in the edge-assisted IOT data monitoring system. IEEE Internet Things J 8(16):12679–12693

    Article  MATH  Google Scholar 

  19. Zhu T, Shi T, Li J, Cai Z, Zhou X (2019) Task scheduling in deadline-aware mobile edge computing systems. IEEE Internet Things J 6(3):4854–4866

    Article  MATH  Google Scholar 

  20. Kang Y, Hauswald J, Gao C, Rovinski A, Mudge T, Mars J, Tang L (2017) Neurosurgeon: collaborative intelligence between the cloud and mobile edge. SIGPLAN Not. 52(4):615–629

    Article  Google Scholar 

  21. Casella B, Esposito R, Cavazzoni C, Aldinucci M (2023) Benchmarking fedavg and fedcurv for image classification tasks arXiv: 2303.17942 [cs.LG]

  22. Konecný J, Qu Z, Richtárik P (2014) Semi-stochastic coordinate descent. Optim Methods Softw 32:1005–993

    MathSciNet  MATH  Google Scholar 

  23. Ma C, Konecný J, Jaggi M, Smith V, Jordan MI, Richtárik P, Takác M (2015) Distributed optimization with arbitrary local solvers. Optim Methods Softw 32:813–848

    Article  MathSciNet  MATH  Google Scholar 

  24. Yang Z, Chen M, Saad W, Hong CS, Shikh-Bahaei M (2021) Energy efficient federated learning over wireless communication networks. IEEE Trans Wireless Commun 20(3):1935–1949

    Article  Google Scholar 

  25. Konečný J, McMahan HB, Ramage D, Richtárik P (2016) Federated optimization: Distributed machine learning for on-device intelligence. CoRR arXiv: abs/1610.02527

  26. Li X, Huang K, Yang W, Wang S, Zhang Z (2020) On the convergence of fedavg on non-iid data. International conference on learning representations

  27. Wang J, Joshi G (2021) Cooperative SGD: A unified framework for the design and analysis of local-update SGD algorithms. J Mach Learn Res 22(213):1–50

    MathSciNet  MATH  Google Scholar 

  28. Casella B, Esposito R, Sciarappa A, Cavazzoni C, Aldinucci M (2023) Experimenting with normalization layers in federated learning on non-IID scenarios

  29. Chen M, Poor HV, Saad W, Cui S (2021) Convergence time optimization for federated learning over wireless networks. IEEE Trans Wireless Commun 20(4):2457–2471

    Article  MATH  Google Scholar 

  30. Xu J, Wang H (2021) Client selection and bandwidth allocation in wireless federated learning networks: a long-term perspective. IEEE Trans Wireless Commun 20(2):1188–1200

    Article  MATH  Google Scholar 

  31. Amiri MM, Gündüz D, Kulkarni SR, Poor HV (2021) Convergence of update aware device scheduling for federated learning at the wireless edge. IEEE Trans Wireless Commun 20(6):3643–3658

    Article  MATH  Google Scholar 

  32. Chen M, Mao B, Ma T (2021) Fedsa: a staleness-aware asynchronous federated learning algorithm with non-iid data. Futur Gener Comput Syst 120:1–12

    Article  MATH  Google Scholar 

  33. Lee H-S, Lee J-W (2021) Adaptive transmission scheduling in wireless networks for asynchronous federated learning. IEEE J Sel Areas Commun 39(12):3673–3687

    Article  MATH  Google Scholar 

  34. Guo K, Chen Z, Yang HH, Quek TQS (2022) Dynamic scheduling for heterogeneous federated learning in private 5G edge networks. IEEE J Selected Topics Signal Process 16(1):26–40. https://doi.org/10.1109/JSTSP.2021.3126174

    Article  MATH  Google Scholar 

  35. Sun Y, Zhou S, Niu Z, Gündüz D (2022) Dynamic scheduling for over-the-air federated edge learning with energy constraints. IEEE J Sel Areas Commun 40(1):227–242. https://doi.org/10.1109/JSAC.2021.3126078

    Article  MATH  Google Scholar 

  36. Mao Y, Hong W, Wang H, Li Q, Zhong S (2021) Privacy-preserving computation offloading for parallel deep neural networks training. IEEE Trans Parallel Distrib Syst 32(7):1777–1788

    MATH  Google Scholar 

  37. Zhang L, Xu J (2021) Learning the optimal partition for collaborative DNN training with privacy requirements. IEEE Internet Things J pp 1–11

  38. Gupta O, Raskar R (2018) Distributed learning of deep neural network over multiple agents. J Netw Comput Appl 116:1–8. https://doi.org/10.1016/j.jnca.2018.05.003

    Article  MATH  Google Scholar 

  39. Chandra Thapa, S.C ( 2022) Pathum Chamikara Mahawaga Arachchige, Sun, L.: Splitfed: When federated learning meets split learning. Proceedings of the AAAI Conference on Artificial Intelligence 36( 8), 8485– 8493

  40. Shi L, Xu Z, Sun Y, Shi Y, Fan Y, Ding X (2021) A DNN inference acceleration algorithm combining model partition and task allocation in heterogeneous edge computing system. Peer-to-Peer Netw. Appl. 14:4031–4045

    Article  MATH  Google Scholar 

  41. Deng X, Li J, Ma C, Wei K, Shi L, Ding M, Chen W (2022) Low-latency federated learning with DNN partition in distributed industrial IOT networks. IEEE J Sel Areas Commun 41(3):755–775

    Article  MATH  Google Scholar 

Download references

Author information

Author notes

  1. Lei Shi, Ying Ren, Yibin Xie, Chen Fang and Yuqi Fan have contributed equally to this work.

Authors and Affiliations

  1. School of Computer Science and Information Engineering, Hefei University of Technology, HeFei, 230009, China

    Lei Shi, Ying Ren, Jing Xu, Yibin Xie, Chen Fang & Yuqi Fan

  2. Engineering Research Center of Safety Critical Industrial Measurement and Control Technology, Ministry of Education, HeFei, 23009, China

    Lei Shi, Ying Ren, Jing Xu, Yibin Xie, Chen Fang & Yuqi Fan

Authors
  1. Lei Shi
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Ying Ren
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Jing Xu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Yibin Xie
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Chen Fang
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Yuqi Fan
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Jing Xu.

Ethics declarations

Conflict of interest

The work is supported by the Anhui Provincial Natural Science Foundation (Grant No. 2308085MF212) and Joint Fund Project of Natural Science Foundation of Anhui Province (grant No. 2208085US05, No. 2308085US05).

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, L., Ren, Y., Xu, J. et al. An optimal scheme for speeding the training process of the asynchronous federated learning based on model partition. Computing 107, 67 (2025). https://doi.org/10.1007/s00607-025-01418-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00607-025-01418-x

Keywords

Mathematics Subject Classification