Skip to main content
Springer Nature Link
Log in

Deep reinforcement learning-based fountain coding for concurrent multipath transfer in high-speed railway networks

  • Published:
Peer-to-Peer Networking and Applications Aims and scope Submit manuscript

Abstract

Concurrent multipath transfer (CMT) has been proved to significantly improve the end-to-end throughput with its multihoming property. However, due to the extremely high unpredictability around high-speed railway (HSR) environment, the receive buffer blocking problem seriously degrades the overall transmission reliability. To address this issue, this paper proposes a learning-based fountain coding for CMT (FC-CMT) scheme to mitigate the negative influence of the path diversity of HSR networks. Specifically, we first formulate a multi-dimensional optimal problem to mitigate receive buffer blocking phenomenon and improve the transmission rate with requirement constrains. Then, we transform the data scheduling and redundancy coding rate problem into a Markov decision process, and propose a deep reinforcement learning (DRL)-based fountain coding algorithm to dynamically adjust data scheduling policy and redundancy coding rate. We conduct the extensive experiments in a P4-based programmable network platform. Experimental results indicate the proposed algorithm mitigates the packet out-of-order problem, and improves the average throughput compared with traditional multipath transmission scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. A shortened version has been accepted in 2022 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

  2. P4: https://p4.org/

References

  1. Lyu F, Cheng N, Zhu H, Zhou H, Xu W, Li M, Shen X (2020) Towards rear-end collision avoidance: Adaptive beaconing for connected vehicles. IEEE Trans Intell Transp Syst 22(2):1248–1263

    Article  Google Scholar 

  2. Zhang H, Dong P, Quan W, Hu B (2015) Promoting efficient communications for high-speed railway using smart collaborative networking. IEEE Wirel Commun 22(6):92–97

    Article  Google Scholar 

  3. Zhang X, Wang Y, Zhang J, Wang L, Zhao Y (2019b) RINGLM: A link-level packet loss monitoring solution for software-defined networks. IEEE J Sel Areas Commun 37(8):1703–1720

  4. Chen C, Berry RA, Honig ML, Subramanian VG (2020) Pricing, bandwidth allocation, and service competition in heterogeneous wireless networks. IEEE/ACM Trans Networking 28(5):2299–2308

    Article  Google Scholar 

  5. Zhou H, Cheng N, Wang J, Chen J, Yu Q, Shen X (2019) Toward dynamic link utilization for efficient vehicular edge content distribution. IEEE Trans Veh Technol 68(9):8301–8313

    Article  Google Scholar 

  6. Li M, Lukyanenko A, Ou Z, Ylä-Jääski A, Tarkoma S, Coudron M, Secci S (2016) Multipath transmission for the internet: A survey. IEEE Communications Surveys & Tutorials 18(4):2887–2925

    Article  Google Scholar 

  7. Xu C, Liu T, Guan J, Zhang H, Muntean GM (2012) CMT-QA: Quality-aware adaptive concurrent multipath data transfer in heterogeneous wireless networks. IEEE Trans Mob Comput 12(11):2193–2205

    Article  Google Scholar 

  8. Quan W, Liu Y, Zhang H, Yu S (2017) Enhancing crowd collaborations for software defined vehicular networks. IEEE Commun Mag 55(8):80–86

    Article  Google Scholar 

  9. Quan W, Cheng N, Qin M, Zhang H, Chan HA, Shen X (2018) Adaptive transmission control for software defined vehicular networks. IEEE Wireless Communications Letters 8(3):653–656

    Article  Google Scholar 

  10. Paasch C, Ferlin S, Alay O, Bonaventure O (2014) Experimental evaluation of multipath TCP schedulers. In: Proceedings of the 2014 ACM SIGCOMM workshop on Capacity sharing workshop, pp 27–32

  11. Zhang X, Gu W, Zhang H, Wang M (2019a) Hybrid communication path orchestration for 5G heterogeneous ultra-dense networks. IEEE Network 33(4):112–118

  12. Barré S, Paasch C, Bonaventure O (2011) Multipath TCP: from theory to practice. In: International conference on research in networking, Springer, pp 444–457

  13. Wischik D, Raiciu C, Greenhalgh A, Handley M (2011) Design, implementation and evaluation of congestion control for multipath TCP. In: NSDI 8(11)

  14. Raiciu C, Paasch C, Barre S, Ford A, Honda M, Duchene F, Bonaventure O, Handley M (2012) How hard can it be? designing and implementing a deployable multipath TCP. In: 9th USENIX symposium on networked systems design and implementation. NSDI 12:399–412

  15. Song F, Li L, You I, Zhang H (2021) Enabling heterogeneous deterministic networks with smart collaborative theory. IEEE Netw 35(3):64–71

    Article  Google Scholar 

  16. Yu C, Quan W, Gao D, Zhang Y, Liu K, Wu W, Zhang H, Shen X (2021) Reliable cybertwin-driven concurrent multipath transfer with deep reinforcement learning. IEEE Internet Things J 8(22):16207–16218

    Article  Google Scholar 

  17. Cumbal R, Palacios H, Hincapié R (2016) Optimum deployment of rsu for efficient communications multi-hop from vehicle to infrastructure on vanet. In: 2016 IEEE Colombian Conference on Communications and Computing (COLCOM), IEEE 1–6

  18. Shi H, Cui Y, Wang X, Hu Y, Dai M, Wang F, Zheng K (2018) STMS: Improving MPTCP throughput under heterogeneous networks. In: 2018 USENIX Annual Technical Conference (USENIXATC 18) 719–730

  19. Xu C, Quan W, Vasilakos AV, Zhang H, Muntean GM (2017) Information-centric cost-efficient optimization for multimedia content delivery in mobile vehicular networks. Comput Commun 99:93–106

    Article  Google Scholar 

  20. Yin C, Dong P, Du X, Zheng T, Zhang H, Guizani M (2020) An adaptive network coding scheme for multipath transmission in cellular-based vehicular networks. Sensors 20(20):5902

    Article  Google Scholar 

  21. Zhang Y, Dong P, Yu Y, Du X, Luo H, Zheng T, Guizani M (2018) A bignum network coding scheme for multipath transmission in vehicular networks. In: 2018 IEEE Global Communications Conference (GLOBECOM), IEEE, pp 206–212

  22. Ahlswede R, Cai N, Li SY, Yeung RW (2000) Network information flow. IEEE Trans Inf Theory 46(4):1204–1216

    Article  MathSciNet  MATH  Google Scholar 

  23. Liu Y, Wang H, Peng M, Guan J, Wang Y (2020) An incentive mechanism for privacy-preserving crowdsensing via deep reinforcement learning. IEEE Internet Things J 8(10):8616–8631

    Article  Google Scholar 

  24. Shen X, Gao J, Wu W, Lyu K, Li M, Zhuang W, Li X, Rao J (2020) AI-assisted network-slicing based next-generation wireless networks. IEEE Open J Veh Technol 1(1):45–66

    Article  Google Scholar 

  25. Zhang M, Zhou Y, Quan W, Zhu J, Zheng R, Wu Q (2020) Online learning for IoT optimization: A frank-wolfe adam-based algorithm. IEEE Internet Things J 7(9):8228–8237

    Article  Google Scholar 

  26. Peng K, Huang H, Zhao B, Jolfaei A, Xu X, Bilal M (2022) Intelligent computation offloading and resource allocation in IIoT with end-edge-cloud computing using NSGA-III. IEEE Transactions on Network Science and Engineering pp 1, https://doi.org/10.1109/TNSE.2022.3155490

  27. Hen X, Gao J, Wu W, Li M, Zhou C, Zhuang W (2021) Holistic network virtualization and pervasive network intelligence for 6G. IEEE Commun Surveys Tuts. https://doi.org/10.1109/COMST.2021.3135829

  28. Liu G, Quan W, Cheng N, Gao D, Lu N, Zhang H, Shen X (2021) Softwarized iot network immunity against eavesdropping with programmable data planes. IEEE Internet Things J 8(8):6578–6590

    Article  Google Scholar 

  29. Shi J, Quan W, Gao D, Liu M, Liu G, Yu C, Su W (2020) Flowlet-based stateful multipath forwarding in heterogeneous Internet of things. IEEE Access 8:74875–74886

    Article  Google Scholar 

  30. Zhang H, Quan W, Song J, Jiang Z, Yu S (2016) Link state prediction-based reliable transmission for high-speed railway networks. IEEE Trans Veh Technol 65(12):9617–9629

    Article  Google Scholar 

  31. Dong P, Zheng T, Yu S, Zhang H, Yan X (2017) Enhancing vehicular communication using 5G-enabled smart collaborative networking. IEEE Wirel Commun 24(6):72–79

    Article  Google Scholar 

  32. Li L, Xu K, Li T, Zheng K, Peng C, Wang D, Wang X, Shen M, Mijumbi R (2018) A measurement study on multi-path TCP with multiple cellular carriers on high speed rails. In: Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication, pp 161–175

  33. Dong P, Song B, Zhang H, Du X (2016) Improving onboard internet services for high-speed vehicles by multipath transmission in heterogeneous wireless networks. IEEE Trans Veh Technol 65(12):9493–9507

    Article  Google Scholar 

  34. Xu C, Wang P, Xiong C, Wei X, Muntean GM (2016) Pipeline network coding-based multipath data transfer in heterogeneous wireless networks. IEEE Trans Broadcast 63(2):376–390

    Article  Google Scholar 

  35. Arianpoo N, Aydin I, Leung VC (2016) Network coding as a performance booster for concurrent multi-path transfer of data in multi-hop wireless networks. IEEE Trans Mob Comput 16(4):1047–1058

    Article  Google Scholar 

  36. Cui Y, Wang L, Wang X, Wang H, Wang Y (2014) Fmtcp: A fountain code-based multipath transmission control protocol. IEEE/ACM Trans Networking 23(2):465–478

    Article  Google Scholar 

  37. Garcia-Saavedra A, Karzand M, Leith DJ (2017) Low delay random linear coding and scheduling over multiple interfaces. IEEE Trans Mob Comput 16(11):3100–3114

    Article  Google Scholar 

  38. Hagenauer J, Lutz E (1987) Forward error correction coding for fading compensation in mobile satellite channels. IEEE J Sel Areas Commun 5(2):215–225

    Article  Google Scholar 

  39. Hellge C, Gómez-Barquero D, Schierl T, Wiegand T (2011) Layer-aware forward error correction for mobile broadcast of layered media. IEEE Trans Multimedia 13(3):551–562

    Article  Google Scholar 

  40. MacKay DJ (2005) Fountain codes. IEE Proceedings-Communications 152(6):1062–1068

    Article  Google Scholar 

  41. Ford A, Raiciu C, Handley M, Barre S, Iyengar J et al (2011) Architectural guidelines for multipath TCP development. IETF, Informational RFC 6182:2070–1721

    Google Scholar 

  42. Wu H, Lyu F, Zhou C, Chen J, Wang L, Shen X (2020) Optimal uav caching and trajectory in aerial-assisted vehicular networks: A learning-based approach. IEEE J Sel Areas Commun 38(12):2783–2797

    Article  Google Scholar 

  43. Silver D, Lever G, Heess N, Degris T, Wierstra D, Riedmiller M (2014) Deterministic policy gradient algorithms. In: International conference on machine learning. PMLR, pp 387–395

Download references

Funding

This work was supported in part by the National Key Research and Development Program of China under Grant No. 2019YFBI802503 and in part by the National Natural Science Foundation of Beijing, China, under Grant No. 4212010.

Author information

Authors and Affiliations

  1. School of Electronic and Information Engineering, Beijing Jiaotong University, No. 3 Shangyuancun, Beijing, 100044, PR China

    Chengxiao Yu, Wei Quan, Kang Liu, Mingyuan Liu, Ziheng Xu & Hongke Zhang

Authors
  1. Chengxiao Yu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Wei Quan
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Kang Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Mingyuan Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Ziheng Xu
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Hongke Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Wei Quan.

Ethics declarations

Conflicts of interest

The authors of this paper declare that there is no conflict of interest in this research paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, C., Quan, W., Liu, K. et al. Deep reinforcement learning-based fountain coding for concurrent multipath transfer in high-speed railway networks. Peer-to-Peer Netw. Appl. 15, 2744–2756 (2022). https://doi.org/10.1007/s12083-022-01321-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12083-022-01321-8

Keywords