skip to main content
10.1145/3555776.3577658acmconferencesArticle/Chapter ViewAbstractPublication PagessacConference Proceedingsconference-collections
research-article
Public Access

Fairness Analysis of Deep Reinforcement Learning based Multi-Path QUIC Scheduling

Published: 07 June 2023 Publication History

Abstract

Computing devices with multiple active network interfaces, such as cellular, wired, and WiFi, are becoming more and more common. Typically, such devices select a single interface for communication, but throughput and availability can increase by utilizing multipath protocols. Multipath TCP (MPTCP) is the predominant protocol in this space; however, Multipath QUIC (MPQUIC) provides several advantages over MPTCP and is increasing in adoption. Multipath protocols use a multipath scheduler to determine which packets use which interface. Legacy schedulers exhibit good performance but often poorly handle adjusting to dynamic changes in the network. Recent research includes the development of several Deep Reinforcement Learning (DRL) based schedulers that outperform legacy schedulers and improve adaptability to changing network conditions. Evaluation of any packet scheduling approach must include an assessment of fairness to concurrent TCP flows. Specifically, under congestion conditions, all flows (multipath or unipath) should tend toward an equal share of the bandwidth. Unfortunately, MPQUIC DRL-based scheduler research does not include a rigorous analysis of the fairness aspect under various network conditions, risking significant network problems as adoption increases. We present an efficiency and fairness comparison of MPQUIC using DRL-based schedulers with classic agents like DQN, Deep SARSA, and Double DQN. Experimental results over a bi-path network show that these schedulers are TCP-friendly in many cases on both paths and converge to link-centric fairness on one path. However, our work shows that they are not TCP-friendly or can be bullied under certain conditions, degrading TCP or MPQUIC performance.

References

[1]
Mahmoud Abbasi, Amin Shahraki, and Amir Taherkordi. 2021. Deep learning for network traffic monitoring and analysis (ntma): A survey. Computer Communications (2021).
[2]
Martin Becke, Thomas Dreibholz, Hakim Adhari, and Erwin Paul Rathgeb. 2012. On the fairness of transport protocols in a multi-path environment. In 2012 IEEE International Conference on Communications (ICC). IEEE, 2666--2672.
[3]
O Bonaventure, U catholique de Louvain, and C Paasch. 2020. TCP Extensions for Multipath Operation with Multiple Addresses. (2020).
[4]
Raouf Boutaba, Mohammad A Salahuddin, Noura Limam, Sara Ayoubi, Nashid Shahriar, Felipe Estrada-Solano, and Oscar M Caicedo. 2018. A comprehensive survey on machine learning for networking: evolution, applications and research opportunities. Journal of Internet Services and Applications 9, 1 (2018), 1--99.
[5]
Bob Braden, David Clark, Jon Crowcroft, Bruce Davie, Steve Deering, Deborah Estrin, Sally Floyd, Van Jacobson, Greg Minshall, Craig Partridge, et al. 1998. Recommendations on queue management and congestion avoidance in the Internet. Technical Report.
[6]
Sandeep Chinchali, Pan Hu, Tianshu Chu, Manu Sharma, Manu Bansal, Rakesh Misra, Marco Pavone, and Sachin Katti. 2018. Cellular network traffic scheduling with deep reinforcement learning. In Thirty-second AAAI conference on artificial intelligence.
[7]
Tran-Tuan Chu, Mohamed Aymen Labiod, Hai-Anh Tran, and Abdelhamid Mellouk. 2022. GADaM: Generic Adaptive Deep-learning-based Multipath Scheduler Selector for Dynamic Heterogeneous Environment. In ICC 2022-IEEE International Conference on Communications. IEEE, 4908--4913.
[8]
Quentin De Coninck and Olivier Bonaventure. 2017. Multipath quic: Design and evaluation. In Proceedings of the 13th international conference on emerging networking experiments and technologies. 160--166.
[9]
Simone Ferlin, Özgü Alay, Olivier Mehani, and Roksana Boreli. 2016. BLEST: Blocking estimation-based MPTCP scheduler for heterogeneous networks. In 2016 IFIP networking conference (IFIP networking) and workshops. IEEE, 431--439.
[10]
Bo He, Jingyu Wang, Qi Qi, Haifeng Sun, Jianxin Liao, Chunning Du, Xiang Yang, and Zhu Han. 2021. DeepCC: Multi-agent deep reinforcement learning congestion control for multi-path TCP based on self-attention. IEEE Transactions on Network and Service Management 18, 4 (2021), 4770--4788.
[11]
Bruno YL Kimura, Demetrius CSF Lima, and Antonio AF Loureiro. 2020. Packet scheduling in multipath TCP: Fundamentals, lessons, and opportunities. IEEE Systems Journal 15, 1 (2020), 1445--1457.
[12]
Seunghwa Lee and Joon Yoo. 2022. Reinforcement Learning Based Multipath QUIC Scheduler for Multimedia Streaming. Sensors 22, 17 (2022), 6333.
[13]
Wenzhong Li, Han Zhang, Shaohua Gao, Chaojing Xue, Xiaoliang Wang, and Sanglu Lu. 2019. SmartCC: A reinforcement learning approach for multipath TCP congestion control in heterogeneous networks. IEEE Journal on Selected Areas in Communications 37, 11 (2019), 2621--2633.
[14]
Yeon-sup Lim, Erich M Nahum, Don Towsley, and Richard J Gibbens. 2017. ECF: An MPTCP path scheduler to manage heterogeneous paths. In Proceedings of the 13th international conference on emerging networking experiments and technologies. 147--159.
[15]
Marc Mollà Roselló. 2019. Multi-path Scheduling with Deep Reinforcement Learning. In 2019 European Conference on Networks and Communications (EuCNC). IEEE, 400--405.
[16]
Michael Scharf and Alan Ford. 2013. Multipath TCP (MPTCP) application interface considerations. Technical Report.
[17]
Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction. MIT press.
[18]
Tobias Viernickel, Alexander Froemmgen, Amr Rizk, Boris Koldehofe, and Ralf Steinmetz. 2018. Multipath QUIC: A deployable multipath transport protocol. In 2018 IEEE International Conference on Communications (ICC). IEEE, 1--7.
[19]
Hongjia Wu, Özgü Alay, Anna Brunstrom, Simone Ferlin, and Giuseppe Caso. 2020. Peekaboo: Learning-based multipath scheduling for dynamic heterogeneous environments. IEEE Journal on Selected Areas in Communications 38, 10 (2020), 2295--2310.
[20]
Hongjia Wu, Giuseppe Caso, Simone Ferlin, Özgü Alay, and Anna Brunstrom. 2021. Multipath scheduling for 5G networks: Evaluation and outlook. IEEE Communications Magazine 59, 4 (2021), 44--50.
[21]
Hongjia Wu, Simone Ferlin, Giuseppe Caso, Özgü Alay, and Anna Brunstrom. 2021. A Survey on Multipath Transport Protocols Towards 5G Access Traffic Steering, Switching and Splitting. IEEE Access 9 (2021), 164417--164439.
[22]
Zhiyuan Xu, Jian Tang, Jingsong Meng, Weiyi Zhang, Yanzhi Wang, Chi Harold Liu, and Dejun Yang. 2018. Experience-driven networking: A deep reinforcement learning based approach. In IEEE INFOCOM 2018-IEEE conference on computer communications. IEEE, 1871--1879.
[23]
Zhiyuan Xu, Jian Tang, Chengxiang Yin, Yanzhi Wang, and Guoliang Xue. 2019. Experience-driven congestion control: When multi-path TCP meets deep reinforcement learning. IEEE Journal on Selected Areas in Communications 37, 6 (2019), 1325--1336.
[24]
Fan Yang, Qi Wang, and Paul D Amer. 2014. Out-of-order transmission for in-order arrival scheduling for multipath TCP. In 2014 28th international conference on advanced information networking and applications workshops. IEEE, 749--752.
[25]
Han Zhang, Wenzhong Li, Shaohua Gao, Xiaoliang Wang, and Baoliu Ye. 2019. ReLeS: A neural adaptive multipath scheduler based on deep reinforcement learning. In IEEE INFOCOM 2019-IEEE Conference on Computer Communications. IEEE, 1648--1656.

Cited By

View all
  • (2024)On the Fairness of Internet Congestion Control over WiFi with Deep Reinforcement LearningFuture Internet10.3390/fi1609033016:9(330)Online publication date: 10-Sep-2024
  • (2024)A novel multipath QUIC protocol with minimized flow complete time for internet content distributionComputer Science and Information Systems10.2298/CSIS230818078L21:2(625-643)Online publication date: 2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SAC '23: Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing
March 2023
1932 pages
ISBN:9781450395175
DOI:10.1145/3555776
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 June 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. multipath QUIC
  2. deep reinforcement learning
  3. fairness

Qualifiers

  • Research-article

Funding Sources

Conference

SAC '23
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Upcoming Conference

SAC '25
The 40th ACM/SIGAPP Symposium on Applied Computing
March 31 - April 4, 2025
Catania , Italy

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)97
  • Downloads (Last 6 weeks)12
Reflects downloads up to 30 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)On the Fairness of Internet Congestion Control over WiFi with Deep Reinforcement LearningFuture Internet10.3390/fi1609033016:9(330)Online publication date: 10-Sep-2024
  • (2024)A novel multipath QUIC protocol with minimized flow complete time for internet content distributionComputer Science and Information Systems10.2298/CSIS230818078L21:2(625-643)Online publication date: 2024

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media