Jeongyoon Eo
Francis Y. Yan
ABSTRACT
With the growing importance of real-time communications (RTC), designing congestion control (CC) algorithms for RTC that achieve high network performance and QoE is gaining attention. Recently, data-driven, reinforcement learning (RL)-based CC algorithms for RTC have shown great potential, outperforming traditional rule-based counterparts. However, there are no open platforms tailored for training, evaluation, and validation of the algorithms that can facilitate this emerging research area.
We present OpenNetLab, an open platform for fast training, reproducible end-to-end evaluation, and performance validation of RL-based CC algorithms for RTC. Preliminary use cases confirm that OpenNetLab concretely aided the training of novel RL-based CC algorithms for RTC that outperform a well-established rule-based baseline in both network performance and QoE metrics.
- 2021. Grand Challenge on Bandwidth Estimation for Real-Time Communications. (2021). https://2021.acmmmsys.org/rtc_challenge.php.Google Scholar
- 2022. Google Meet.(2022). https://meet.google.com.Google Scholar
- 2022. iPerf - The ultimate speed test tool for TCP, UDP and SCTP. (2022). https://iperf.fr.Google Scholar
- 2022. Microsoft Teams.(2022). https://www.microsoft.com/en-us/microsoft-teams/group-chat-software.Google Scholar
- 2022. ns-3 Network Simulator.(2022). https://www.nsnam.org.Google Scholar
- 2022. OpenAI Gym.(2022). https://gym.openai.com/.Google Scholar
- 2022. Zoom.(2022). https://zoom.us.Google Scholar
- Venkat Arun and Hari Balakrishnan. 2018. Copa: Practical delay-based congestion control for the internet. In NSDI.Google ScholarDigital Library
- Per Block, Marion Hoffman, Isabel J Raabe, Jennifer Beam Dowd, Charles Rahal, Ridhi Kashyap, and Melinda C Mills. 2020. Social network-based distancing strategies to flatten the COVID-19 curve in a post-lockdown world. Nature Human Behaviour 4, 6 (2020), 588–596.Google ScholarCross Ref
- Niklas Blum, Serge Lachapelle, and Harald Alvestrand. 2021. WebRTC: real-time communication for the open web platform. Commun. ACM 64, 8 (2021), 50–54.Google ScholarDigital Library
- Francesca De Simone, Jie Li, Henrique Galvan Debarba, Abdallah El Ali, Simon NB Gunkel, and Pablo Cesar. 2019. Watching videos together in social virtual reality: An experimental study on user’s QoE. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 890–891.Google ScholarCross Ref
- Mo Dong, Tong Meng, Doron Zarchy, Engin Arslan, Yossi Gilad, Brighten Godfrey, and Michael Schapira. 2018. {PCC} vivace: Online-learning congestion control. In NSDI.Google Scholar
- Piotr Gawłowicz and Anatolij Zubow. 2019. Ns-3 meets openai gym: The playground for machine learning in networking research. In Proceedings of the 22nd International ACM Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems. 113–120.Google ScholarDigital Library
- Simon NB Gunkel, Hans M Stokking, Martin J Prins, Nanda van der Stap, Frank B ter Haar, and Omar A Niamut. 2018. Virtual Reality Conferencing: Multi-user immersive VR experiences on the web. In Proceedings of the 9th ACM Multimedia Systems Conference. 498–501.Google ScholarDigital Library
- Sangtae Ha, Injong Rhee, and Lisong Xu. 2008. CUBIC: a new TCP-friendly high-speed TCP variant. ACM SIGOPS operating systems review 42, 5 (2008), 64–74.Google ScholarDigital Library
- Stefan Holmer, Henrik Lundin, Gaetano Carlucci, Luca De Cicco, and Saverio Mascolo. 2015. A Google Congestion Control Algorithm for Real-Time Communication. (2015). https://datatracker.ietf.org/doc/html/draft-alvestrand-rmcat-congestion-03Google Scholar
- Gazi Karam Illahi, Thomas Van Gemert, Matti Siekkinen, Enrico Masala, Antti Oulasvirta, and Antti Ylä-Jääski. 2020. Cloud gaming with foveated video encoding. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 16, 1 (2020), 1–24.Google ScholarDigital Library
- Nathan Jay, Noga Rotman, Brighten Godfrey, Michael Schapira, and Aviv Tamar. 2019. A deep reinforcement learning perspective on internet congestion control. In International Conference on Machine Learning. PMLR, 3050–3059.Google Scholar
- Tong Li, Kai Zheng, Ke Xu, Rahul Arvind Jadhav, Tao Xiong, Keith Winstein, and Kun Tan. 2020. Tack: Improving wireless transport performance by taming acknowledgments. In ACM SIGCOMM.Google ScholarDigital Library
- Zhi Li, Christos Bampis, Julie Novak, Anne Aaron, Kyle Swanson, Anush Moorthy, and JD Cock. 2018. VMAF: The journey continues. Netflix Technology Blog 25 (2018).Google Scholar
- Eric Liang, Richard Liaw, Robert Nishihara, Philipp Moritz, Roy Fox, Ken Goldberg, Joseph Gonzalez, Michael Jordan, and Ion Stoica. 2018. RLlib: Abstractions for distributed reinforcement learning. In International Conference on Machine Learning. PMLR, 3053–3062.Google Scholar
- Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. 2017. Neural adaptive video streaming with pensieve. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. 197–210.Google ScholarDigital Library
- Shubham Pateria, Budhitama Subagdja, Ah-hwee Tan, and Chai Quek. 2021. Hierarchical reinforcement learning: A comprehensive survey. ACM Computing Surveys (CSUR) 54, 5 (2021), 1–35.Google ScholarDigital Library
- Chandan KA Reddy, Vishak Gopal, and Ross Cutler. 2021. Dnsmos: A non-intrusive perceptual objective speech quality metric to evaluate noise suppressors. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 6493–6497.Google ScholarCross Ref
- Saeed Shafiee Sabet, Steven Schmidt, Saman Zadtootaghaj, Carsten Griwodz, and Sebastian Möller. 2020. Delay sensitivity classification of cloud gaming content. In Proceedings of the 12th ACM International Workshop on Immersive Mixed and Virtual Environment Systems. 25–30.Google ScholarDigital Library
- Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. 1999. Policy gradient methods for reinforcement learning with function approximation. Advances in neural information processing systems 12 (1999).Google Scholar
- Bo Wang, Yuan Zhang, Size Qian, Zipeng Pan, and Yuhong Xie. 2021. A Hybrid Receiver-side Congestion Control Scheme for Web Real-time Communication. In ACM MMSys.Google Scholar
- Mowei Wang, Yong Cui, Xin Wang, Shihan Xiao, and Junchen Jiang. 2017. Machine learning for networking: Workflow, advances and opportunities. IEEE Network 32, 2 (2017), 92–99.Google ScholarCross Ref
- Francis Y Yan, Hudson Ayers, Chenzhi Zhu, Sadjad Fouladi, James Hong, Keyi Zhang, Philip Levis, and Keith Winstein. 2020. Learning in situ: a randomized experiment in video streaming. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20). 495–511.Google Scholar
- Francis Y Yan, Jestin Ma, Greg D Hill, Deepti Raghavan, Riad S Wahby, Philip Levis, and Keith Winstein. 2018. Pantheon: the training ground for Internet congestion-control research. In USENIX ATC.Google Scholar
- Huanhuan Zhang, Anfu Zhou, Yuhan Hu, Chaoyue Li, Guangping Wang, Xinyu Zhang, Huadong Ma, Leilei Wu, Aiyun Chen, and Changhui Wu. 2021. Loki: improving long tail performance of learning-based real-time video adaptation by fusing rule-based models. In MobiCom.Google Scholar
- Huanhuan Zhang, Anfu Zhou, Jiamin Lu, Ruoxuan Ma, Yuhan Hu, Cong Li, Xinyu Zhang, Huadong Ma, and Xiaojiang Chen. 2020. OnRL: improving mobile video telephony via online reinforcement learning. In MobiCom.Google Scholar
- Huanhuan Zhang, Anfu Zhou, Ruoxuan Ma, Jiamin Lu, and Huadong Ma. 2021. Arsenal: Understanding Learning-based Wireless Video Transport via In-depth Evaluation. IEEE Transactions on Vehicular Technology 70, 10 (2021), 10832–10844.Google ScholarCross Ref
- Zhengxu Xia, Yajie Zhou, Francis Y. Yan, Junchen Jiang. 2022. Automatic Curriculum Generation for Learning Adaptation in Networking. arXiv preprint arXiv: 2202.05940(2022).Google Scholar
- Anfu Zhou, Huanhuan Zhang, Guangyuan Su, Leilei Wu, Ruoxuan Ma, Zhen Meng, Xinyu Zhang, Xiufeng Xie, Huadong Ma, and Xiaojiang Chen. 2019. Learning to coordinate video codec with transport protocol for mobile video telephony. In MobiCom.Google Scholar
Index Terms
- OpenNetLab: Open Platform for RL-based Congestion Control for Real-Time Communications
Index terms have been assigned to the content through auto-classification.
Recommendations
Unreliable transport protocol using congestion control for high-speed networks
Currently there is no control for the real-time traffic of multimedia applications using UDP (User Datagram Protocol) in high-speed networks. Therefore, although a number of high-speed TCP (Transmission Control Protocol) protocols have been developed ...
Limitations of equation-based congestion control
Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer communicationsWe study limitations of an equation-based congestion control protocol, called TFRC (TCP Friendly Rate Control). It examines how the three main factors that determine TFRC throughput, namely, the TCP friendly equation, loss event rate estimation and ...
Limitations of equation-based congestion control
SIGCOMM '05: Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer communicationsWe study limitations of an equation-based congestion control protocol, called TFRC (TCP Friendly Rate Control). It examines how the three main factors that determine TFRC throughput, namely, the TCP friendly equation, loss event rate estimation and ...
Comments