skip to main content
10.1145/3343031.3356063acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Continuous Bitrate & Latency Control with Deep Reinforcement Learning for Live Video Streaming

Published: 15 October 2019 Publication History

Abstract

In this paper, we introduce a continuous bitrate control and latency control model for the Live Video Streaming Challenge. Our model is based on Deep Deterministic Policy Gradient, popular on continuous control tasks. Simultaneously, it can take a fine-grained control through continuous control and does not need to discrete the continuous "latency limit", which is a buffer threshold to minimize end-to-end delay by frame skipping. In all considered live video scenarios, our model can provide a better quality of experience with improvements in average QoE of 3.6% than DQN which discrete the "latency limit". Additionally, challenge results show the effectiveness and applicability of the proposed model, which achieved top performance in 3 different networks that include high, low and oscillating throughput, and ranked the second place in the network with medium throughput.

References

[1]
Zahaib Akhtar, Yun Seong Nam, Ramesh Govindan, Sanjay Rao, Jessica Chen, Ethan Katz-Bassett, Bruno Ribeiro, Jibin Zhan, and Hui Zhang. 2018. Oboe: auto-tuning video ABR algorithms to network conditions. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication. ACM, 44--58.
[2]
Abdelhak Bentaleb, Bayan Taani, Ali C Begen, Christian Timmerer, and Roger Zimmermann. 2018. A survey on bitrate adaptation schemes for streaming media over http. IEEE Communications Surveys & Tutorials, Vol. 21, 1 (2018), 562--585.
[3]
VNI Cisco. 2018. Cisco Visual Networking Index: Forecast and Trends, 2017--2022. White Paper (2018).
[4]
Matteo Gadaleta, Federico Chiariotti, Michele Rossi, and Andrea Zanella. 2017. D-DASH: A deep Q-learning framework for DASH video streaming. IEEE Transactions on Cognitive Communications and Networking, Vol. 3, 4 (2017), 703--718.
[5]
Tianchi Huang, Rui-Xiao Zhang, Zhou Chao, and Lifeng Sun. 2018. QARC: Video Quality Aware Rate Control for Real-Time Video Streaming based on Deep Reinforcement Learning. (2018).
[6]
Te Yuan Huang, Ramesh Johari, Nick Mckeown, Matthew Trunnell, and Mark Watson. 2014. A Buffer-Based Approach to Rate Adaptation: Evidence from a Large Video Streaming Service. Acm Sigcomm Computer Communication Review, Vol. 44, 4 (2014), 187--198.
[7]
Junchen Jiang, Vyas Sekar, and Hui Zhang. 2014. Improving fairness, efficiency, and stability in http-based adaptive video streaming with festive. IEEE/ACM Transactions on Networking (ToN), Vol. 22, 1 (2014), 326--340.
[8]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980(2014).
[9]
Yuxi Li. 2017. Deep reinforcement learning: An overview. arXiv preprint arXiv:1701.07274 (2017).
[10]
Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2015. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 (2015).
[11]
Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. 2017. Neural adaptive video streaming with pensieve. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. ACM, 197--210.
[12]
Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asynchronous methods for deep reinforcement learning. In International conference on machine learning. 1928--1937.
[13]
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et almbox. 2015. Human-level control through deep reinforcement learning. Nature, Vol. 518, 7540 (2015), 529.
[14]
Kevin Spiteri, Rahul Urgaonkar, and Ramesh K Sitaraman. 2016. BOLA: Near-optimal bitrate adaptation for online videos. In IEEE INFOCOM 2016-The 35th Annual IEEE International Conference on Computer Communications. IEEE, 1--9.
[15]
Yi Sun, Xiaoqi Yin, Junchen Jiang, Vyas Sekar, Fuyuan Lin, Nanshu Wang, Tao Liu, and Bruno Sinopoli. 2016. CS2P: Improving video bitrate selection and adaptation with data-driven throughput prediction. In Proceedings of the 2016 ACM SIGCOMM Conference. ACM, 272--285.
[16]
Pawel Wawrzynski. 2015. Control policy with autocorrelated noise in reinforcement learning for robotics. International Journal of Machine Learning and Computing, Vol. 5, 2 (2015), 91.
[17]
Xiaoqi Yin, Abhishek Jindal, Vyas Sekar, and Bruno Sinopoli. 2015. A control-theoretic approach for dynamic adaptive video streaming over HTTP. In ACM SIGCOMM Computer Communication Review, Vol. 45. ACM, 325--338.

Cited By

View all
  • (2024)Intelligent QoE Management for IoMT Streaming Services in Multiuser Downlink RSMA NetworksIEEE Internet of Things Journal10.1109/JIOT.2023.333447311:7(12602-12618)Online publication date: 1-Apr-2024
  • (2024)Real-time rate control of WebRTC video streams in 5G networks: Improving quality of experience with Deep Reinforcement LearningJournal of Systems Architecture10.1016/j.sysarc.2024.103066148(103066)Online publication date: Mar-2024
  • (2023)TDS-KRFI: Reference Frame Identification for Live Web Streaming Toward HTTP Flash Video ProtocolIEEE Transactions on Network and Service Management10.1109/TNSM.2023.328256320:4(4198-4215)Online publication date: Dec-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MM '19: Proceedings of the 27th ACM International Conference on Multimedia
October 2019
2794 pages
ISBN:9781450368896
DOI:10.1145/3343031
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. continuous control
  2. deep reinforcement learning
  3. live video streaming

Qualifiers

  • Research-article

Funding Sources

Conference

MM '19
Sponsor:

Acceptance Rates

MM '19 Paper Acceptance Rate 252 of 936 submissions, 27%;
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)29
  • Downloads (Last 6 weeks)1
Reflects downloads up to 19 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Intelligent QoE Management for IoMT Streaming Services in Multiuser Downlink RSMA NetworksIEEE Internet of Things Journal10.1109/JIOT.2023.333447311:7(12602-12618)Online publication date: 1-Apr-2024
  • (2024)Real-time rate control of WebRTC video streams in 5G networks: Improving quality of experience with Deep Reinforcement LearningJournal of Systems Architecture10.1016/j.sysarc.2024.103066148(103066)Online publication date: Mar-2024
  • (2023)TDS-KRFI: Reference Frame Identification for Live Web Streaming Toward HTTP Flash Video ProtocolIEEE Transactions on Network and Service Management10.1109/TNSM.2023.328256320:4(4198-4215)Online publication date: Dec-2023
  • (2023)BoB: Bandwidth Prediction for Real-Time Communications Using Heuristic and Reinforcement LearningIEEE Transactions on Multimedia10.1109/TMM.2022.321645625(6930-6945)Online publication date: 1-Jan-2023
  • (2023)Toward Low-Latency and High-Quality Adaptive 360$^\circ$ StreamingIEEE Transactions on Industrial Informatics10.1109/TII.2022.319239819:5(6326-6336)Online publication date: May-2023
  • (2023)Live360: Viewport-Aware Transmission Optimization in Live 360-Degree Video StreamingIEEE Transactions on Broadcasting10.1109/TBC.2023.323440569:1(85-96)Online publication date: Mar-2023
  • (2023)Adaptive Live Streaming for Multi-user Access with Fairness Guarantee2023 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB)10.1109/BMSB58369.2023.10211604(1-6)Online publication date: 14-Jun-2023
  • (2022)A Hybrid Control Scheme for 360-Degree Dynamic Adaptive Video Streaming Over Mobile DevicesIEEE Transactions on Mobile Computing10.1109/TMC.2021.305809921:10(3428-3442)Online publication date: 1-Oct-2022
  • (2022)Cratus: A Lightweight and Robust Approach for Mobile Live StreamingIEEE Transactions on Mobile Computing10.1109/TMC.2020.304882621:8(2761-2775)Online publication date: 1-Aug-2022
  • (2022)Media Production Using Cloud and Edge Computing: Recent Progress and NBMP-Based ImplementationIEEE Transactions on Broadcasting10.1109/TBC.2022.314070468:2(545-558)Online publication date: Jun-2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media