skip to main content
10.1145/3304112.3325607acmconferencesArticle/Chapter ViewAbstractPublication PagesmmsysConference Proceedingsconference-collections
research-article

Enhancing the crowdsourced live streaming: a deep reinforcement learning approach

Published: 21 June 2019 Publication History

Abstract

With the growing demand for crowdsourced live streaming (CLS), how to schedule the large-scale dynamic viewers effectively among different Content Delivery Network (CDN) providers has become one of the most significant challenges for CLS platforms. Although abundant algorithms have been proposed in recent years, they suffer from a critical limitation: due to their inaccurate feature engineering or naive rules, they cannot optimally schedule viewers. To address this concern, we propose LTS (Learn to schedule), a deep reinforcement learning (DRL) based scheduling approach that can dynamically adapt to the variation of both viewer traffics and CDN performance. After the extensive evaluation the real data from a leading CLS platform in China, we demonstrate that LTS improves the average quality of experience (QoE) over state-of-the-art approach by 8.71%-15.63%.

References

[1]
Vijay Kumar Adhikari and et al. 2012. Unreeling netflix: Understanding and improving multi-cdn movie delivery. In INFOCOM, 2012 Proceedings IEEE. IEEE, 1620--1628.
[2]
Vijay Kumar Adhikari et al. 2012. A tale of three CDNs: An active measurement study of Hulu and its CDNs. In Computer Communications Workshops (INFOCOM WKSHPS), 2012 IEEE Conference on. IEEE, 7--12.
[3]
Florin Dobrian and et al. 2011. Understanding the impact of video quality on user engagement. In ACM SIGCOMM Computer Communication Review, Vol. 41. ACM, 362--373.
[4]
Mnih et al. 2016. Asynchronous methods for deep reinforcement learning. In International Conference on Machine Learning. 1928--1937.
[5]
Chen Fei et al. 2015. Cloud-assisted live streaming for crowdsourced multimedia content. IEEE Transactions on Multimedia 17, 9 (2015), 1471--1483.
[6]
Aurélien Garivier and Eric Moulines. 2008. On upper-confidence bound policies for non-stationary bandit problems. arXiv preprint arXiv:0805.3415 (2008).
[7]
Junchen Jiang et al. 2016. CFA: A Practical Prediction System for Video QoE Optimization. In NSDI. 137--150.
[8]
Junchen Jiang et al. 2017. Pytheas: Enabling Data-Driven Quality of Experience Optimization Using Group-Based Exploration-Exploitation. In NSDI, Vol. 1. 3.
[9]
Jessica Klein. 2018. Twitch Ended 2017 With 15 Million Daily Visitors, 27K Partnered Streamers. https://www.tubefilter.com/2018/02/06/twitch-2017-year-in-review/. Accessed February 6, 2018.
[10]
Xi Liu et al. 2012. A case for a coordinated internet video control plane. ACM SIGCOMM Computer Communication Review 42, 4 (2012), 359--370.
[11]
Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. 2017. Neural adaptive video streaming with pensieve. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. ACM, 197--210.
[12]
Volodymyr Mnih et al. 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529.
[13]
Mark Stemm and et al. 2000. A network measurement architecture for adaptive applications. In INFOCOM 2000. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings. IEEE, Vol. 1. IEEE, 285--294.
[14]
Richard S Sutton and Andrew G Barto. 1998. Reinforcement learning: An introduction. Vol. 1. MIT press Cambridge.
[15]
Ruben Torres et al. 2011. Dissecting video server selection strategies in the youtube cdn. In Distributed Computing Systems (ICDCS), 2011 31st International Conference on. IEEE, 248--257.
[16]
Andrew Trask et al. 2018. Neural arithmetic logic units. In Advances in Neural Information Processing Systems. 8046--8055.
[17]
Yifei Wei and et al. 2018. User Scheduling and Resource Allocation in HetNets With Hybrid Energy Supply: An Actor-Critic Reinforcement Learning Approach. IEEE Transactions on Wireless Communications 17, 1 (2018), 680--692.
[18]
Cheng-Zhong Xu, Jia Rao, and Xiangping Bu. 2012. URL: A unified reinforcement learning approach for autonomic cloud management. J. Parallel and Distrib. Comput. 72, 2 (2012), 95--105.
[19]
Zhiyuan Xu and et al. 2018. Experience-driven networking: A deep reinforcement learning based approach. arXiv preprint arXiv:1801.05757 (2018).
[20]
Zhu Yifei and et al. 2017. When cloud meets uncertain crowd: An auction approach for crowdsourced livecast transcoding. In Proceedings of the 25th ACM international conference on Multimedia. ACM, 1372--1380.

Cited By

View all
  • (2024)LiFteRProceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation10.5555/3691825.3691855(533-548)Online publication date: 16-Apr-2024
  • (2024)Hierarchical Game‐Theoretic Framework for Live Video Transmission with Dynamic Network Computing IntegrationInternational Journal of Intelligent Systems10.1155/2024/99289572024:1Online publication date: 28-Jun-2024
  • (2024)ImmerScope: Multi-view Video Aggregation at Edge towards Immersive Content ServicesProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems10.1145/3666025.3699324(82-96)Online publication date: 4-Nov-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
NOSSDAV '19: Proceedings of the 29th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video
June 2019
86 pages
ISBN:9781450362986
DOI:10.1145/3304112
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 June 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. crowdsourced live streaming
  2. reinforcement learning
  3. scheduling

Qualifiers

  • Research-article

Funding Sources

Conference

MMSys '19
Sponsor:
MMSys '19: 10th ACM Multimedia Systems Conference
June 21, 2019
Massachusetts, Amherst

Acceptance Rates

NOSSDAV '19 Paper Acceptance Rate 12 of 32 submissions, 38%;
Overall Acceptance Rate 118 of 363 submissions, 33%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)17
  • Downloads (Last 6 weeks)0
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)LiFteRProceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation10.5555/3691825.3691855(533-548)Online publication date: 16-Apr-2024
  • (2024)Hierarchical Game‐Theoretic Framework for Live Video Transmission with Dynamic Network Computing IntegrationInternational Journal of Intelligent Systems10.1155/2024/99289572024:1Online publication date: 28-Jun-2024
  • (2024)ImmerScope: Multi-view Video Aggregation at Edge towards Immersive Content ServicesProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems10.1145/3666025.3699324(82-96)Online publication date: 4-Nov-2024
  • (2024)VesperProceedings of the 15th ACM Multimedia Systems Conference10.1145/3625468.3647621(166-177)Online publication date: 15-Apr-2024
  • (2024)Adaptive Video Streaming in Multi-Tier Computing Networks: Joint Edge Transcoding and Client EnhancementIEEE Transactions on Mobile Computing10.1109/TMC.2023.326304623:4(2657-2670)Online publication date: Apr-2024
  • (2024)Seer: Proactive Revenue-Aware Scheduling for Live Streaming Services in Crowdsourced Cloud-Edge PlatformsIEEE INFOCOM 2024 - IEEE Conference on Computer Communications10.1109/INFOCOM52122.2024.10621424(1801-1810)Online publication date: 20-May-2024
  • (2024)A Survey on QoE Management Schemes for HTTP Adaptive Video Streaming: Challenges, Solutions, and OpportunitiesIEEE Access10.1109/ACCESS.2024.349161312(170803-170839)Online publication date: 2024
  • (2023)Practical Cloud-Edge Scheduling for Large-Scale Crowdsourced Live StreamingIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.326773134:7(2055-2071)Online publication date: Jul-2023
  • (2023)A Super-Resolution Flexible Video Coding Solution for Improving Live Streaming QualityIEEE Transactions on Multimedia10.1109/TMM.2022.320758025(6341-6355)Online publication date: 2023
  • (2023)Bi-Criteria Approximation for a Multi-Origin Multi-Channel Auto-Scaling Live Streaming CloudIEEE Transactions on Multimedia10.1109/TMM.2022.315209325(2839-2850)Online publication date: 2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media