Abstract
In video streaming over HTTP, the bitrate adaptation selects the quality of video chunks depending on the current network condition. Some previous works have applied deep reinforcement learning (DRL) algorithms to determine the chunk’s bitrate from the observed states to maximize the quality-of-experience (QoE). However, to build an intelligent model that can predict in various environments, such as 3G, 4G, Wifi, etc., the states observed from these environments must be sent to a server for training centrally.
In this work, we integrate federated learning (FL) to DRL-based rate adaptation to train a model appropriate for different environments. The clients in the proposed framework train their model locally and only update the weights to the server. The simulations show that our federated DRL-based rate adaptations, called FDRLABR with different DRL algorithms, such as deep Q-learning, advantage actor-critic, and proximal policy optimization, yield better performance than the traditional bitrate adaptation methods in various environments.
This research is funded by Vietnam National University HoChiMinh City (VNU-HCM) under grant number DS2020-28-01, and in part by the Vietnam National Foundation for Science and Technology Development (NAFOSTED) under Grant 102.02-2019.321.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Stockhammer, T.: Dynamic adaptive streaming over HTTP- standards and design principles. In: Proceedings of the Second Annual ACM Conference on Multimedia Systems, pp. 133–144 (2011)
Lederer, S., Müller, C., Timmerer, C.: Dynamic adaptive streaming over HTTP dataset. In: Proceedings of the 3rd Multimedia Systems Conference, pp. 89–94 (2012). https://ftp.itec.aau.at/datasets/mmsys12/ElephantsDream/ed_4s/
Spiteri, K., Urgaonkar, R., Sitaraman, R.K.: BOLA: near-optimal bitrate adaptation for online videos. IEEE/ACM Trans. Netw. 28(4), 1698–1711 (2020)
Claeys, M., Latré, S., Famaey, J., Wu, T., Van Leekwijck, W., De Turck, F.: Design and optimisation of a (FA) Q-learning-based HTTP adaptive streaming client. Connect. Sci. 26(1), 25–43 (2014)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT press, Cambridge (2018)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Mnih, V., et al.: Asynchronous methods for deep reinforcement learning. In: International Conference on Machine Learning PMLR, pp. 1928–1937 (2016)
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)
Mao, H., Netravali, R., Alizadeh, M.: Neural adaptive video streaming with pensieve. In: Proceedings of the Conference of the ACM Special Interest Group on Data Communication, pp. 197–210 (2017)
Gadaleta, M., Chiariotti, F., Rossi, M., Zanella, A.: D-DASH: a deep Q-learning framework for DASH video streaming. IEEE Trans. Cogn. Commun. Netw. 3(4), 703–718 (2017)
Liu, J., Tao, X., Lu, J.: QoE-oriented rate adaptation for DASH with enhanced deep Q-learning. IEEE Access 7, 8454–8469 (2018)
Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., Meger, D.: Deep reinforcement learning that matters. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1 (2018)
McMahan, B., Moore, E., Ramage, D., Hampson, S., y Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics PMLR, pp. 1273–1282 (2017)
Qi, J., Zhou, Q., Lei, L., Zheng, K.: Federated reinforcement learning: techniques, applications, and open challenges. arXiv preprint arXiv:2108.11887 (2021)
Zhang, C., Xie, Y., Bai, H., Yu, B., Li, W., Gao, Y.: A survey on federated learning. Knowl.-Based Syst. 216, 106775 (2021)
Raca, D., Quinlan, J.J., Zahran, A.H., Sreenan, C.J.: Beyond throughput: a 4G LTE dataset with channel and context metrics. In: Proceedings of the 9th ACM Multimedia Systems Conference, pp. 460–465 (2018)
US Federal Communications Commission (FCC). https://data.fcc.gov/download/measuring-broadband-america/2019/data-raw-2019-sept.tar.gz
Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., Dormann, N.: Stable-Baselines3: reliable reinforcement learning implementations. J. Mach. Learn. Res. 22(1), 12348–12355 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Vo, P.L., Nguyen, N.T., Luu, L., Dinh, C.T., Tran, N.H., Le, TA. (2023). Federated Deep Reinforcement Learning - Based Bitrate Adaptation for Dynamic Adaptive Streaming over HTTP. In: Nguyen, N.T., et al. Recent Challenges in Intelligent Information and Database Systems. ACIIDS 2023. Communications in Computer and Information Science, vol 1863. Springer, Cham. https://doi.org/10.1007/978-3-031-42430-4_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-42430-4_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-42429-8
Online ISBN: 978-3-031-42430-4
eBook Packages: Computer ScienceComputer Science (R0)