Abstract
Cloud data centers rely on virtualization to run a diverse set of applications. Container technology allows for a more lightweight execution, in comparison with popular Virtual Machines. Efficient scheduling of containers is still challenging due to varying request arrival patterns, application-specific resource consumption and resource heterogeneity in physical servers. Besides, containers are also more prone to resource contention and performance interference. Cloud providers need to overcome these challenges with a goal in mind: maximize resource utilization to satisfy as many requests as possible. This paper introduces RLSched, a deep reinforcement learning-based (DRL) scheduler that is self-adaptive and automatically captures the resource usage dynamics in the data center. The scheduler is based on a decentralized actor-critic multi-agent architecture that enables for parallel execution and faster convergence. RLSched relies on an enhanced network model with action shaping, which filters invalid actions and prevents the agent to fall into a sub-optimal policy. The proposed scheduler is compared against other state-of-the-art DRL methods on a simulated data center environment based on real traces from Microsoft Azure. The results show faster convergence and higher number of containers placed per session.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Csirik, J., Woeginger, G.J.: On-line packing and covering problems. In: Online Algorithms, pp. 147–177 (1998)
Mnih, V., et al.: Playing atari with deep reinforcement learning. In: arXiv preprint arXiv:1312.5602 (2013)
Shenglin, L., Ni, M., Zhang, H.-B.: The optimization of scheduling strategy based on the Docker swarm cluster. Inf. Technol. 40(7), 147–151 (2016)
Mnih, V., et al.: Asynchronous methods for deep reinforcement learning. In: International Conference on Machine Learning. PMLR, pp. 1928–1937 (2016)
Kaewkasi, C., Chuenmuneewong, K.: Improvement of container scheduling for docker using ant colony optimization. In : 2017 9th International Conference on Knowledge and Smart Technology (KST), pp. 254–259. IEEE (2017)
Schulman, J., et al.: Proximal policy optimization algorithms. In: arXiv preprint arXiv:17 07.06347 (2017)
Wang , Z., et al.: Automated cloud provisioning on aws using deep reinforcement learning. In: arXiv preprint arXiv:1709.04305 (2017)
Zhang, D., et al.: Container oriented job scheduling using linear programming model. In: 2017 3rd International Conference on Information Management (ICIM), pp. 174–180. IEEE (2017)
Bhimani, J., et al.: Docker container scheduler for I/O intensive applications running on NVMe SSDs. In: IEEE Transactions on Multi-Scale Computing Systems, 4.3, pp. 313–326 (2018)
Zhang, Q., et al.: A double deep Q-learning model for energy-efficient edge scheduling. IEEE Trans. Serv. Comput. 12(5), 739–749 (2018)
Bingqian, D., Chuan, W., Huang, Z.: Learning resource allocation and pricing for cloud profit maximization. In: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33(01), pp. 7570–7577 (2019)
Mseddi, A., et al.: Joint container placement and task provisioning in dynamic fog computing. IEEE Internet of Things J. 6(6), 10028–10040 (2019)
Zhang, R., et al.: A genetic algorithm-based energy-efficient container placement strategy in CaaS. IEEE Access 7, 121360–121373 (2019)
Funika, W., Koperek, P., Kitowski, J.: Automatic management of cloud applications with use of Proximal Policy Optimization. In: International Conference on Computational Science, pp. 73–87 Springer (2020)
Kanervisto, A., Scheller, C., Hautamäki, V.: Action space shaping in deep reinforcement learning. In: 2020 IEEE Conference on Games (CoG), pp. 479–486 IEEE (2020)
Zhang, S., et al.: A-SARSA: a predictive container auto-scaling algorithm based on reinforcement learning. In: 2020 IEEE International Conference on Web Services (ICWS), pp. 489–497 IEEE (2020)
Docker. Docker Swarm. https://docs.docker.com/engine/swarm/how-swarm-mode-works/services/. Accessed Jan 2021
Kubernetes. Google Kubernetes. https://kubernetes.io/docs/concepts/scheduling-eviction/kube-scheduler/. Accessed Jan 2021
MS. Microsoft Azure Kubernetes Service (AKS). https://azure.microsoft.com/en-us/services/kubernetes-service/. Accessed Jan 2021
Trace. Azure Public Dataset. https://github.com/Azure/AzurePublicDa%20taset. Accessed Jan 2021
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lorido-Botran, T., Bhatti, M.K. (2021). Adaptive Container Scheduling in Cloud Data Centers: A Deep Reinforcement Learning Approach. In: Barolli, L., Woungang, I., Enokido, T. (eds) Advanced Information Networking and Applications. AINA 2021. Lecture Notes in Networks and Systems, vol 227. Springer, Cham. https://doi.org/10.1007/978-3-030-75078-7_57
Download citation
DOI: https://doi.org/10.1007/978-3-030-75078-7_57
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-75077-0
Online ISBN: 978-3-030-75078-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)