Location-Aware and Budget-Constrained Service Brokering in Multi-Cloud via Deep Reinforcement Learning

Abstract

Multi-cloud makes it possible to effectively utilize various cloud services provided by multiple cloud providers at different locations. To process the requests for latency-sensitive applications, cloud brokers must select proper cloud services in multi-cloud to minimize the network latency without running into the risk of over-spending. The problem of location-aware and budget-constrained service brokering in multi-cloud demands a machine learning approach to handle the highly dynamic requests. In this paper, we apply deep reinforcement learning to solve the problem. The proposed algorithm, named DeepBroker, can dynamically and adaptively select virtual machines in multi-cloud for new arriving requests at a global scale. Specifically, DeepBroker trains brokering policies by employing a deep Q-network combined with the newly designed state extractor and action executor. To ensure financial viability, we introduce a penalty-based reward function to prevent over-budget situations. Evaluation based on real-world datasets shows that DeepBroker can significantly outperform several commonly used heuristic-based algorithms in terms of network latency minimization and budget satisfaction.