Skip to main content
SpringerLink
Log in

Two-stage selection of distributed data centers based on deep reinforcement learning

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Cloud computing is a computing service provided on demand through the Internet, which can provide sufficient computing resources for applications such as big data and the Internet of Things. Before using cloud computing resources, users need to lease the corresponding virtual machines in the data centers, and then submit the tasks to the leased virtual machines for execution. In these two stages, how to choose the optimal data center for resource deployment and task submission is particularly important for cloud users. To tackle the problem that it is difficult to select optimal data center to lease and use virtual machines in distributed data centers, we proposed data center selection algorithms based on deep reinforcement learning. In the stage of virtual machine lease, aiming to minimizing user costs, we considered both the cost of computing resources and the cost of network communication. Then, we used a deep enforcement learning algorithm to obtain the shortest communication path between the users and the data centers,and solve the problem of network communication costs which are difficult to calculate. We achieved an optimal selection of data centers and effectively reduce the overall user cost. In the stage of virtual machine use, to improve quality of service for the users, we use a deep reinforcement learning algorithm to obtain an optimal task scheduling strategy. This effectively solved the problem that user tasks cannot be effectively scheduled due to dynamic changes in the type and size of user tasks and in the state of the virtual machines. Moreover, the proposed scheme reduces the overall task completion time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data Availability Statement

The data used to support the findings of this study are available from the corresponding author upon request.

References

  1. Xu, Z., Liang, W., Xia, Q.: Efficient Embedding of Virtual Networks to Distributed Clouds via Exploring Periodic Resource Demands. IEEE Trans. Cloud Comput. 6(3), 694–707 (2018)

    Article  Google Scholar 

  2. Xu, J., Palanisamy, B.: Cost-aware resource management for federated clouds using resource sharing contracts. In Procceding of IEEE 10th International Conference on Cloud Computing, Honolulu, HI, USA, pp. 238–245 (2017)

  3. Shen, H., Chen, L.: A resource usage intensity aware load balancing method for virtual machine migration in cloud datacenters. IEEE Trans. Cloud Comput. 8(1), 17–31 (2020)

    Article  Google Scholar 

  4. Mei, J., Li, K., Tong, Z., Li, Q., Li, K.: Profit maximization for cloud brokers in cloud computing. IEEE Trans. Parallel Distrib. Syst. 30(1), 190–203 (2019)

    Article  Google Scholar 

  5. Huang, B., Du, C., Sun, M., Yuan, X.: A dynamic virtual datacenter selection strategy for integrated cloud service platform construction with multiclouds. IEEE Access. 7, 172879–172891 (2019)

    Article  Google Scholar 

  6. Tram, T., Chen, T.: A novel model for competition and cooperation among cloud providers. IEEE Trans. Cloud Comput. 2(3), 251–265 (2014)

    Article  Google Scholar 

  7. Wang, W., Niu, D., Liang, B.: Dynamic cloud instance acquisition via iaas cloud brokerage. IEEE Trans. Parallel Distrib. Syst. 26(6), 1580–1593 (2014)

    Article  Google Scholar 

  8. Zhao, S., Zhu, Z.: Network service reconfiguration in hybrid optical/electrical datacenter networks. In: Proceedings of International Conference on Optical Network Design and Modeling, Castelldefels, Barcelona, Spain, pp. 1–6 (2020)

  9. Peng, Z., Cui, D., Ma, Y., Xiong, J., Xu B., Lin, W.: R-Learning and Gaussian process regression algorithm for cloud job access control. In: Proceedings of IEEE 3rd International Conference on Cyber Security and Cloud Computing, Beijing, China, pp. 163–166 (2016)

  10. Cui, D., Peng, Z., Xiong, J., Xu, B., Lin, W.: A reinforcement learning-based mixed job scheduler scheme for grid or IaaS cloud. IEEE Trans. Cloud Comput. 8(4), 1030–1039 (2020)

    Article  Google Scholar 

  11. Peng, Z., Lin, J., Cui, D., Li, Q., He, J.: A multi-objective trade-off framework for cloud resource scheduling based on the deep Q-network algorithm. Clust. Comput. 23, 2753–2767 (2020)

    Article  Google Scholar 

  12. Xiong, X., Zheng, K., Lei, L., Hou, L.: Resource allocation based on deep reinforcement learning in IoT edge computing. IEEE J. Sel. Areas Commun. 38(6), 1133–1146 (2020)

    Article  Google Scholar 

  13. Tomas, L., Kokkinos, P., Anagnostopoulos, V., Feder, O., Kyriazis, D., Meth, K., Varvarigos, E., Varvarigou, T.: Disaster recovery layer for distributed openstack deployments. IEEE Trans. Cloud Comput. 8(1), 112–123 (2017)

    Article  Google Scholar 

  14. Korte, B., Vygen, J.: Combinatorial Optimization Theory and Algorithms. Springer, Heidelberg (2008)

    MATH  Google Scholar 

  15. Li, Q., Peng, Z., Cui, D., He, J., Chen, K., Zhou, J.: Data center selection based on reinforcement learning. In: Proceedings of the 4th International Conference on Cloud Computing and Internet of Things, Changchun, China, pp. 14–19 (2019)

  16. Antti, M., Jukka, N.: Energy efficiency of mobile clients in cloud computing. In: Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing, Boston, MA, USA, pp. 1–10 (2010)

Download references

Acknowledgements

The work presented in this paper was supported by: National Natural Science Foundation of China(61672174, 61772145); Key platform and scientific research project of Guangdong education department(2017KTSCX128). Guangdong basic and applied basic research foundation (2020A 1515010727). Maoming Science and Technology Project(mmkj 2020008). Zhiping Peng and Delong Cui are corresponding authors.

Funding

This work was sponsored by National Natural Science Foundation of China(No.61672174,No.61772145), and so on.

Author information

Authors and Affiliations

  1. School of Computer, Guangdong University of Petrochemical Technology, Maoming, China

    Qirui Li, Zhiping Peng, Delong Cui, Jianpeng Lin & Jieguang He

  2. School of Mathematics and Information Science, Guangzhou University, Guangzhou, China

    Qirui Li

Authors
  1. Qirui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhiping Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Delong Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianpeng Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jieguang He
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Qirui Li and Zhiping Peng contributed to the conception of the study; Jianpeng Lin performed the experiment; Delong Cui contributed significantly to analysis and manuscript preparation; Jieguang He performed the data analyses and wrote the manuscript.

Corresponding authors

Correspondence to Zhiping Peng or Delong Cui.

Ethics declarations

Ethical statement

This work did not involved humans and animals. Ethics approval was not required for this research.

Consent statement

All authors signed the consent form.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Q., Peng, Z., Cui, D. et al. Two-stage selection of distributed data centers based on deep reinforcement learning. Cluster Comput 25, 2699–2714 (2022). https://doi.org/10.1007/s10586-021-03525-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-021-03525-8

Keywords

Search

Navigation