Skip to main content
SpringerLink
Log in

Raven: Scheduling Virtual Machine Migration During Datacenter Upgrades with Reinforcement Learning

  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

Physical machines in modern datacenters are routinely upgraded due to their maintenance requirements, which involve migrating all the virtual machines they currently host to alternative physical machines. For this kind of datacenter upgrades, it is critical to minimize the time it takes to upgrade all the physical machines in the datacenter, so as to reduce disruptions to cloud services. To minimize the upgrade time, it is essential to carefully schedule the migration of virtual machines on each physical machine during its upgrade, without violating any constraints imposed by virtual machines that are currently running. Rather than resorting to heuristic algorithms as existing work, we propose a new scheduler, Raven, that uses an experience-driven approach with deep reinforcement learning to schedule the virtual machine migration. With our design of the state space, action space and reward function, Raven trains a fully-connected neural network using the cross-entropy method to approximate the policy of choosing a destination physical machine for each virtual machine before its migration. We compare Raven with state-of-the-art algorithms in the literature, and our results show that Raven can effectively shorten the time to complete the datacenter upgrade under different datacenter settings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Beloglazov A, Abawajy J, Buyya R (2012) Energy-Aware Resource Allocation Heuristics for Efficient Management of Data Centers for Cloud Computing. Future generation computer systems 28(5):755–768

    Article  Google Scholar 

  2. Maurya K, Sinha R (2013) Energy conscious dynamic provisioning of virtual machines using adaptive migration thresholds in cloud data center. International Journal of Computer Science and Mobile Computing 2 (3):74–82

    Google Scholar 

  3. Luo J, Fan X, Yin L (2019) Communication-Aware and Energy Saving Virtual Machine Allocation Algorithm in Data Center. In: 21st IEEE International Conference on High Performance Computing and Communications. IEEE, pp 819–826

  4. Ji S, Li M D, Ji N, Li B (2018) An online virtual machine placement algorithm in an over-committed cloud. In: 2018 IEEE International Conference on Cloud Engineering, IC2E 2018, Orlando, FL, USA, April 17-20, 2018, pp 106–112

  5. Sutton R S, Barto A G (1998) Reinforcement learning: An introduction. MIT Press Cambridge, MA, USA.

  6. Mnih V, Kavukcuoglu K, Silver D, Rusu A A, Veness J, Bellemare M G, Graves A, Riedmiller M, Fidjeland A K, Ostrovski G, Petersen S, Beattie C, Sadik A, Antonoglou I, King H, Kumaran D, Wierstra D, Legg S, Hassabis D (2015) Human-level control through deep reinforcement learning. Nature 518:529–533

    Article  Google Scholar 

  7. Mao H, Alizadeh M, Menache I, Kandula S (2016) Resource management with deep reinforcement learning. In: Proceedings of the 15th ACM Workshop on Hot Topics in Networks, ACM, pp 50–56

  8. Silver D, Huang A, Maddison C J, et al. (2016) Mastering the game of go with deep neural networks and tree search. nature 529(7587):484–489

    Article  Google Scholar 

  9. Silver D, Schrittwieser J, Simonyan K, Antonoglou I, et al. (2017) Mastering the game of go without human knowledge. Nature 550:354–359

    Article  Google Scholar 

  10. Rubinstein RY, Kroese DP (2004) The cross-entropy method. Springer New York, NY, USA.

  11. Ahmad R W, Gani A, Hamid S H A, Shiraz M, Xia F, Madani S A (2015) Virtual Machine Migration in Cloud Data Centers: a Review, Taxonomy, and Open Research Issues. The Journal of Supercomputing 71(7):2473–2515

    Article  Google Scholar 

  12. Borgetto D, Maurer M, et al. (2012) Energy-Efficient and SLA-Aware Management of IaaS Clouds. In: Proceedings of the 3rd International Conference on Energy-Efficient Computing and Networking, Madrid, Spain, May 9-11, 2012. ACM, p 25

  13. Garg S K, Toosi A N, Gopalaiyengar S K, Buyya R (2014) SLA-Based Virtual Machine Management for Heterogeneous Workloads in a Cloud Datacenter. J. Netw. Comput. Appl. 45:108–120

    Article  Google Scholar 

  14. Hieu N T, Francesco M D, et al. (2014) A Virtual Machine Placement Algorithm for Balanced Resource Utilization in Cloud Data Centers. In: 2014 IEEE 7th International Conference on Cloud Computing, Anchorage, AK, USA, June 27 - July 2, 2014. IEEE Computer Society, pp 474–481

  15. Zhang X, Shae Z-Y, Zheng S, Jamjoom H (2012) Virtual Machine Migration in an Over-Committed Cloud. In: Proc. IEEE Network Operations and Management Symposium (NOMS)

  16. Dabbagh M, Hamdaoui B, Guizani M, Rayes A (2015) Efficient Datacenter Resource Utilization Through Cloud Resource Overcommitment. In: Proc. IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)

Download references

Author information

Authors and Affiliations

  1. University of Toronto, Toronto, Canada

    Chen Ying & Baochun Li

  2. Huawei Canada, Markham, Canada

    Xiaodi Ke & Lei Guo

Authors
  1. Chen Ying
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Baochun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaodi Ke
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lei Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chen Ying.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ying, C., Li, B., Ke, X. et al. Raven: Scheduling Virtual Machine Migration During Datacenter Upgrades with Reinforcement Learning. Mobile Netw Appl 27, 303–314 (2022). https://doi.org/10.1007/s11036-020-01632-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-020-01632-1

Keywords

Search

Navigation