Skip to main content

Advertisement

SpringerLink
Log in

Constraint-aware VM placement in heterogeneous computing clusters

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Virtualized systems consist of a large number of machines that are configured with different hardware and software, and execute a large number of virtual machines (VMs) for diverse applications. There can be various constraint conditions of placing VMs in such systems due to the concerns on security, availability, performance, etc. However, VM placement constraints can limit the choice of hosts for VMs, affecting the performance of the systems negatively. In this paper, we study constraint-aware VM placement in heterogeneous computing clusters. We first present a model of VM placement constraints that supports all types of constraints between VMs, and between VMs and hosts. Second, we discuss six constraint-aware VM placement algorithms which optimize the performance for either energy saving or load balancing. Third, using simulations, we analyze the effects of different types of VM placement constraints on VM placement, and evaluate the performance of the algorithms over various settings. We also run experiments of the algorithms in a small cluster. Our extensive simulation results demonstrate that the effects of VM placement constraints vary, depending on the optimization goal, the types of the constraints, and the system configurations. For the constraint-aware algorithms, we show that an energy saving algorithm which attempts to place a new VM on one of active hosts by utilizing VM migrations, and a load balancing algorithm which attempts to migrate some VMs from a selected host for a new VM, i.e. a potential hotspot, to another host provide good performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Sharma, B., Chudnovsky, V., Hellerstein, J.L., Rifaat, R., Das, C.R.: Modeling and synthesizing task placement constraints in Google compute clusters. In: Proceedings of the 2nd ACM Symposium on Cloud Computing (SOCC) (2011)

  2. Reiss, C., Tumanov, A., Ganger, G.R., Katz, R.H., Kozuch, M.A.: Heterogeneity and dynamicity of clouds at scale: Google trace analysis. In: Proceedings of the Third ACM Symposium on Cloud Computing (SOCC) (2012)

  3. VMware VMotion and CPU Compatibility. http://www.vmware.com/files/vmotion-info-guide.pdf

  4. ISV Licensing in Virtualized Environments. https://www.vmware.com/08Q1_wp_vmw_ISV_Licensing.pdf

  5. Ristenpart, T., Tromer, E., Shacham, H., Savage, S.: Hey, you, get off of my cloud: exploring information leakage in third-party compute clouds. In: Proceedings of the 16th ACM Conference on Computer and Communications Security (CCS) (2009)

  6. Scales, D.J., Nelson, M., Venkitachalam, G.: The design of a practical system for fault-tolerant virtual machines. SIGOPS Oper. Syst. Rev. 44, 30–39 (2010)

    Article  Google Scholar 

  7. Wood, T., Tarasuk-Levin, G., Shenoy, P., Desnoyers, P., Cecchet, E., Corner, M.D.: Memory buddies: exploiting page sharing for smart colocation in virtualized data centers. In: Proceedings of the ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE) (2009)

  8. Sonnek, J., Greensky, J., Reutiman, R., Chandra, A.: Starling: Minimizing communication overhead in virtualized computing platforms using decentralized affinity-aware migration. In: Proceedings of the 39th International Conference on Parallel Processing (ICPP) (2010)

  9. Thinakaran, P., Gunasekaran, J.R., Sharma, B., Kandemir, M.T., Das, C.R.: Phoenix: A constraint-aware scheduler for heterogeneous datacenters. In: Proceedings of the IEEE 37th International Conference on Distributed Computing Systems (ICDCS), pp. 977–987 (2017)

  10. VMware vSphere. http://www.vmware.com

  11. Microsoft SCVMM. http://www.microsoft.com

  12. XenServer. http://www.xenserver.org

  13. Bobroff, N., Kochut, A., Beaty, K.: Dynamic placement of virtual machines for managing sla violations. In: Proceedings of the 10th IFIP/IEEE International Symposium on Integrated Network Management (2007)

  14. Dong, J., Jin, X., Wang, H., Li, Y., Zhang, P., Cheng, S.: Energy-saving virtual machine placement in cloud data centers. In: Proceedings of the 13th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) (2013)

  15. Arzuaga, E., Kaeli, D.R.: Quantifying load imbalance on virtualized enterprise servers. In: Proceedings of the First Joint WOSP/SIPEW International Conference on Performance Engineering (2010)

  16. Tumanov, A., Cipar, J., Ganger, G.R., Kozuch, M.A.: alsched: algebraic scheduling of mixed workloads in heterogeneous clouds. In: Proceedings of the Third ACM Symposium on Cloud Computing (SOCC) (2012)

  17. Ghodsi, A., Zaharia, M., Shenker, S., Stoica, I.: Choosy: max–min fair sharing for datacenter jobs with constraints. In: Proceedings of the 8th ACM European Conference on Computer Systems (EuroSys) (2013)

  18. Sonnek, J., Chandra, A.: Virtual putty: Reshaping the physical footprint of virtual machines. In: Proceedings of the 3rd USENIX Workshop on Hot Topics in Cloud Computing, HotCloud ’09 (2009)

  19. Waldspurger, C.A.: Memory resource management in VMware ESX server. SIGOPS Oper. Syst. Rev. 36(SI), 181–194 (2002)

    Article  Google Scholar 

  20. Jayasinghe, D., Pu, C., Eilam, T., Steinder, M., Whally, I., Snible, E.: Improving performance and availability of services hosted on iaas clouds with structural constraint-aware virtual machine placement. In: Proceedings of IEEE International Conference on Services Computing (SCC) (2011)

  21. VMware Distributed Resource Management. https://labs.vmware.com/academic/publications/gulati-vmtj-spring2012

  22. Gulati, A., Shanmuganathan, G., Holler, A.: Cloud-scale resource management: Challenges and techniques. In: Proceedings of the 3rd USENIX Conference on Hot Topics in Cloud Computing (HotCloud) (2011)

  23. Calheiros, R.N., Ranjan, R., Beloglazov, A., De Rose, C.A.F., Buyya, R.: Cloudsim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw. Pract. Exp. 41, 23–50 (2011)

    Article  Google Scholar 

  24. BMC documentation. https://docs.bmc.com/docs/display/bcmco90/Managing+Cloud+Capacity+Visibility+view+threshold+settings

  25. Clark, C., Fraser, K., Hand, S., Hansen, J.G., Jul, E., Limpach, C., Pratt, I., Warfield, A.: Live migration of virtual machines. In: Proceedings of the 2nd Conference on Symposium on Networked Systems Design & Implementation, Vol. 2, NSDI’05, pp. 273–286 (2005)

  26. Jo, C., Cho, Y., Egger, B.: A machine learning approach to live migration modeling. In: Proceedings of the 2017 Symposium on Cloud Computing, vol. 17, pp. 351–364. SoCC (2017)

  27. Reiss, C., Wilkes, J., Hellerstein, J.L.: Google cluster-usage traces: format+ schema. Google Inc., White Paper pp. 1–14 (2011)

  28. Bin, E., Biran, O., Boni, O., Hadad, E., Kolodner, E., Moatti, Y., Lorenz, D.: Guaranteeing high availability goals for virtual machine placement. In: Proceedings of the 31st International Conference on Distributed Computing Systems (ICDCS) (2011)

  29. Keller, G., Lutfiyya, H.: Dynamic management of applications with constraints in virtualized data centres. In: Proceedings of IFIP/IEEE International Symposium on Integrated Network Management (IM) (2015)

  30. Jhawar, R., Piuri, V., Samarati, P.: Supporting security requirements for resource management in cloud computing. In: Proceedings of IEEE 15th International Conference on Computational Science and Engineering (CSE) (2012)

  31. Shi, L., Butler, B., Botvich, D., Jennings, B.: Provisioning of requests for virtual machine sets with placement constraints in iaas clouds. In: Proceedings of IFIP/IEEE International Symposium on Integrated Network Management (IM) (2013)

  32. Zhang, Y., Prekas, G., Fumarola, G.M., Fontoura, M., Goiri, I., Bianchini, R.: History-based harvesting of spare cycles and storage in large-scale datacenters. In: Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), pp. 755–770 (2016)

  33. Google ROADEF’12. http://challenge.roadef.org/2012/en/

  34. OptaPlanner. https://www.optaplanner.org

  35. Van, H.N., Tran, F.D., Menaud, J.: Performance and power management for cloud infrastructures. In: Proceedings of the IEEE 3rd International Conference on Cloud Computing, pp. 329–336 (2010)

  36. Zhang, L., Zhuang, Y., Zhu, W.: Constraint programming based virtual cloud resources allocation model. Int. J. Hybrid Inf. Technol. 6, 333–344 (2013)

    Article  Google Scholar 

  37. Raman, R., Livny, M., Solomon, M.: Matchmaking: distributed resource management for high throughput computing. In: Proceedings of the Seventh International Symposium on High Performance Distributed Computing, pp. 140–146 (1998)

  38. Majumder, T., Borgens, M.E., Pande, P.P., Kalyanaraman, A.: On-chip network-enabled multicore platforms targeting maximum likelihood phylogeny reconstruction. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 31(7), 1061–1073 (2012)

    Article  Google Scholar 

  39. Guérout, T., Medjiah, S., Costa, G.D., Monteil, T.: Quality of service modeling for green scheduling in clouds. Sustain. Comput. 4, 225–240 (2014)

    Google Scholar 

  40. Sal, M.I., Vikas, M., Adarsh, J.: DRS Performance—VMware vSphere 6.5. VMware Inc., White Paper pp. 1–27 (2016)

Download references

Acknowledgements

This work was partly supported by Institute for Information & Communications Technology Promotion (IITP) Grant funded by the Korea Government (MSIT) (No. 2015-0-00590, High Performance Big Data Analytics Platform Performance Acceleration Technologies Development) and National Research Foundation of Korea (NRF) funded by the Korea Government (MSIT) (NRF-2018R1A2B6006107 and NRF-2016M3C4A7952634).

Author information

Authors and Affiliations

  1. School of Electrical and Computer Engineering, Ulsan National Institute of Science and Technology (UNIST), Ulsan, Korea

    Seontae Kim & Young-ri Choi

Authors
  1. Seontae Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Young-ri Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Young-ri Choi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, S., Choi, Yr. Constraint-aware VM placement in heterogeneous computing clusters. Cluster Comput 23, 71–85 (2020). https://doi.org/10.1007/s10586-019-02966-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-019-02966-6

Keywords

Search

Navigation