Waltenegus Dargie
ABSTRACT
The introduction of virtualization and cloud computing has enabled a large number of containers/virtual machines to share computing resources. Nevertheless, the number and size of data centres are still on the rise, partly on account of an ever increasing amount of generated data and workloads worldwide. On the other hand, independent studies indicate that a large number of servers in contemporary data centres are underutilised. One of the strategies currently adopted by the research community in order to deal with resource inefficiency is dynamic workload consolidation. The idea behind is dynamically balancing the supply of computing, communication, and storage resources with the demand for resources. This entails populating physical servers with an optimal number of complementary workloads. Most existing or proposed approaches employ multi-variate optimisation to achieve this goal but do not easily lend themselves to fast and intuitive solutions. In this paper, we investigate the scope and usefulness of dimensionality reduction techniques (tensor decomposition) to identify execution and resource utilisation patterns in hosted containers/virtual machines. Our analysis is based on two large-scale data centres, one of them hosts 1190 commercial virtual machines on 59 physical computing servers and 29 physical storage servers organised in 9 clusters and the other 44373 containers on 3985 physical servers. Our analysis shows that spatial and temporal patters can be uncovered with tensor decomposition, based on which efficient clustering can be realised.
- Claudia Canali and Riccardo Lancellotti. 2014. Improving scalability of cloud monitoring through PCA-based clustering of virtual machines. Journal of Computer Science and Technology 29, 1 (2014), 38--52.Google Scholar
Cross Ref
- Carlo Curino, Subru Krishnan, Konstantinos Karanasos, Sriram Rao, Giovanni Matteo Fumarola, Botong Huang, Kishore Chaliparambil, Arun Suresh, Young Chen, Solom Heddaya, et al. 2019. Hydra: a federated resource manager for data-center scale analytics.. In NSDI. 177--192.Google Scholar
- Waltenegus Dargie. 2014. A stochastic model for estimating the power consumption of a processor. IEEE Trans. Comput. 64, 5 (2014), 1311--1322.Google ScholarCross Ref
- Md Hasanul Ferdaus, Manzur Murshed, Rodrigo N Calheiros, and Rajkumar Buyya. 2014. Virtual machine consolidation in cloud data centers using ACO metaheuristic. In European Conference on Parallel Processing. Springer, 306--317.Google ScholarCross Ref
- Mostafa Ghobaei-Arani, Sam Jabbehdari, and Mohammad Ali Pourmina. 2018. An autonomic resource provisioning approach for service-based cloud applications: A hybrid approach. Future Generation Computer Systems 78 (2018), 191--210.Google ScholarCross Ref
- Paul C Gilmore and Ralph E Gomory. 1961. A linear programming approach to the cutting-stock problem. Operations research 9, 6 (1961), 849--859.Google Scholar
- Gene H Golub and Christian Reinsch. 1971. Singular value decomposition and least squares solutions. In Linear Algebra. Springer, 134--151.Google Scholar
- Markus Haehnel, John Martinovic, Guntram Scheithauer, Andreas Fischer, Alexander Schill, and Waltenegus Dargie. 2018. Extending the Cutting Stock Problem for Consolidating Services with StochasticWorkloads. IEEE Transactions on Parallel and Distributed Systems (2018).Google Scholar
- Kai Hwang and Naresh Jotwani. 2016. Advanced Computer Architecture, 3e. McGraw-Hill Education.Google Scholar
- Tamara G Kolda and BrettWBader. 2009. Tensor decompositions and applications. SIAM review 51, 3 (2009), 455--500.Google Scholar
- John Martinovic, Markus Hähnel, Guntram Scheithauer, Waltenegus Dargie, and Andreas Fischer. 2019. Cutting stock problems with nondeterministic item lengths: a new approach to server consolidation. 4OR 17, 2 (2019), 173--200.Google Scholar
- Christoph Möbius, Waltenegus Dargie, and Alexander Schill. 2013. Power consumption estimation models for processors, virtual machines, and servers. IEEE Transactions on Parallel and Distributed Systems 25, 6 (2013), 1600--1614.Google ScholarDigital Library
- Roberto Morabito, Vittorio Cozzolino, Aaron Yi Ding, Nicklas Beijar, and Jorg Ott. 2018. Consolidate IoT edge computing with lightweight virtualization. IEEE Network 32, 1 (2018), 102--111.Google ScholarCross Ref
- Trung Hieu Nguyen, Mario Di Francesco, and Antti Yla-Jaaski. 2017. Virtual machine consolidation with multiple usage prediction for energy-efficient cloud data centers. IEEE Transactions on Services Computing (2017).Google Scholar
- Ali Pahlevan, Xiaoyu Qu, Marina Zapater, and David Atienza. 2017. Integrating Heuristic and Machine-Learning Methods for Efficient Virtual Machine Allocation in Data Centers. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (2017).Google Scholar
- Maolin Tang and Shenchen Pan. 2015. A hybrid genetic algorithm for the energyefficient virtual machine placement problem in data centers. Neural Processing Letters 41, 2 (2015), 211--221.Google ScholarDigital Library
- Adel Nadjaran Toosi, Richard O Sinnott, and Rajkumar Buyya. 2018. Resource provisioning for data-intensive applications with deadline constraints on hybrid clouds using Aneka. Future Generation Computer Systems 79 (2018), 765--775.Google ScholarDigital Library
- Thang X Vu, Symeon Chatzinotas, and Bjorn Ottersten. 2018. Edge-caching wireless networks: Performance analysis and optimization. IEEE Transactions on Wireless Communications 17, 4 (2018), 2827--2839.Google ScholarCross Ref
- Lei Yu, Liuhua Chen, Zhipeng Cai, Haiying Shen, Yi Liang, and Yi Pan. 2016. Stochastic Load Balancing for Virtual Resource Management in Datacenters. IEEE Transactions on Cloud Computing PP, 99 (2016), 1--1. https://doi.org/10. 1109/TCC.2016.2525984Google Scholar
- Alice Zheng and Amanda Casari. 2018. Feature engineering for machine learning: principles and techniques for data scientists. " O'Reilly Media, Inc.".Google Scholar
Index Terms
- Tensor-Based Resource Utilization Characterization in a Large-Scale Cloud Infrastructure
Recommendations
Autonomous learning for efficient resource utilization of dynamic VM migration
ICS '08: Proceedings of the 22nd annual international conference on SupercomputingDynamic migration of virtual machines on a cluster of physical machines is designed to maximize resource utilization by balancing loads across the cluster. When the utilization of a physical machine is beyond a fixed threshold, the machine is deemed ...
Optimizing Resource allocation while handling SLA violations in Cloud Computing platforms
IPDPS '13: Proceedings of the 2013 IEEE 27th International Symposium on Parallel and Distributed ProcessingIn this paper, we study a resource allocation problem in the context of Cloud Computing, in which a set of Virtual Machines (VM) has to be allocated on a set of Physical Machines (PM). Each VM has a given demand (e.g. CPU demand), and each PM has a ...
Cloud resource allocation schemes: review, taxonomy, and opportunities
Cloud computing has emerged as a popular computing model to process data and execute computationally intensive applications in a pay-as-you-go manner. Due to the ever-increasing demand for cloud-based applications, it is becoming difficult to ...
Comments