Lixi Wang
ABSTRACT
Existing resource management solutions in datacenters and cloud systems typically treat VMs as black boxes when making resource allocation decisions. This paper advocates the cooperation between VM host- and guest-layer schedulers for optimizing the resource management and application performance. It presents an approach to such cross-layer optimization upon fuzzy-modeling-based resource management. This approach exploits guest-layer application knowledge to capture workload characteristics and improve VM modeling, and enables the host-layer scheduler to feedback resource allocation decision and adapt guest-layer application configuration. As a case study, this approach is applied to virtualized databases which have challenging dynamic, complex resource usage behaviors. Specifically, it characterizes query workloads based on a database's internal cost estimation and adapts query executions by tuning the cost model parameters according to changing resource availability. A prototype of the proposed approach is implemented on Xen VMs and evaluated using workloads based on TPC-H and RUBiS. The results show that with guest-to-host workload characterization, resources can be efficiently allocated to database VMs serving workloads with changing intensity and composition while meeting Quality-of-Service (QoS) targets. For TPC-H, the prediction error for VM resource demand is less than 3.5%; for RUBiS, the response time target is met for 92% of the time. Both significantly outperform the resource allocation scheme without workload characterization. With host-to-guest database adaptation, the performance of TPC-H-based workloads is also improved by 17% when the VM's available I/O bandwidth is reduced due to contention.
- VMware, URL: http://www.vmware.com.Google Scholar
- P. Barham, Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebauer, R., Pratt, I. and Warfield, A, "Xen and the Art of Virtualization", SOSP, 2003. Google ScholarDigital Library
- Amazon Elastic Compute Cloud, URL: http://aws.amazon.com/ec2/.Google Scholar
- Windows Azure, URL: http://www.microsoft.com/windowsazure/.Google Scholar
- L. Wang, J. Xu, M. Zhao, Y. Tu and J. A.B. Fortes, "Fuzzy Modeling Based Resource Management for Virtualized Database Systems", MASCOTS, 2011. Google ScholarDigital Library
- TPC-H Benchmark Specification, URL: http://www. tcp. org.Google Scholar
- C. Amza, A. Chanda, A. Cox, S. Elnikety, R. Gil, K. Rajamani and W. Zwaenepoel, "Specification and Implementation of Dynamic Web Site Benchmarks", WWC-5, 2002.Google Scholar
- J. Xu, M. Zhao and J. Fortes, "Autonomic Resource Management in Virtualized Data Centers Using Fuzzy-logic-based Control", Cluster Computing, 2008. Google ScholarDigital Library
- S. Chiu, "Fuzzy Model Identification Based on Cluster Estimation", Journal of Intelligent and Fuzzy Systems, 1994.Google Scholar
- J. Liu, R. Rangaswami, and M. Zhao, "Model-Driven Network Emulation With Virtual Time Machine", Winter Simulation Conference, December 2010. Google ScholarDigital Library
- A. Chen, P. Goes, A. Gupta and J. Marsden, "Heuristics for Selecting Robust Database Structures with Dynamic Query Patterns", EJOR, 2006.Google ScholarCross Ref
- M. Wang, T. Madhyastha, N. Chan, S. Papadimitriou and C. Faloutsos, "Data Mining Meets Performance Evaluation: Fast Algorithms for Modeling Bursty Traffic", ICDE, 2002.Google Scholar
- S. Chaudhuri, "Relational Query Optimization -- Data Management Meets Statistical Estimation", Communications of ACM, 2009. Google ScholarDigital Library
- dm-ioband, URL: http://sourceforge.net/apps/trac/ioband.Google Scholar
- M. Arlitt and T. Jin, "Workload Characterization of the 1998 World Cup Web Site," in HP Technical Report, 1999.Google Scholar
- Z. Gong and X. Gu, "PAC: Pattern-driven Application Consolidation for Efficient Cloud Computing", MASCOTS, 2010. Google ScholarDigital Library
- G. Jung, M. Hiltunen, K. Joshi, R. Schlichting and C. Pu, "Mistral: Dynamically Managing Power, Performance, and Adaptation Cost in Cloud Infrastructures", ICDCS, 2010. Google ScholarDigital Library
- J. Wildstrom, P. Stone and E. Witchel, "CARVE: A Cognitive Agent for Resource Value Estimation", ICAC, 2008. Google ScholarDigital Library
- J. Rao, X. Bu, C. Xu, L. Wang and G. Yin, "VCONF: A Reinforcement Learning Approach to Virtual Machines Auto-configuration", ICAC, 2009. Google ScholarDigital Library
- S. Kundu, R. Rangaswami, K. Dutta and M. Zhao, "Application Performance Modeling in a Virtualized Environment," HPCA, 2010.Google Scholar
- X. Liu, X. Zhu, S. Singhal and M. Arlitt, "Adaptive Entitlement Control of Resource Containers on Shared Servers", IM, 2005.Google Scholar
- Z. Wang, X. Zhu and S. Singhal, "Utilization and SLO-Based Control for Dynamic Sizing of Resource Partitions", DSOM, 2005. Google ScholarDigital Library
- P. Padala, K. Hou, K. Shin, X. Zhu, M. Uysal, Z. Wang, S. Singhal and A. Merchant, "Automated Control of Multiple Virtualized Resources", SIGOPS/EuroSys, 2009. Google ScholarDigital Library
- X. Liu, X. Zhu, P. Padala, Z. Wang and S. Singhal, "Optimal Multivariate Control for Differentiated Services on a Shared Hosting Platform", CDC, 2007.Google Scholar
- R.Nathuji and A. Kansal, "Q-Clouds: Managing Performance Interference Effects for QoS-Aware Clouds", Eurosys, 2010. Google ScholarDigital Library
- P. Lama and X. Zhou, "PERFUME: Power and Performance Guarantee with Fuzzy MIMO Control in Virtualized Servers", IWQoS, 2011. Google ScholarDigital Library
- L.Wang, J. Xu, M. Zhao and J. A.B. Fortes, "Adaptive Virtual Resource Management with Fuzzy Model Predictive Control" FeBID, 2011. Google ScholarDigital Library
- R. Singh, U. Sharma, E. Cecchet, and P.J. Shenoy, "Autonomic Mix-Aware Provisioning for Non-Stationary Data Center Workloads", ICAC. 2010 Google ScholarDigital Library
- A. Soror, U. Minhas, A. Aboulnaga, K. Salem, P. Kokosielis and S. Kamath, "Automatic Virtual Machine Configuration for Database Workloads", SIGMOD, 2008 Google ScholarDigital Library
- G. Weikum, A. Moenkeberg, C. Hasse and P. Zabback, "Self-tuning Database Technology and Information Services: From Wishful Thinking to Viable Engineering", VLDB, 2002. Google ScholarDigital Library
- S. Chaudhuri and G. Weikum, "Foundations of Automated Database Tuning", ICDE, 2006. Google ScholarDigital Library
- B. Schroeder, M. Harchol-Balter, A. Iyengar and E. Nahum, "Achieving Class-based QoS for Transactional Workloads", ICDE, 2006. Google ScholarDigital Library
- P. Martin, S. Elnaffar and T. Wasserman, "Workload Models for Autonomic Database Management Systems", ICAS, 2006. Google ScholarDigital Library
- T. Wasserman, P. Martin and D. Skillicorn, "Developing a Characterization of Business Intelligence Workloads for Sizing New Database Systems", DOLAP, 2004. Google ScholarDigital Library
Index Terms
- Application-aware cross-layer virtual machine resource management
Recommendations
Replication and Migration as Resource Management Mechanisms for Virtualized Environments
ICAS '10: Proceedings of the 2010 Sixth International Conference on Autonomic and Autonomous SystemsVirtualization has become an essential technology in the data center. Virtualization improves resource utilization through server consolidation, but it also makes resource management more complex. Golondrina, an autonomic resource management system, was ...
Adaptive virtual resource management with fuzzy model predictive control
ICAC '11: Proceedings of the 8th ACM international conference on Autonomic computingResource management in virtualized systems remains a key challenge where the applications have dynamically changing workloads and the virtual machines (VMs) compete for the shared resources in a convolved manner. To address this challenge, this paper ...
Autonomic resource management in virtualized data centers using fuzzy logic-based approaches
Data centers, as resource providers, are expected to deliver on performance guarantees while optimizing resource utilization to reduce cost. Virtualization techniques provide the opportunity of consolidating multiple separately managed containers of ...
Comments