skip to main content
10.1145/2391229.2391249acmconferencesArticle/Chapter ViewAbstractPublication PagesmodConference Proceedingsconference-collections
research-article

More for your money: exploiting performance heterogeneity in public clouds

Published:14 October 2012Publication History

ABSTRACT

Infrastructure-as-a-system compute clouds such as Amazon's EC2 allow users to pay a flat hourly rate to run their virtual machine (VM) on a server providing some combination of CPU access, storage, and network. But not all VM instances are created equal: distinct underlying hardware differences, contention, and other phenomena can result in vastly differing performance across supposedly equivalent instances. The result is striking variability in the resources received for the same price.

We initiate the study of customer-controlled placement gaming: strategies by which customers exploit performance heterogeneity to lower their costs. We start with a measurement study of Amazon EC2. It confirms the (oft-reported) performance differences between supposedly identical instances, and leads us to identify fruitful targets for placement gaming, such as CPU, network, and storage performance. We then explore simple heterogeneity-aware placement strategies that seek out better-performing instances. Our strategies require no assistance from the cloud provider and are therefore immediately deployable. We develop a formal model for placement strategies and evaluate potential strategies via simulation. Finally, we verify the efficacy of our strategies by implementing them on EC2; our experiments show performance improvements of 5% for a real-world CPU-bound job and 34% for a bandwidth-intensive job.

References

  1. Amazon Ltd. Amazon elastic compute cloud (EC2). http://aws.amazon.com/ec2/.Google ScholarGoogle Scholar
  2. Amazon Web Services. Amazon EC2 instance types. http://aws.amazon.com/ec2/instance-types/.Google ScholarGoogle Scholar
  3. Amazon Web Services. Amazon Elastic Block Store (EBS). http://aws.amazon.com/ebs/.Google ScholarGoogle Scholar
  4. Apache Software Foundation. ab - apache http server benchmarking tool. http://httpd.apache.org/docs/2.0/programs/ab.html.Google ScholarGoogle Scholar
  5. Apache Software Foundation. Apache HTTP server project. http://httpd.apache.org/.Google ScholarGoogle Scholar
  6. Manjari Asawa and Demosthenis Teneketzis. Multi-armed bandits with switching penalties. IEEE Transactions on Automatic Control, 41(3), March 1996.Google ScholarGoogle ScholarCross RefCross Ref
  7. Sean Kenneth Barker and Prashant Shenoy. Empirical evaluation of latency-sensitive application performance in the cloud. In MMsys, pages 35--46, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Russker Coker. Bonnie++ benchmark version 1.03e. http://www.coker.com.au/bonnie++/, 2008.Google ScholarGoogle Scholar
  9. Jiang Dejun, Guillaume Pierre, and Chi-Hung Chi. Ec2 performance analysis for resource provisioning of service-oriented applications. In Proceedings of the 2009 international conference on Service-oriented computing, pages 197--207, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Jiang Dejun, Guillaume Pierre, and Chi-Hung Chi. Resource provisioning of web applications in heterogeneous clouds. In Proceedings of the 2nd USENIX conference on Web application development, pages 5--5, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Diwaker Gupta, Ludmila Cherkasova, Rob Gardner, and Amin Vahdat. Enforcing performance isolation across virtual machines in xen. In Middleware, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. J. L. Henning. Spec cpu2006 benchmark descriptions. In SIGARCH Computer Architecture News, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Alexandru Iosup, Nezih Yigitbasi, and Dick H. J. Epema. On the performance variability of production cloud services. In 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), pages 104--113, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Aamer Jaleel, Hashem H. Najaf-abadi, Samantika Subramaniam, Simon C. Steely, and Joel Emer. Cruise: cache replacement and utility-aware scheduling. In ASPLOS, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Vivek Kale, Jayanta Mukherjee, and Indranil Gupta. Hadoopjitter: The ghost in the machine and how to tame it. http://hdl.handle.net/2142/17084, 2010.Google ScholarGoogle Scholar
  16. Yaakoub El Khamra, Hyunjoo Kim, Shantenu Jha, and Manish Parashar. Exploring the performance fluctuations of hpc workloads on clouds. In CloudCom, pages 383--387, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Dan Klein, Joseph Smarr, Huy Nguyen, and Christopher D. Manning. Named entity recognition with character-level models. In Proceedings the Seventh Conference on Natural Language Learning, pages 180--183, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. A. Li, X. Yang, S. Kandula, and M. Zhang. Cloudcmp: Comparing public cloud providers. In IMC, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Seung-Hwan Lim, Jae-Seok Huh, Youngjae Kim, and Chita R. Das. Migration, assignment, and scheduling of jobs in virtualized environment. In HotCloud, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Dave Mangot. EC2 variability: The numbers revealed. http://tech.mangot.com/roller/dave/entry/ec2_variability_the_numbers_re%vealed, May 2009.Google ScholarGoogle Scholar
  21. Microsoft Corp. Windows azure: Pricing details. http://www.windowsazure.com/en-us/pricing/details/.Google ScholarGoogle Scholar
  22. Zhonghong Ou, Hao Zhuang, Jukka K. Nurminen, Antti Ylä-Jääski, and Pan Hui. Exploiting hardware heterogeneity within the same instance type of amazon EC2. In 4th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud), 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. MIT Press, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Rackspace Inc. How we price cloud servers. http://www.rackspace.com/cloud/cloud_hosting_products/servers/pricing/.Google ScholarGoogle Scholar
  25. M. S. Rehman and M. F. Sakr. Initial findings for provisioning variation in cloud computing. In IEEE Second International Conference on Cloud Computing Technology and Science (CloudCom), pages 473--479, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. T. Ristenpart, E. Tromer, H. Shacham, and S. Savage. Hey, you, get off my cloud: exploring information leakage in third party compute clouds. In CCS, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Juan Carlos Saez, Manuel Prieto, Alexandra Fedorova, and Sergey Blagodurov. A comprehensive scheduler for asymmetric multicore systems. In Proc. EuroSys, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Jorg Schad, Jens Dittrich, and Jorge-Arnulfo Quiané-Ruiz. Runtime measurements in the cloud: Observing, analyzing, and reducing variance. In VLDB, September 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Malte Schwarzkopf, Derek G. Murray, and Steven Hand. The seven deadly sins of cloud computing research. In HotCloud, June 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Burr Settles. Biomedical named entity recognition using conditional random fields and rich feature sets. In NLPBA, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Alan Shieh, Srikanth Kandula, Albert Greenberg, and Changhoon Kim. Seawall: Performance isolation for cloud datacenter networks. In HotCloud, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. The Stanford Natural Language Processing Group. Stanford named entity recognizer (NER) version 1.2.4. http://nlp.stanford.edu/software/CRF-NER.shtml, 2012.Google ScholarGoogle Scholar
  33. A. Tirumala, F. Qin, J Dugan, J. Ferguson, and K. Gibbs. Iperf: The TCP/UDP bandwidth measurement tool, version 2.0.5. http://sourceforge.net/projects/iperf/, 2010.Google ScholarGoogle Scholar
  34. Guohui Wang and T. S. Eugene Ng. The impact of virtualization on network performance of amazon EC2 data center. In IEEE INFOCOM, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. P. Whittle. Sequential scheduling and the multi-armed bandit, chapter 14. In Optimization over Time--Dynamic Programming and Stechastic Control. vol. 1, pages 210--219. John Wiley and Sons Ltd., 1982.Google ScholarGoogle Scholar
  36. Wikipedia. Markov decision process. http://en.wikipedia.org/wiki/Markov_decision_process.Google ScholarGoogle Scholar
  37. J. Xie, S. Yin, X.-J. Ruan, Z.-Y. Ding, Y. Tian, J. Majors, and X. Qin. Improving MapReduce performance via data placement in heterogeneous Hadoop Clusters. In Proc. 19th Int'l Heterogeneity in Computing Workshop, April 2010.Google ScholarGoogle Scholar
  38. Jun-Ming Xu, Kwang-Sung Jun, Xiaojin Zhu, and Amy Bellmore. Learning from bullying traces in social media. In NAACL HLT, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Sungkap Yeo and Hsien-Hsin S. Lee. Using mathematical modeling in provisioning a heterogeneous cloud computing environment. IEEE Computer, 44(8), August 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Matei Zaharia, Andy Konwinski, Anthony D. Joseph, Randy Katz, and Ion Stoica. Improving mapreduce performance in heterogeneous environments. In OSDI, pages 29--42, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Ce Zhang, Feng Niu, Christopher Ré, and Jude Shavlik. Big data versus the crowd: Looking for relationships in all the right places. In ACL, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. More for your money: exploiting performance heterogeneity in public clouds

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      SoCC '12: Proceedings of the Third ACM Symposium on Cloud Computing
      October 2012
      325 pages
      ISBN:9781450317610
      DOI:10.1145/2391229

      Copyright © 2012 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 14 October 2012

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      Overall Acceptance Rate169of722submissions,23%

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader