Skip to main content
SpringerLink
Log in

Adaptive Online Compression in Clouds—Making Informed Decisions in Virtual Machine Environments

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

Infrastructure as a Service clouds often use virtual machines to host different customers on the same physical hardware. This form of resource sharing can lead to unpredicatable performance degradations for the individual customer, especially with regard to data-intensive applications, which heavily depend on stable I/O characteristics. One traditional approach to cope with I/O fluctuations is adaptive online compression. In this paper we present a new scheme for adaptive online compression which has been explicitly designed to work in co-located virtual machine environments. In contrast to existing adaptive online compression schemes, the decision model of our approach does not rely on the system metrics CPU utilization and I/O bandwidth, which we demonstrate to be often displayed inaccurately inside XEN, KVM, and Amazon EC2-based virtual machines. Instead, it only considers the application data rate. Without requiring any calibration or training phase our adaptive compression scheme can improve the I/O throughput of virtual machines up to a factor of four as shown through extended experimental evaluations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Amazon Web Services LLC: Amazon Elastic Compute Cloud (Amazon EC2). http://aws.amazon.com/ec2/ (2010). Accessed 21 Feb 2010

  2. Arnold, R., Bell, T.: A corpus for the evaluation of lossless compression algorithms. In: Proc. of the 1997 Data Compression Conference, DCC ’97, pp. 201–210 (1997). doi:10.1109/DCC.1997.582019

  3. Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebauer, R., Pratt, I., Warfield, A.: Xen and the art of virtualization. In: Proc. of the 19th ACM symposium on Operating Systems Principles, SOSP ’03, pp. 164–177. ACM, New York (2003). doi:10.1145/945445.945462

    Chapter  Google Scholar 

  4. Cherkasova, L., Gardner, R.: Measuring CPU overhead for I/O processing in the Xen virtual machine monitor. In: Proc. of the USENIX Annual Technical Conference, ATEC ’05, pp. 24–24. USENIX Association, Berkeley (2005)

    Google Scholar 

  5. Duda, K.J., Cheriton, D.R.: Borrowed-virtual-time (BVT) scheduling: supporting latency-sensitive threads in a general-purpose scheduler. Oper. Syst. Rev. 33, 261–276 (1999). doi:10.1145/319344.319169

    Article  Google Scholar 

  6. GoGrid, LLC: Cloud hosting, cloud servers, hybrid hosting, cloud infrastructure from GoGrid. http://www.gogrid.com/ (2011). Accessed 31 Jul 2011

  7. Hovestadt, M., Kao, O., Kliem, A., Warneke, D.: Evaluating adaptive compression to mitigate the effects of shared I/O in clouds. In: Proc. of the 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and PhD Forum, IPDPSW ’11, pp. 1042–1051. IEEE Computer Society, Washington (2011). doi:10.1109/IPDPS.2011.256

    Chapter  Google Scholar 

  8. Jeannot, E., Knutsson, B., Björkman, M.: Adaptive online data compression. In: Proc. of the 11th IEEE International Symposium on High Performance Distributed Computing, HPDC ’11, pp. 379–388 (2002). doi:10.1109/HPDC.2002.1029938

  9. Kesavan, M., Gavrilovska, A., Schwan, K.: Differential virtual time (DVT): rethinking I/O service differentiation for virtual machines. In: Proc. of the 1st ACM Symposium on Cloud Computing, SoCC ’10, pp. 27–38. ACM, New York (2010). doi:10.1145/1807128.1807135

    Chapter  Google Scholar 

  10. Kivity, A., Kamay, Y., Laor, D., Lublin, U., Liguori, A.: kvm: the Linux virtual machine monitor. In: Proc. of the 2007 Ottawa Linux Symposium, OLS ’07, pp. 225–230 (2007)

  11. Krintz, C., Sucu, S.: Adaptive on-the-fly compression. IEEE Trans. Parallel Distrib. Syst. 17(1), 15–24 (2006). doi:10.1109/TPDS.2006.3

    Article  Google Scholar 

  12. Menon, A., Santos, J.R., Turner, Y., Janakiraman, G.J., Zwaenepoel, W.: Diagnosing performance overheads in the Xen virtual machine environment. In: Proc. of the 1st International Conference on Virtual Execution Environments, VEE ’05, pp. 13–23. ACM, New York (2005). doi:10.1145/1064979.1064984

    Chapter  Google Scholar 

  13. Motgi, N., Mukherjee, A.: Network conscious text compression system (NCTCSys). In: Proc. of the International Conference on Information Technology: Coding and Computing, ITCC ’01, p. 440. IEEE Computer Society, Washington (2001). doi:10.1109/ITCC.2001.918836

    Chapter  Google Scholar 

  14. Nurmi, D., Wolski, R., Grzegorczyk, C., Obertelli, G., Soman, S., Youseff, L., Zagorodnov, D.: The eucalyptus open-source cloud-computing system. In: Proc. of the 9th IEEE/ACM International Symposium on Cluster Computing and the Grid, CCGRID ’09, pp. 124–131. IEEE Computer Society, Washington (2009). doi:10.1109/CCGRID.2009.93

    Chapter  Google Scholar 

  15. Pavlow, I.: LZMA SDK (Software Development Kit). http://www.7-zip.org/sdk.html (2010). Accessed 22 Oct 2010

  16. QuickLZ: Fast compression library for C, C# and Java. http://www.quicklz.com/ (2010). Accessed 22 Oct 2010

  17. Rackspace US, Inc.: The Rackspace Cloud. http://www.rackspacecloud.com/ (2010)

  18. Santos, J.R., Turner, Y., Janakiraman, G., Pratt, I.: Bridging the gap between software and hardware techniques for I/O virtualization. In: Proc. of the USENIX Annual Technical Conference, ATEC ’08, pp. 29–42. USENIX Association, Berkeley, CA, USA (2008)

    Google Scholar 

  19. Schad, J., Dittrich, J., Quiané-Ruiz, J.A.: Runtime measurements in the cloud: observing, analyzing, and reducing variance. PVLDB 3(1), 460–471 (2010)

    Google Scholar 

  20. Seelam, S.R., Teller, P.J.: Virtual I/O scheduler: a scheduler of schedulers for performance virtualization. In: Proc. of the 3rd International Conference on Virtual Execution Environments, VEE ’07, pp. 105–115. ACM, New York (2007). doi:10.1145/1254810.1254826

    Chapter  Google Scholar 

  21. Tickoo, O., Iyer, R., Illikkal, R., Newell, D.: Modeling virtual machine performance: challenges and approaches. Perform. Eval. Rev. 37, 55–60 (2010). doi:10.1145/1710115.1710126

    Article  Google Scholar 

  22. Walker, E.: Benchmarking Amazon EC2 for high-performance scientific computing. USENIX; login: Magazine 33(5), 18–23 (2008)

    Google Scholar 

  23. Wang, G., Ng, T.E.: The impact of virtualization on network performance of Amazon EC2 data center. In: Proc. of the 29th IEEE International Conference on Computer Communications, INFOCOM ’10, pp. 1–9 (2010). doi:10.1109/INFCOM.2010.5461931

  24. Warneke, D., Kao, O.: Nephele: efficient parallel data processing in the cloud. In: Proc. of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers, MTAGS ’09, pp. 8:1–8:10. ACM, New York (2009). doi:10.1145/1646468.1646476

    Google Scholar 

  25. Warneke, D., Kao, O.: Exploiting dynamic resource allocation for efficient parallel data processing in the cloud. IEEE Trans. Parallel Distrib. Syst. 22(6), 985–997 (2011). doi:10.1109/TPDS.2011.65

    Article  Google Scholar 

  26. Wiseman, Y., Schwan, K., Widener, P.: Efficient end to end data exchange using configurable compression. In: Proc. of the 24th International Conference on Distributed Computing Systems, pp. 228–235 (2004). doi:10.1109/ICDCS.2004.1281587

  27. Wolski, R.: Experiences with predicting resource performance on-line in computational grid settings. Perform. Eval. Rev. 30(4), 41–49 (2003). doi:10.1145/773056.773064

    Article  Google Scholar 

  28. Wood, T., Cherkasova, L., Ozonat, K., Shenoy, P.: Profiling and modeling resource usage of virtualized applications. In: Proc. of the 9th ACM/IFIP/USENIX International Conference on Middleware, Middleware ’08, pp. 366–387. Springer New York, Inc., New York (2008)

    Google Scholar 

  29. Yamada, H., Kono, K.: Foxytechnique: Tricking operating system policies with a virtual machine monitor. In: Proc. of the 3rd International Conference on Virtual Execution Environments, VEE ’07, pp. 55–64. ACM, New York (2007). doi:10.1145/1254810.1254818

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Hannover University of Applied Sciences, Ricklinger Stadtweg 120, 30459, Hannover, Germany

    Matthias Hovestadt

  2. Technische Universität Berlin, Sekr. EN 59, Einsteinufer 17, 10587, Berlin, Germany

    Odej Kao & Andreas Kliem

  3. International Computer Science Institute (ICSI), 1947 Center Street, Suite 600, 94704, CA, Berkeley, USA

    Daniel Warneke

Authors
  1. Matthias Hovestadt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Odej Kao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andreas Kliem
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniel Warneke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel Warneke.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hovestadt, M., Kao, O., Kliem, A. et al. Adaptive Online Compression in Clouds—Making Informed Decisions in Virtual Machine Environments. J Grid Computing 11, 167–186 (2013). https://doi.org/10.1007/s10723-013-9249-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10723-013-9249-4

Keywords

Search

Navigation