Abstract
Many web information systems and applications are now run as cloud-hosted systems. The organization that owns the information system or application is thus a consumer of cloud services, and often relies on the cloud provider to monitor the virtual infrastructure and alert them of any disruption of the offered services. For example, Amazon Web Services’ cloud disruptions are announced by the cloud provider on a dedicated RSS feed so that the consumers can watch and act quickly. In this paper, we report on a long-running experiment for the monitoring and continuous benchmarking of a number of cloud resources on Amazon Cloud from a consumer’s perspective, aiming to check whether the service disruptions announced by the cloud provider are consistent with what we observe. We evaluate the performance of cloud resources over several months. We find that the performance of the cloud can vary significantly over time which leads to unpredictable application performance. Our analysis shows also that continuous benchmarking data can help detect failures before any announcement is made by the provider, as well as significant degradation of performance that is not always connected with Amazon service disruption announcements.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barker, S.K., Shenoy, P.: Empirical evaluation of latency-sensitive application performance in the cloud. In: Proceedings of the First Annual ACM SIGMM Conference on Multimedia Systems, MMSys 2010, pp. 35–46. ACM, New York (2010)
Chen, W., Toueg, S., Aguilera, M.K.: On the quality of service of failure detectors. IEEE Trans. Comput. 51(5), 561–580 (2002)
Chhetri, M.B., Chichin, S., Vo, Q.B., Kowalczyk, R.: Smart cloudbench – automated performance benchmarking of the cloud. In: 2013 IEEE Sixth International Conference on Cloud Computing (CLOUD), pp. 414–421 (June 2013)
Chiang, R.C., Howie Huang, H.: Tracon: Interference-aware scheduling for data-intensive applications in virtualized environments. In: Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2011, pp. 47:1–47:12. ACM, New York (2011)
Dejun, J., Pierre, G., Chi, C.-H.: Ec2 performance analysis for resource provisioning of service-oriented applications. In: Dan, A., Gittler, F., Toumani, F. (eds.) ICSOC/ServiceWave 2009. LNCS, vol. 6275, pp. 197–207. Springer, Heidelberg (2010)
Dfago, X., Urbn, P., Hayashibara, N., Katayama, T.: The accrual failure detector. In: RR IS-RR-2004-010. Japan Advanced Institute of Science and Technology, 66–78 (2004)
Giannini, D., Paggiaro, P.L., Moscato, G., Gherson, G., Bacci, E., Bancalari, L., Dente, F.L., Di Franco, A., Vagaggini, B., Giuntini, C.: Comparison between peak expiratory flow and forced expiratory volume in one second (fev1) during bronchoconstriction induced by different stimuli. Journal of Asthma 34(2), 105–111 (1997)
Gray, C., Cheriton, D.: Leases: An efficient fault-tolerant mechanism for distributed file cache consistency. SIGOPS Oper. Syst. Rev. 23(5), 202–210 (1989)
Iosup, A., Yigitbasi, N., Epema, D.: On the performance variability of production cloud services. In: Proceedings of the 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2011, pp. 104–113. IEEE Computer Society, Washington, DC (2011)
Kossmann, D., Kraska, T., Loesing, S.: An evaluation of alternative architectures for transaction processing in the cloud. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, SIGMOD 2010, pp. 579–590. ACM, New York (2010)
Lakshman, A., Malik, P.: Cassandra: A decentralized structured storage system. SIGOPS Oper. Syst. Rev. 44(2), 35–40 (2010)
Li, A., Yang, X., Kandula, S., Zhang, M.: Cloudcmp: Shopping for a cloud made easy. In: Proceedings of the 2Nd USENIX Conference on Hot Topics in Cloud Computing, HotCloud 2010, p. 5. USENIX Association, Berkeley (2010)
Li, A., Yang, X.: Srikanth Kandula, and Ming Zhang. Comparing public-cloud providers. IEEE Internet Computing 15(2), 50–53 (2011)
Makhija, V., Herndon, B., Smith, P., Roderick, L., Zamost, E., Anderson, J.: Vmmark: A scalable benchmark for virtualized systems. VMware Inc, CA, Tech. Rep. VMware-TR-2006-002, September (September 2006)
Dave Mangot. Measuring ec2 system performance (2009), http://bit.ly/48Wui (May 2009)
Novaković, D., Vasić, N., Novaković, S., Kostić, D., Bianchini, R.: Deepdive: Transparently identifying and managing performance interference in virtualized environments. In: Presented as part of the 2013 USENIX Annual Technical Conference (USENIX ATC 2013), San Jose, CA, pp. 219–230. USENIX (2013)
Pannu, H.S., Liu, J., Guan, Q., Fu, S.: Afd: Adaptive failure detection system for cloud computing infrastructures. In: IPCCC 2012, pp. 71–80 (2012)
Ren, X., Dong, J., Liu, H., Li, Y., Yang, X.: Low-overhead accrual failure detector (2012)
Satzger, B., Pietzowski, A., Trumler, W., Ungerer, T.: A new adaptive accrual failure detector for dependable distributed systems. In: Proceedings of the 2007 ACM Symposium on Applied Computing, SAC 2007, pp. 551–555. ACM, New York (2007)
Satzger, B., Pietzowski, A., Trumler, W., Ungerer, T.: Variations and evaluations of an adaptive accrual failure detector to enable self-healing properties in distributed systems. In: Lukowicz, P., Thiele, L., Tröster, G. (eds.) ARCS 2007. LNCS, vol. 4415, pp. 171–184. Springer, Heidelberg (2007)
Schad, J., Dittrich, J., Quiané-Ruiz, J.-A.: Runtime measurements in the cloud: Observing, analyzing, and reducing variance. Proc. VLDB Endow. 3(1-2), 460–471 (2010)
Kelvin, C., So, W., Sirer, E.G.: Latency and bandwidth-minimizing failure detectors. SIGOPS Oper. Syst. Rev. 41(3), 89–99 (2007)
Turner, A., Fox, A., Payne, J., Kim, H.S.: C-mart: Benchmarking the cloud. IEEE Trans. Parallel Distrib. Syst. 24(6), 1256–1266 (2013)
Walker, E.: Benchmarking amazon ec2 for high-performance scientific computing. LOGIN 33(5), 18–23 (2008)
Wang, G., Eugene Ng, T.S.: The impact of virtualization on network performance of amazon ec2 data center. In: Proceedings of the 29th Conference on Information Communications, INFOCOM 2010, Piscataway, NJ, USA, pp. 1163–1171. IEEE Press (2010)
Xiong, N., Vasilakos, A.V., Wu, J., Richard Yang, Y., Rindos, A., Zhou, Y., Song, W.-Z., Pan, Y.: A self-tuning failure detection scheme for cloud computing service. In: 2012 IEEE 26th International Parallel & Distributed Processing Symposium (IPDPS), pp. 668–679. IEEE (2012)
Xu, Y., Musgrave, Z., Noble, B., Bailey, M.: Bobtail: Avoiding long tails in the cloud. In: Proceedings of the 10th USENIX Conference on Networked Systems Design and Implementation, nsdi 2013, pp. 329–342. USENIX Association, Berkeley (2013)
Zhang, X., Tune, E., Hagmann, R., Jnagal, R., Gokhale, V., Wilkes, J.: Cpi2: Cpu performance isolation for shared compute clusters. In: SIGOPS European Conference on Computer Systems (EuroSys), Prague, Czech Republic, pp. 379–391 (2013)
Zhao, L., Liu, A., Keung, J.: Evaluating cloud platform architecture with the care framework. In: 2010 17th Asia Pacific Software Engineering Conference (APSEC), pp. 60–69 (November 2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Chaudry, R., Guabtni, A., Fekete, A., Bass, L., Liu, A. (2014). Consumer Monitoring of Infrastructure Performance in a Public Cloud. In: Benatallah, B., Bestavros, A., Manolopoulos, Y., Vakali, A., Zhang, Y. (eds) Web Information Systems Engineering – WISE 2014. WISE 2014. Lecture Notes in Computer Science, vol 8787. Springer, Cham. https://doi.org/10.1007/978-3-319-11746-1_31
Download citation
DOI: https://doi.org/10.1007/978-3-319-11746-1_31
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11745-4
Online ISBN: 978-3-319-11746-1
eBook Packages: Computer ScienceComputer Science (R0)