skip to main content
10.1145/2744210.2744213acmconferencesArticle/Chapter ViewAbstractPublication PageseurosysConference Proceedingsconference-collections
research-article

State-based robustness testing of IaaS cloud platforms

Published: 21 April 2015 Publication History

Abstract

An uncountable number of services are deployed over cloud platforms and provided to millions of consumers. As this paradigm spreads over, the quality of provided services becomes a primary concern. Testing helps in making software reliable, but it has been overlooked for cloud. In this paper, we present a method for the robustness testing of software platforms for IaaS cloud. The method stresses the importance of considering the state for these systems, which are characterized by phase-based interactions of many software components with multiple concurrent users. Applied to a real cloud platform, the method exposes failures hard to uncover with common robustness testing approaches.

References

[1]
A. Avizienis et al. Basic concepts and taxonomy of dependable and secure computing. Dependable and Secure Computing, IEEE Trans. on, 1(1): 11--33, 2004.
[2]
A. Bovenzi et al. Workload characterization for software aging analysis. In IEEE 22nd International Symposium on Software Reliability Engineering (ISSRE), December 2011.
[3]
A. Kalakech et al. Benchmarking the dependability of Windows NT4, 2000 and XP. In Int. Conf. on Dependable Systems and Networks, June 2004.
[4]
A. Pecchia et al. Identifying compromised users in shared computing infrastructures: A data-driven bayesian network approach. In Symp. on Reliable Distributed Systems, 2011.
[5]
A. Vallone. Testing Google's New API Infrastructure, Feb 2015. http://googletesting.blogspot.it/2012/08/testing-googles-new-api-infrastructure.html.
[6]
A. M. Ambrosio et al. Designing fault injection experiments using state-based model to test a space software. In Latin-American Conference on Dependable Computing, Berlin, Heidelberg, 2007.
[7]
B. Lei et al. State based robustness testing for components. Electronic Notes in Theoretical Computer Science, 2010.
[8]
D. Cotroneo et al. A measurement-based ageing analysis of the JVM. Software Testing Verification and Reliability, 2011.
[9]
F. Frattini et al. Analysis of bugs in Apache Virtual Computing Lab. In Dependable Systems and Networks (DSN), 2013 43rd Annual IEEE/IFIP International Conference on, 2013.
[10]
G. N. Iyer et al. PCTF: An Integrated, Extensible Cloud Test Framework for Testing Cloud Platforms and Applications. In International Conference on Quality Software, July 2013.
[11]
IEEE - Standards Association. IEEE STANDARD 610.12-1990 - IEEE Standard Glossary of Software Engineering Terminology, July 2014.
[12]
P. Koopman and J. DeVale. The Exception Handling Effectiveness of POSIX Operating Systems. IEEE Trans. on Software Engineering, 26(9): 837--848, Sept. 2000.
[13]
L. Riungu-Kalliosaari et al. Testing in the cloud: Exploring the practice. IEEE Software, 29(2): 46--51, 2012.
[14]
OakLeaf Systems. Azure Storage Services Test Harness, Feb 2015. http://oakleafblog.blogspot.it/2008/11/azure-storage-services-test-harness.html.

Cited By

View all
  • (2023)Rise of the Planet of Serverless Computing: A Systematic ReviewACM Transactions on Software Engineering and Methodology10.1145/357964332:5(1-61)Online publication date: 21-Jul-2023
  • (2023)Intrusion Injection for Virtualized Systems: Concepts and Approach2023 53rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)10.1109/DSN58367.2023.00047(417-430)Online publication date: Jun-2023
  • (2021)A Systematic Review on Software Robustness AssessmentACM Computing Surveys10.1145/344897754:4(1-65)Online publication date: 3-May-2021
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CloudDP '15: Proceedings of the 5th International Workshop on Cloud Data and Platforms
April 2015
35 pages
ISBN:9781450334785
DOI:10.1145/2744210
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 April 2015

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Research-article

Conference

EuroSys '15
Sponsor:
EuroSys '15: Tenth EuroSys Conference 2015
April 21 - 24, 2015
Bordeaux, France

Acceptance Rates

Overall Acceptance Rate 6 of 16 submissions, 38%

Upcoming Conference

EuroSys '25
Twentieth European Conference on Computer Systems
March 30 - April 3, 2025
Rotterdam , Netherlands

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)3
  • Downloads (Last 6 weeks)0
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Rise of the Planet of Serverless Computing: A Systematic ReviewACM Transactions on Software Engineering and Methodology10.1145/357964332:5(1-61)Online publication date: 21-Jul-2023
  • (2023)Intrusion Injection for Virtualized Systems: Concepts and Approach2023 53rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)10.1109/DSN58367.2023.00047(417-430)Online publication date: Jun-2023
  • (2021)A Systematic Review on Software Robustness AssessmentACM Computing Surveys10.1145/344897754:4(1-65)Online publication date: 3-May-2021
  • (2020)A verified and optimized Stream X‐Machine testing method, with application to cloud service certificationSoftware Testing, Verification and Reliability10.1002/stvr.172930:3Online publication date: 18-Feb-2020
  • (2019)A Systematic Review on Cloud TestingACM Computing Surveys10.1145/333144752:5(1-42)Online publication date: 13-Sep-2019
  • (2019)Combining State and Interface - Based Robustness Testing for OpenStack Components2019 9th Latin-American Symposium on Dependable Computing (LADC)10.1109/LADC48089.2019.8995720(1-10)Online publication date: Nov-2019
  • (2019)An Overview of Cloud Computing Testing ResearchCloud Computing – CLOUD 201910.1007/978-3-030-23502-4_21(303-313)Online publication date: 25-Jun-2019
  • (2018)Using a Search and Model Based Framework to Improve Robustness Tests in Cloud PlatformsProceedings of the III Brazilian Symposium on Systematic and Automated Software Testing10.1145/3266003.3266011(67-76)Online publication date: 17-Sep-2018

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media