skip to main content
10.1145/2602458.2602475acmconferencesArticle/Chapter ViewAbstractPublication PagescbseConference Proceedingsconference-collections
research-article

Enriching software architecture models with statistical models for performance prediction in modern storage environments

Published: 27 June 2014 Publication History

Abstract

Model-based performance prediction approaches on the software architecture-level provide a powerful tool for capacity planning due to their high abstraction level. To process the increasing amount of data produced by today's applications, modern storage systems are becoming increasingly complex having multiple tiers and intricate optimization strategies. Current software architecture-level modeling approaches, however, struggle to account for this development and are not well-suited in complex storage environments due to overly simplistic storage assumptions, which consequently leads to inaccurate performance predictions. To address this problem, in this paper we present a novel approach to combine software architecture-level performance models with statistical models that capture the complex behavior of modern storage systems. More specifically, we first propose a general methodology for enriching software architecture modeling approaches with statistical I/O performance models. Then, we present how we realize the modeling concepts as well as model solving to obtain performance results. Finally, we evaluate our approach extensively in the context of three case studies with two state-of-the-art environments based on Sun Fire and IBM System z server hardware. Using our approach, we are able to successfully predict the application performance within 20 % prediction error in almost all cases.

References

[1]
I. Ahmad, J. Anderson, A. Holler, R. Kambo, and V. Makhija. An Analysis of Disk Performance in VMware ESX Server Virtual Machines. In WWC-6, 2003.
[2]
S. Balsamo, A. Di Marco, P. Inverardi, and M. Simeoni. Model-based performance prediction in software development: A survey. IEEE TSE, 2004.
[3]
S. Becker. Coupled model transformations for QoS enabled component-based software design. PhD thesis, Universit\"at Oldenburg, 2008.
[4]
S. Becker, H. Koziolek, and R. Reussner. The palladio component model for model-driven performance prediction. J. of Systems and Software, 82(1), 2009.
[5]
L. Breiman, J. Friedman, C. J. Stone, and R. Olshen. Classification and Regression Trees. The Wadsworth and Brooks-Cole statistics-probability series. Chapman & Hall, 1984.
[6]
R. C. Chiang and H. H. Huang. TRACON: Interference-aware Scheduling for Data-intensive Applications in Virtualized Environments. In SC '11.
[7]
B. Dufrasne, W. Bauer, B. Careaga, J. Myyrrylainen, A. Rainero, and P. Usong. IBM System Storage DS8700 Architecture and Implementation. http://www.redbooks.ibm.com/abstracts/sg248786.html, 2010.
[8]
J. H. Friedman. Multivariate Adaptive Regression Splines. Annals of Statistics, 19(1):1--141, 1991.
[9]
J. Gantz and D. Reinsel (IDC). THE DIGITAL UNIVERSE IN 2020: Big Data, Bigger Digital Shadows, and Biggest Growth in the Far East. http://idcdocserv.com/1414, 2012. Last accessed: Jan 2014.
[10]
P. Harrison and S. Zertal. Queueing models of RAID systems with maxima of waiting times. Performance Evaluation, 64:664--689, 2007.
[11]
M. Hauck, M. Kuperberg, K. Krogmann, and R. Reussner. Modelling Layered Component Execution Environments for Performance Prediction. In CBSE '09.
[12]
N. Huber, S. Becker, C. Rathfelder, J. Schweflinghaus, and R. Reussner. Performance Modeling in Industry: A Case Study on Storage Virtualization. In ICSE '10, Software Engineering in Practice Track.
[13]
L. Kapova and T. Goldschmidt. Automated feature model-based generation of refinement transformations. In SEAA '09.
[14]
Y. Koh, R. Knauerhase, P. Brett, M. Bowman, Z. Wen, and C. Pu. An Analysis of Performance Interference Effects in Virtual Environments. In ISPASS '07.
[15]
H. Koziolek. Performance evaluation of component-based software systems: A survey. Perform. Eval., 2010.
[16]
S. Kraft, G. Casale, D. Krishnamurthy, D. Greer, and P. Kilpatrick. Performance Models of Storage Contention in Cloud Environments. SoSyM, 2012.
[17]
K. Krogmann. Reconstruction of Software Component Architectures and Behaviour Models using Static and Dynamic Analysis. PhD thesis, Karlsruhe Institute of Technology (KIT), 2010.
[18]
S. Kundu, R. Rangaswami, A. Gulati, M. Zhao, and K. Dutta. Modeling Virtualized Applications using Machine Learning Techniques. In VEE '12.
[19]
M. Kuperberg, K. Krogmann, and R. Reussner. Performance Prediction for Black-Box Components using Reengineered Parametric Behaviour Models. In CBSE '08.
[20]
A. S. Lebrecht, N. J. Dingle, and W. J. Knottenbelt. Analytical and Simulation Modelling of Zoned RAID Systems. The Computer Journal, 54:691--707, 2011.
[21]
E. K. Lee and R. H. Katz. An analytic performance model of disk arrays. SIGMETRICS Perform. Eval. Rev., 21(1), 1993.
[22]
X. Ling, S. Ibrahim, H. Jin, S. Wu, and T. Songqiao. Exploiting Spatial Locality to Improve Disk Efficiency in Virtualized Environments. In MASCOTS '13.
[23]
Q. Noorshams, D. Bruhn, A. Busch, S. Kounev, and R. Reussner. The Storage Performance Analyzer (SPA), 2014. http://sdqweb.ipd.kit.edu/wiki/SPA.
[24]
Q. Noorshams, D. Bruhn, S. Kounev, and R. Reussner. Predictive Performance Modeling of Virtualized Storage Systems using Optimized Statistical Regression Techniques. In ICPE '13.
[25]
Q. Noorshams, A. Busch, A. Rentschler, D. Bruhn, S. Kounev, P. T\ruma, and R. Reussner. Automated Modeling of I/O Performance and Interference Effects in Virtualized Storage Systems. In DCPerf '14.
[26]
Q. Noorshams, S. Kounev, and R. Reussner. Experimental Evaluation of the Performance-Influencing Factors of Virtualized Storage Systems. In EPEW '12.
[27]
Q. Noorshams, K. Rostami, S. Kounev, P. T\ruma, and R. Reussner. I/O Performance Modeling of Virtualized Storage Systems. In MASCOTS '13.
[28]
S. Oliveira, K. Furlinger, and D. Kranzlmuller. Trends in Computation, Communication and Storage and the Consequences for Data-intensive Science. In HPCC-ICESS'12.
[29]
J. G. Shanthikumar and R. G. Sargent. A unifying view of hybrid simulation/analytic models and modeling. Operations Research, 31(6):pp. 1030--1052, 1983.
[30]
E. Varki and S. X. Wang. A performance model of disk array storage systems. In Int. CMG Conference, 2000.
[31]
M. Wang, K. Au, A. Ailamaki, A. Brockwell, C. Faloutsos, and G. R. Ganger. Storage Device Performance Prediction with CART Models. In MASCOTS '04.
[32]
A. Wert, J. Happe, and D. Westermann. Integrating Software Performance Curves with the Palladio Component Model. In ICPE '12.
[33]
M. Woodside, D. Petriu, and K. Siddiqui. Performance-related completions for software specifications. In ICSE '02.
[34]
M. Woodside, V. Vetland, M. Courtois, and S. Bayarov. Resource Function Capture for Performance Aspects of Software Components and Sub-systems. In R. Dumke, C. Rautenstrauch, A. Scholz, and A. Schmietendorf, editors, Performance Engineering, State of the Art and Current Trends. 2001.

Cited By

View all
  • (2023)A Large-Scale Empirical Study of Real-Life Performance Issues in Open Source ProjectsIEEE Transactions on Software Engineering10.1109/TSE.2022.316762849:2(924-946)Online publication date: 1-Feb-2023
  • (2019)Integrating Statistical Response Time Models in Architectural Performance Models2019 IEEE International Conference on Software Architecture (ICSA)10.1109/ICSA.2019.00016(71-80)Online publication date: Mar-2019
  • (2017)Cloning IO Intensive Workloads Using Synthetic BenchmarkProceedings of the 8th ACM/SPEC on International Conference on Performance Engineering10.1145/3030207.3030238(317-320)Online publication date: 17-Apr-2017
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CBSE '14: Proceedings of the 17th international ACM Sigsoft symposium on Component-based software engineering
June 2014
200 pages
ISBN:9781450325776
DOI:10.1145/2602458
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 June 2014

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. i/o
  2. performance
  3. prediction
  4. software architecture
  5. statistical model
  6. storage

Qualifiers

  • Research-article

Funding Sources

Conference

CompArch'14
Sponsor:

Acceptance Rates

CBSE '14 Paper Acceptance Rate 21 of 62 submissions, 34%;
Overall Acceptance Rate 55 of 147 submissions, 37%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 07 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2023)A Large-Scale Empirical Study of Real-Life Performance Issues in Open Source ProjectsIEEE Transactions on Software Engineering10.1109/TSE.2022.316762849:2(924-946)Online publication date: 1-Feb-2023
  • (2019)Integrating Statistical Response Time Models in Architectural Performance Models2019 IEEE International Conference on Software Architecture (ICSA)10.1109/ICSA.2019.00016(71-80)Online publication date: Mar-2019
  • (2017)Cloning IO Intensive Workloads Using Synthetic BenchmarkProceedings of the 8th ACM/SPEC on International Conference on Performance Engineering10.1145/3030207.3030238(317-320)Online publication date: 17-Apr-2017
  • (2015)The Storage Performance AnalyzerProceedings of the 6th ACM/SPEC International Conference on Performance Engineering10.1145/2668930.2693845(107-108)Online publication date: 28-Jan-2015

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media