Skip to main content

Statistical Inference of Software Performance Models for Parametric Performance Completions

  • Conference paper
Research into Practice – Reality and Gaps (QoSA 2010)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 6093))

Included in the following conference series:

Abstract

Software performance engineering (SPE) enables software architects to ensure high performance standards for their applications. However, applying SPE in practice is still challenging. Most enterprise applications include a large software basis, such as middleware and legacy systems. In many cases, the software basis is the determining factor of the system’s overall timing behavior, throughput, and resource utilization. To capture these influences on the overall system’s performance, established performance prediction methods (model-based and analytical) rely on models that describe the performance-relevant aspects of the system under study. Creating such models requires detailed knowledge on the system’s structure and behavior that, in most cases, is not available. In this paper, we abstract from the internal structure of the system under study. We focus on message-oriented middleware (MOM) and analyze the dependency between the MOM’s usage and its performance. We use statistical inference to conclude these dependencies from observations. For ActiveMQ 5.3, the resulting functions predict the performance with a relative mean square error 0.1.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Arlitt, M., Krishnamurthy, D., Rolia, J.: Characterizing the scalability of a large web-based shopping system. ACM Trans. on Internet Technology (TOIT), 44–69 (2001)

    Google Scholar 

  2. Avritzer, A., Kondek, J., Liu, D., Weyuker, E.J.: Software performance testing based on workload characterization. In: WOSP, pp. 17–24 (2002)

    Google Scholar 

  3. Balsamo, S., Di Marco, A., Inverardi, P., Simeoni, M.: Model-Based Performance Prediction in Software Development: A Survey. IEEE Trans. on Software Engineering, 295–310 (2004)

    Google Scholar 

  4. Barber, S.: Creating effective load models for performance testing with incomplete empirical data. In: WSE, pp. 51–59 (2004)

    Google Scholar 

  5. Beasley, D., Bull, D.R., Martin, R.R.: An overview of genetic algorithms: Part 1, fundamentals (1993)

    Google Scholar 

  6. Becker, S., Koziolek, H., Reussner, R.: The Palladio component model for model-driven performance prediction. Journal of Systems and Software, 3–22 (2009)

    Google Scholar 

  7. Brosig, F., Kounev, S., Krogmann, K.: Automated Extraction of Palladio Component Models from Running Enterprise Java Applications. In: ROSSA (2009)

    Google Scholar 

  8. Courtois, M., Woodside, M.: Using regression splines for software performance analysis and software characterization. In: WOSP, pp. 105–114 (2000)

    Google Scholar 

  9. Denaro, G., Polini, A., Emmerich, W.: Early performance testing of distributed software applications. SIGSOFT Software Engineering Notes, 94–103 (2004)

    Google Scholar 

  10. Friedman, J.H.: Multivariate adaptive regression splines. Annals of Statistics, 1–141 (1991)

    Google Scholar 

  11. Happe, J., Becker, S., Rathfelder, C., Friedrich, H., Reussner, R.H.: Parametric Performance Completions for Model-Driven Performance Prediction. Performance Evaluation (2009)

    Google Scholar 

  12. Happe, J., Li, H., Theilmann, W.: Black-box Performance Models: Prediction based on Observation. In: QUASOSS, pp. 19–24 (2009)

    Google Scholar 

  13. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data mining, Inference and Prediction. Springer Series in Statistics (2009)

    Google Scholar 

  14. Holland, J.H.: Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor (1975)

    Google Scholar 

  15. Kapova, L., Goldschmidt, T.: Automated feature model-based generation of refinement transformations. In: SEAA-EUROMICRO, pp. 141–148 (2009)

    Google Scholar 

  16. Kapova, L., Zimmerova, B., Martens, A., Happe, J., Reussner, R.H.: State dependence in performance evaluation of component-based software systems. In: WOSP/SIPEW (2010)

    Google Scholar 

  17. Kounev, S., Sachs, K.: Benchmarking and performance modeling of event-based systems. IT - Information Technology, 262–269 (2009)

    Google Scholar 

  18. Koziolek, H.: Performance evaluation of component-based software systems: A survey. Performance Evaluation (2009)

    Google Scholar 

  19. Kraft, S., Pacheco-Sanchez, S., Casale, G., Dawson, S.: Estimating service resource consumption from response time measurements. In: Valuetools (2006)

    Google Scholar 

  20. Krogmann, K., Kuperberg, M., Reussner, R.: Using Genetic Search for Reverse Engineering of Parametric Behaviour Models for Performance Prediction. IEEE Trans. on Software Engineering (2010)

    Google Scholar 

  21. Kumar, D., Zhang, L., Tantawi, A.: Enhanced inferencing: Estimation of a workload dependent performance model. In: Valuetools (2009)

    Google Scholar 

  22. Liu, Y., Fekete, A., Gorton, I.: Design-level performance prediction of component-based applications. IEEE Trans. Software Eng. 31(11), 928–941 (2005)

    Article  Google Scholar 

  23. Pacifici, G., Segmuller, W., Spreitzer, M., Tantawi, A.: Dynamic estimation of cpu demand of web traffic. In: Valuetools, p. 26 (2006)

    Google Scholar 

  24. Sachs, K., Appel, S., Kounev, S., Buchmann, A.: Benchmarking publish/subscribe-based messaging systems. In: DASFAA Workshops: Benchmar’X10 (2010)

    Google Scholar 

  25. Sachs, K., Kounev, S., Bacon, J., Buchmann, A.: Performance evaluation of message-oriented middleware using the SPECjms 2007 benchmark. Performance Evaluation (2009)

    Google Scholar 

  26. Sharma, A., Bhagwan, R., Choudhury, M., Golubchik, L., Govindan, R., Voelker, G.M.: Automatic request categorization in internet services (2008)

    Google Scholar 

  27. Smith, C.U.: Performance Engineering of Software Systems. Addison-Wesley, USA (1990)

    Google Scholar 

  28. Sun Microsystems, Inc. Java Message Service (JMS) Specification - Version 1.1 (2002)

    Google Scholar 

  29. Woodside, M., Franks, G., Petriu, D.C.: The Future of Software Performance Engineering. In: ICSE, pp. 171–187 (2007)

    Google Scholar 

  30. Zheng, T., Woodside, C.M., Litoiu, M.: Performance model estimation and tracking using optimal filters. IEEE Trans. Software Engineering, 391–406 (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Happe, J., Westermann, D., Sachs, K., Kapová, L. (2010). Statistical Inference of Software Performance Models for Parametric Performance Completions. In: Heineman, G.T., Kofron, J., Plasil, F. (eds) Research into Practice – Reality and Gaps. QoSA 2010. Lecture Notes in Computer Science, vol 6093. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13821-8_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-13821-8_4

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-13820-1

  • Online ISBN: 978-3-642-13821-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics