Skip to main content
Springer Nature Link
Log in

An Empirical Investigation of the Applicability of a Component-Based Performance Prediction Method

  • Conference paper
Computer Performance Engineering (EPEW 2008)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 5261))

Included in the following conference series:

  • 317 Accesses

Abstract

Component-based software performance engineering (CBSPE) methods shall enable software architects to assess the expected response times, throughputs, and resource utilization of their systems already during design. This avoids the violation of performance requirements. Existing approaches for CBSPE either lack tool support or rely on prototypical tools, who have only been applied by their authors. Therefore, industrial applicability of these methods is unknown. On this behalf, we have conducted a controlled experiment involving 19 computer science students, who analysed the performance of two component-based designs using our Palladio performance prediction approach, as an example for a CBSPE method. Our study is the first of its type in this area and shall help to mature CBSPE to industrial applicability. In this paper, we report on results concerning the prediction accuracy achieved by the students and list several lessons learned, which are also relevant for other methods than Palladio.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Balsamo, S., Di Marco, A., Inverardi, P., Simeoni, M.: Model-Based Performance Prediction in Software Development: A Survey. IEEE Trans. on Softw. Eng. 30(5), 295–310 (2004)

    Article  Google Scholar 

  2. Balsamo, S., Marzolla, M., Di Marco, A., Inverardi, P.: Experimenting different software architectures performance techniques: A case study. In: Proc. of WOSP, pp. 115–119. ACM Press, New York (2004)

    Chapter  Google Scholar 

  3. Basili, V.R., Caldiera, G., Rombach, H.D.: The Goal Question Metric Approach. In: Marciniak, J.J. (ed.) Encyclopedia of Software Engineering - 2 Volume Set, pp. 528–532. John Wiley & Sons, Chichester (1994)

    Google Scholar 

  4. Becker, S., Grunske, L., Mirandola, R., Overhage, S.: Performance Prediction of Component-Based Systems: A Survey from an Engineering Perspective. In: Reussner, R., Stafford, J.A., Szyperski, C.A. (eds.) Architecting Systems with Trustworthy Components. LNCS, vol. 3938, pp. 169–192. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  5. Becker, S., Koziolek, H., Reussner, R.: Model-based Performance Prediction with the Palladio Component Model. In: Proc. of WOSP, February5–8, 2007, pp. 54–65. ACM Sigsoft, New York (2007)

    Google Scholar 

  6. Bertolino, A., Mirandola, R.: CB-SPE Tool: Putting Component-Based Performance Engineering into Practice. In: Crnković, I., Stafford, J.A., Schmidt, H.W., Wallnau, K. (eds.) CBSE 2004. LNCS, vol. 3054, pp. 233–248. Springer, Heidelberg (2004)

    Google Scholar 

  7. Bondarev, E., Chaudron, M.R.V., de Kock, E.A.: Exploring performance trade-offs of a JPEG decoder using the DeepCompass framework. In: Proc. of WOSP 2007, pp. 153–163. ACM Press, New York (2007)

    Chapter  Google Scholar 

  8. Eskenazi, E., Fioukov, A., Hammer, D.: Performance Prediction for Component Compositions. In: Crnković, I., Stafford, J.A., Schmidt, H.W., Wallnau, K. (eds.) CBSE 2004. LNCS, vol. 3054, pp. 280–293. Springer, Heidelberg (2004)

    Google Scholar 

  9. Gorton, I., Liu, A.: Performance Evaluation of Alternative Component Architectures for Enterprise JavaBean Applications. IEEE Internet Computing 7(3), 18–23 (2003)

    Article  Google Scholar 

  10. Hamlet, D., Mason, D., Woit, D.: Component-Based Software Development: Case Studies, March 2004. Series on Component-Based Software Development, chapter Properties of Software Systems Synthesized from Components, vol. 1, pp. 129–159. World Scientific, Singapore (2004)

    Google Scholar 

  11. Höst, M., Regnell, B., Wohlin, C.: Using students as subjects - A comparative study of students and professionals in lead-time impact assessment. Empirical Software Engineering 5(3), 201–214 (2000)

    Article  MATH  Google Scholar 

  12. Jones, B., Kenward, M.G.: Design and Analysis of Cross-over Trials, 2nd edn. CRC Press, Boca Raton (2003)

    MATH  Google Scholar 

  13. Kounev, S.: Performance Modeling and Evaluation of Distributed Component-Based Systems Using Queueing Petri Nets. IEEE Trans. of SE 32(7), 486–502 (2006)

    Article  Google Scholar 

  14. Koziolek, H., Firus, V.: Empirical Evaluation of Model-based Performance Predictions Methods in Software Development. In: Reussner, R., Mayer, J., Stafford, J.A., Overhage, S., Becker, S., Schroeder, P.J. (eds.) QoSA 2005. LNCS, vol. 3712, pp. 188–202. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  15. Liu, Y., Fekete, A., Gorton, I.: Design-Level Performance Prediction of Component-Based Applications. IEEE Transactions on Software Engineering 31(11), 928–941 (2005)

    Article  Google Scholar 

  16. Martens, A.: Empirical Validation of the Model-driven Performance Prediction Approach Palladio. Master’s thesis, Universität Oldenburg (November 2007), http://sdq.ipd.uka.de/diploma_theses_study_theses/completed_theses

  17. Martens, A., Becker, S., Koziolek, H., Reussner, R.: An empirical investigation of the effort of creating reusable models for performance prediction. In: CBSE 2008, Karlsruhe, Germany (accepted, 2008)

    Google Scholar 

  18. Menasce, D., Almeida, V., Dowdy, L.: Performance by Design. Prentice Hall, Englewood Cliffs (2004)

    Google Scholar 

  19. The Palladio Component Model, http://palladio-approach.net

  20. Sitaraman, M., Kuczycki, G., Krone, J., Ogden, W.F., Reddy, A.L.N.: Performance Specification of Software Components. In: Proceedings of the 2001 symposium on Software reusability: putting software reuse in context, pp. 3–10. ACM Press, New York (2001)

    Chapter  Google Scholar 

  21. Smith, C.U., Williams, L.G.: Performance Solutions: A Practical Guide to Creating Responsive, Scalable Software. Addison-Wesley, Reading (2002)

    Google Scholar 

  22. Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in Software Engineering: an Introduction. Kluwer Academic Publishers, Norwell (2000)

    MATH  Google Scholar 

  23. Wu, X., Woodside, M.: Performance Modeling from Software Components. SIGSOFT SE Notes 29(1), 290–301 (2004)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Chair for Software Design and Quality, University of Karlsruhe (TH), Am Fasanengarten 5, 76131, Karlsruhe, Germany

    Anne Martens & Ralf Reussner

  2. FZI Forschungszentrum Informatik, Haid-und-Neu-Straße 10-14, 76131, Karlsruhe, Germany

    Steffen Becker

  3. ABB Corporate Research, Wallstadter Str. 59, 68526, Ladenburg, Germany

    Heiko Koziolek

Authors
  1. Anne Martens
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Steffen Becker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Heiko Koziolek
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ralf Reussner
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Nigel Thomas Carlos Juiz

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Martens, A., Becker, S., Koziolek, H., Reussner, R. (2008). An Empirical Investigation of the Applicability of a Component-Based Performance Prediction Method. In: Thomas, N., Juiz, C. (eds) Computer Performance Engineering. EPEW 2008. Lecture Notes in Computer Science, vol 5261. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87412-6_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-87412-6_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-87411-9

  • Online ISBN: 978-3-540-87412-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics