Skip to main content
SpringerLink
Log in

A multiple case study on risk-based testing in industry

  • RBT
  • Published:
International Journal on Software Tools for Technology Transfer Aims and scope Submit manuscript

Abstract

In many development projects, testing has to be conducted under severe pressure due to limited resources and a challenging time schedule. Risk-based testing, which utilizes identified risks of the system for testing purposes, has a high potential to improve testing as it helps to optimize the allocation of resources and provides decision support for management. But for many organizations, the integration of a risk-based approach into established testing activities is a challenging task, and there are several options to do so. In this article, we analyze how risk is defined, assessed, and applied to support and improve testing activities in projects, products, and processes. We investigate these questions empirically by a multiple case study of currently applied risk-based testing activities in industry. The case study is based on three cases from different backgrounds, i.e., a test project in context of the extension of a large Web-based information system, product testing of a measurement and diagnostic equipment for the electrical power industry, as well as a test process of a system integrator of telecommunication solutions. By analyzing and comparing these different industrial cases, we draw conclusions on the state of risk-based testing and discuss possible improvements.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Wendland, M.F., Kranz, M., Schieferdecker, I.: A systematic approach to risk-based testing using risk-annotated requirements models. In: ICSEA 2012. The Seventh International Conference on Software Engineering Advances, pp. 636–642 (2012)

  2. Runeson, P., Höst, M.: Guidelines for conducting and reporting case study research in software engineering. Empir. Softw. Eng. 14(2), 131–164 (2009)

    Article  Google Scholar 

  3. Yin, R.K.: Case study research: design and methods, vol. 5. Sage (2009)

  4. Bach, J.: Heuristic risk-based testing. Softw. Test. Qual. Eng. Mag. 11, 99 (1999)

    Google Scholar 

  5. Amland, S.: Risk-based testing: risk analysis fundamentals and metrics for software testing including a financial application case study. J. Syst. Softw. 53(3), 287–295 (2000)

    Article  Google Scholar 

  6. Redmill, F.: Theory and practice of risk-based testing. Softw. Test. Verif. Reliab. 15(1), 3–20 (2005)

  7. van Veenendaal, E.: Practical risk-based testing: the PRISMA approach. UTN, Cambridge (2012)

    Google Scholar 

  8. Felderer, M., Haisjackl, C., Breu, R., Motz, J.: Integrating manual and automatic risk assessment for risk-based testing. Software Quality. Process Automation in Software Development, pp. 159–180 (2012)

  9. ISO: ISO/IEC/IEEE 29119 Software Testing (2013). Available at http://www.softwaretestingstandard.org/. Accessed May 6 2014

  10. van Veenendaal, E. (ed.): Standard glossary of terms used in software testing, version 2.2. Technical report, International Software Testing Qualifications Board, Glossary Working Party (2012)

  11. ISO: ISO/IEC/IEEE 24765:2010 System and software engineering—Vocabulary (2010)

  12. Cavano, J., McCall, J.: A framework for the measurement of software quality. ACM SIGMETRICS Perform. Eval. Rev. 7(3–4), 133–139 (1978)

  13. Redmill, F.: Exploring risk-based testing and its implications. Softw. Test. Verif. Reliab. 14(1), 3–15 (2004)

  14. Standards Australia/New Zealand: Risk Management AS/NZS 4360, 2004 (2004)

  15. Karolak, D., Karolak, N.: Software Engineering Risk Management: A Just-in-Time Approach. IEEE Computer Society Press, Los Alamitos (1995)

  16. Chen, Y., Probert, R.L., Sims, D.P.: Specification-based regression test selection with risk analysis. In: Proceedings of the 2002 Conference of the Centre for Advanced Studies on Collaborative Research. IBM Press (2002)

  17. Stallbaum, H., Metzger, A.: Employing requirements metrics for automating early risk assessment. In: Proceedings of MeReP07, Palma de Mallorca, Spain, pp. 1–12 (2007)

  18. Stallbaum, H., Metzger, A., Pohl, K.: An automated technique for risk-based test case generation and prioritization. In: Proceedings of the 3rd International Workshop on Automation of Software Test, pp. 67–70. ACM Press, New York (2008)

  19. Felderer, M., Ramler, R.: Experiences and challenges of introducing risk-based testing in an industrial project. In: Software Quality. Increasing Value in Software and Systems Development, pp. 10–29. Springer, Berlin (2013)

  20. Yoon, H., Choi, B.: A test case prioritization based on degree of risk exposure and its empirical study. Int. J. Softw. Eng. Knowl. Eng. 21(02), 191–209 (2011)

    Article  Google Scholar 

  21. Souza, E., Gusmão, C., Venâncio, J.: Risk-based testing: a case study. In: IEEE 2010 Seventh International Conference on Information Technology: New Generations (ITNG), pp. 1032–1037 (2010)

  22. Souza, E., Gusmao, C., Alves, K., Venancio, J., Melo, R.: Measurement and control for risk-based test cases and activities. In: 10th Latin American Test Workshop, pp. 1–6. IEEE Press, New York (2009)

  23. Borland: SilkCentral (2013). Available at http://www.borland.com/products/silkcentral/. Accessed November 30 2013

  24. Microtool: in-Step (2013). Available at http://www.microtool.de/inStep. Accessed November 30 2013

  25. McCabe, T.: A complexity measure. IEEE Trans. Softw. Eng. 308–320 (1976)

  26. SonarSource: Sonar (2013). Available at http://www.sonarsource.org/. Accessed November 30 2013

  27. Felderer, M., Beer, A.: Using defect taxonomies to improve the maturity of the system test process: results from an industrial case study. In: Software Quality. Increasing Value in Software and Systems Development, LNBIP 133, pp. 125–146. Springer, Berlin (2013)

Download references

Acknowledgments

This work has been supported by the COMET Competence Center program of the Austrian Research Promotion Agency (FFG), the project QE LaB Living Models for Open Systems (http://www.qe-lab.at) funded by the Austrian Federal Ministry of Economics (Bundesministerium für Wirtschaft und Arbeit), the project MOBSTECO funded by the Austrian Science Fund (FWF) as well as the competence network Softnet Austria (http://www.soft-net.at) funded by the Austrian Federal Ministry of Economics (Bundesministerium für Wirtschaft und Arbeit), the province of Styria, the Steirische Wirtschaftsförderungsgesellschaft mbH (SFG), and the city of Viennas Center for Innovation and Technology (ZIT).

Author information

Authors and Affiliations

  1. University of Innsbruck, Innsbruck, Austria

    Michael Felderer

  2. Software Competence Center Hagenberg, Hagenberg, Austria

    Rudolf Ramler

Authors
  1. Michael Felderer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rudolf Ramler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Felderer.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Felderer, M., Ramler, R. A multiple case study on risk-based testing in industry. Int J Softw Tools Technol Transfer 16, 609–625 (2014). https://doi.org/10.1007/s10009-014-0328-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10009-014-0328-z

Keywords

Search

Navigation