Skip to main content
SpringerLink
Log in

Prioritizing Test Cases with Markov Chains: A Preliminary Investigation

  • Conference paper
  • First Online:
Testing Software and Systems (ICTSS 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14131))

Included in the following conference series:

  • 271 Accesses

Abstract

Test Case Prioritization reduces the cost of software testing by executing earlier the subset of test cases showing higher priorities. The methodology consists of ranking test cases so that, in case of a limited budget, only the top-ranked tests are exercised. One possible direction for prioritizing test cases relies on considering the usage frequency of a software sub-system. To this end, a promising direction is to identify the likelihood of events occurring in software systems, and this can be achieved by adopting Markov chains. This paper presents a novel approach that analyzes the system scenarios modeled as a Markov chain and ranks the generated test sequences to prioritize test cases. To assess the proposed approach, we developed an algorithm and conducted a preliminary and experimental study that investigates the feasibility of using Markov chains as an appropriate means to prioritize test cases. We demonstrate the strength of the novel strategy by evaluating two heuristics, namely H1 (based on the transition probabilities) and H2 (based on the steady-state probabilities), with established metrics. Results show (i) coverage of 100% for both H1 and H2, and (ii) efficiency equal to 98.4% for H1 and 99.4% for H2, on average.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. SATM: Simple Automatic Teller Machine. https://slideplayer.com/slide/3835819/. Accessed 4 Oct 2022

  2. Barbosa, G., de Souza, É.F., dos Santos, L.B.R., da Silva, M., Balera, J.M., Vijaykumar, N.L.: A systematic literature review on prioritizing software test cases using Markov chains. Inf. Softw. Technol. 147, 106902 (2022)

    Article  Google Scholar 

  3. Bertolino, A., Miranda, B., Pietrantuono, R., Russo, S.: Adaptive coverage and operational profile-based testing for reliability improvement. In: International Conference on Software Engineering (ICSE), pp. 541–551 (2017)

    Google Scholar 

  4. Bohme, M., Pham, V.T., Roychoudhury, A.: Coverage-based Greybox fuzzing as Markov chain. IEEE Trans. Software Eng. 45(5), 489–506 (2019)

    Article  Google Scholar 

  5. Bolch, G., Greiner, S., De Meer, H., Trivedi, K.S.: Queueing Networks and Markov Chains: Modeling and Performance Evaluation with Computer Science Applications. John Wiley & Sons (2006)

    Google Scholar 

  6. Brémaud, P.: Markov Chains: Gibbs fields, Monte Carlo Simulation, and Queues, vol. 31. Springer, New York (2013). https://doi.org/10.1007/978-3-030-45982-6

    Book  MATH  Google Scholar 

  7. Cai, K.Y.: Optimal software testing and adaptive software testing in the context of software cybernetics. Inf. Softw. Technol. 44(14), 841–855 (2002)

    Article  Google Scholar 

  8. Camilli, M., Gargantini, A., Scandurra, P., Trubiani, C.: Uncertainty-aware exploration in model-based testing. In: IEEE Conference on Software Testing, Verification and Validation (ICST), pp. 71–81 (2021)

    Google Scholar 

  9. Cruciani, E., Miranda, B., Verdecchia, R., Bertolino, A.: Scalable approaches for test suite reduction. In: Proceedings of the International Conference on Software Engineering (ICSE), pp. 419–429 (2019)

    Google Scholar 

  10. Devroey, X., et al.: Statistical prioritization for software product line testing: an experience report. Softw. Syst. Model. 16(1), 153–171 (2015)

    Article  Google Scholar 

  11. Elbaum, S., Malishevsky, A.G., Rothermel, G.: Test case prioritization: a family of empirical studies. IEEE Trans. Software Eng. 28(2), 159–182 (2002)

    Article  Google Scholar 

  12. Everett, G.D., McLeod Jr., R.: Software Testing. Testing Across the Entire (2007)

    Google Scholar 

  13. Ferreira, A.R.: Análise e Melhoria de Processos, p. 59 (2013)

    Google Scholar 

  14. Gagniuc, P.A.: Markov Chains: From Theory to Implementation and Experimentation. Wiley (2017)

    Google Scholar 

  15. Jorgensen, P.C.: Software Testing: A Craftsman’s Approach. Auerbach Publications (2013)

    Google Scholar 

  16. Juntao, W., Mishima, N.: Development of resource efficiency index for electrical and electronic equipment. Procedia CIRP 61, 275–280 (2017)

    Article  Google Scholar 

  17. Kashyap, A., Holzer, T., Sarkani, S., Eveleigh, T.: Model based testing for software systems: an application of Markov modulated Markov process. Int. J. Comput. Appl. 46(14), 13–20 (2012)

    Google Scholar 

  18. Kaur, A., Goyal, S.: A genetic algorithm for regression test case prioritization using code coverage. Int. J. Comput. Sci. Eng. 3(5), 1839–1847 (2011)

    Google Scholar 

  19. Konsaard, P., Ramingwong, L.: Total coverage based regression test case prioritization using genetic algorithm. In: 2015 12th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), pp. 1–6. IEEE (2015)

    Google Scholar 

  20. Lee, D., Yannakakis, M.: Principles and methods of testing finite state machines - a survey. Proc. IEEE 84(8), 1090–1123 (1996)

    Article  Google Scholar 

  21. Miranda, B., Cruciani, E., Verdecchia, R., Bertolino, A.: FAST approaches to scalable similarity-based test case prioritization. In: Proceedings of the International Conference on Software Engineering (ICSE), pp. 222–232 (2018)

    Google Scholar 

  22. Morozov, A., Ding, K., Chen, T., Janschek, K.: Test suite prioritization for efficient regression testing of model-based automotive software. In: 2017 International Conference on Software Analysis, Testing and Evolution (SATE), pp. 20–29 (2017)

    Google Scholar 

  23. Muthyala, K., Naidu, R.: A novel approach to test suite reduction using data mining approach. Indian J. Comput. Sci. Eng. 2(3), 500–505 (2011)

    Google Scholar 

  24. Ouriques, J.F.S., Cartaxo, E.G., Machado, P.D.: Test case prioritization techniques for model-based testing: a replicated study. Software Qual. J. 26(4), 1451–1482 (2018)

    Article  Google Scholar 

  25. Ozawa, M., Dohi, T., Okamura, H.: How do software metrics affect test case prioritization? In: Annual Computer Software and Applications Conference (COMPSAC), vol. 01, pp. 245–250 (2018)

    Google Scholar 

  26. Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley (2014)

    Google Scholar 

  27. Raiyani, A.G., Pandya, S.S.: Proritization technique for minimizing number of test cases. Int. J. Softw. Eng. Res. Pract. 1, 3–9 (2011)

    Google Scholar 

  28. Saad, Y.: Numerical Methods for Large Eigenvalue Problems. Manchester University Press, Manchester (1992)

    MATH  Google Scholar 

  29. Sayyari, F., Emadi, S.: Automated generation of software testing path based on ant colony. In: 2015 International Congress on Technology, Communication and Knowledge (ICTCK), pp. 435–440. IEEE, November 2015

    Google Scholar 

  30. Singh, L.K., Tripathi, A.K., Vinod, G.: Software reliability early prediction in architectural design phase: overview and limitations. J. Softw. Eng. Appl. 4, 181–186 (2011)

    Article  Google Scholar 

  31. Tijms, H.C., Tijms, H.C.: Stochastic Models: An Algorithmic Approach, vol. 303. Wiley, New York (1994)

    MATH  Google Scholar 

  32. Walton, G., Poore, J.: Measuring complexity and coverage of software specifications. Inf. Softw. Technol. 42(12), 859–872 (2000)

    Google Scholar 

  33. Whittaker, J.A., Thomason, M.G.: A Markov chain model for statistical software testing. IEEE Trans. Software Eng. 20(10), 812–824 (1994)

    Article  Google Scholar 

  34. Xuan, J., Jiang, H., Ren, Z., Hu, Y., Luo, Z.: A random walk based algorithm for structural test case generation. In: The 2nd International Conference on Software Engineering and Data Mining, pp. 583–588 (2010)

    Google Scholar 

  35. Yoo, S., Harman, M.: Regression testing minimization, selection and prioritization: a survey. Softw. Test. Verification Reliab. 22(2), 67–120 (2012)

    Article  Google Scholar 

  36. Zhou, B., Okamura, H., Dohi, T.: Application of Markov chain Monte Carlo random testing to test case prioritization in regression testing. IEICE Trans. Inf. Syst. E95.D(9), 2219–2226 (2012)

    Google Scholar 

Download references

Acknowledgements

This work has been partially funded by MUR PRIN 2017TWRCNB SEDUCE, and the PNRR MUR VITALITY (ECS00000041) Spoke 2 ASTRA - Advanced Space Technologies and Research Alliance. The authors acknowledge the support of the MUR (Italy) Department of Excellence 2023–2027 for GSSI.

Author information

Authors and Affiliations

  1. Gran Sasso Science Institute - GSSI, L’Aquila, Italy

    Luciana Rebelo & Catia Trubiani

  2. Universidade Tecnológica Federal do Paraná - UTFPR, Cornélio Procópio, Brazil

    Érica Souza

  3. Universidade Federal de Santa Catarina - UFSC, Joinville, Brazil

    Gian Berkenbrock

  4. Universidade Estadual Paulista - Unesp, Guaratinguetá, Brazil

    Gerson Barbosa

  5. Instituto Federal de Educação, Ciência e Tecnologia de São Paulo - IFSP, Jacareí, Brazil

    Luciana Rebelo

  6. Universidade Federal de São Carlos - UFSCar, São Carlos, Brazil

    André Endo

  7. Instituto Nacional de Pesquisas Espaciais - INPE, São José dos Campos, Brazil

    Nandamudi Vijaykumar

  8. Instituto Federal de Educação, Ciência e Tecnologia de São Paulo - IFSP, Campos do Jordão, Brazil

    Marlon Silva

Authors
  1. Luciana Rebelo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Érica Souza
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gian Berkenbrock
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gerson Barbosa
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marlon Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. André Endo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Nandamudi Vijaykumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Catia Trubiani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luciana Rebelo .

Editor information

Editors and Affiliations

  1. University of Bergamo, Dalmine, Italy

    Silvia Bonfanti

  2. University of Bergamo, Dalmine, Italy

    Angelo Gargantini

  3. Salvaneschi & Partners, Bergamo, Italy

    Paolo Salvaneschi

Rights and permissions

Reprints and permissions

Copyright information

© 2023 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rebelo, L. et al. (2023). Prioritizing Test Cases with Markov Chains: A Preliminary Investigation. In: Bonfanti, S., Gargantini, A., Salvaneschi, P. (eds) Testing Software and Systems. ICTSS 2023. Lecture Notes in Computer Science, vol 14131. Springer, Cham. https://doi.org/10.1007/978-3-031-43240-8_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43240-8_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43239-2

  • Online ISBN: 978-3-031-43240-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation