Skip to main content
SpringerLink
Log in

Code-based prioritization: a pre-testing effort to minimize post-release failures

  • Original Paper
  • Published:
Innovations in Systems and Software Engineering Aims and scope Submit manuscript

Abstract

Improving the efficiency of the testing process is a challenging goal. Prior work has shown that often a small number of errors account for the majority of software failures; and often, most errors are found in a small portion of a source code. We argue that prioritizing code elements before conducting testing can help testers focus their testing effort on the parts of the code most likely to expose errors. This can, in turn, promote more efficient testing of software. Keeping this in view, we propose a testing effort prioritization method to guide tester during software development life cycle. Our approach considers five factors of a component such as Influence value, Average execution time, Structural complexity, Severity and Value as inputs and produce the priority value of the component as an output. Once all components of a program have been prioritized, testing effort can be apportioned so that the components causing more frequent and/or more severe failures will be tested more thoroughly. Our proposed approach is effective in guiding testing effort as it is linked to external measure of defect severity and business value, internal measure of frequency and complexity. As a result, the failure rate is decreased and the chance of severe type of failures is also decreased in the operational environment. We have conducted experiments to compare our scheme with a related scheme. The results establish that our proposed approach that prioritizes the testing effort within the source code is able to minimize highly severed types of failures and also number of failures at the post-release time of a software system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Adams EN (1984) Optimizing preventive service of software products. IBM J Res Dev 28(01): 3–14

    Article  Google Scholar 

  2. Andrews J, Briand L, Labiche Y (2005) Is mutation an appropriate tool for testing experiments? In: IEEE internal conference on software engineering, pp 402–411

  3. Binder R (2000) Testing Object-Oriented Systems Models. Patterns and Tools. Addison-Wesley

  4. Boehm B (2003) Value-based software engineering: rein- venting. SIGSOFT Softw Eng Notes 28(2): 3. doi:10.1145/638750.638775

    Article  Google Scholar 

  5. Boehm B, Basili VR (2001) Software defect reduction top 10 list. Computer 34(1): 135–137. doi:10.1109/2.962984

    Article  Google Scholar 

  6. Boehm B, Huang L (2003) Value based software engineering: a case study. IEEE Computer, pp 21–29

  7. Boehm B, Clark B, Horowitz E, Westland C, Madachy R, Selby R (1995) Cost models for future software life cycle processes: Cocomo 2.0

  8. Booth T (1979) Performance optimization of software systems processing information sequences modeled by probabilistic languages. IEEE Transactions Softw Eng 5(1): 31–44

    Article  MATH  Google Scholar 

  9. Briand LC, Wüst J, Daly JW, Porter DV (2000) Exploring the relationship between design measures and software quality in object-oriented systems. J Syst Softw 51(3): 245–273. doi:10.1016/S0164-1212(99)00102-8

    Article  Google Scholar 

  10. Briand LC, Labiche Y, Wang Y (2003) A comprehensive and systematic methodology for client-server class integration testing. In: ISSRE’03: proceedings of the 14th international symposium on software reliability engineering. IEEE Computer Society. Washington, USA, p 14

  11. Cheung RC (1980) A user-oriented software reliability model. IEEE Trans Softw Eng 6(2): 118–125

    Article  MATH  Google Scholar 

  12. Chidamber SR, Kemerer CF (1994) A metrics suite for object-oriented design. IEEE Trans Softw Eng 20(6): 476–493. doi:10.1109/32.295895

    Article  Google Scholar 

  13. Cobb RH, Mills HD (1990) Engineering software under statistical quality control. IEEE Softw 7(6): 44–54

    Article  Google Scholar 

  14. Delamaro ME, Maldonado JC, Pasquini A, Mathur AP (2001) Interface mutation test adequacy criterion: An empirical evaluation. Empir Softw Eng 6(2): 111–142. doi:10.1023/A:1011429104252

    Article  MATH  Google Scholar 

  15. Department D (1984) Procedures for performing a failure mode effects. and criticality analysis. US MIL STD 1629A/Notice 2

  16. Elbaum S, Malishevsky A, Rothermel G (2001) Incorporating varying test costs and fault severities into test case prioritization. In: ICSE’01: proceedings of the 23rd international conference on software engineering. IEEE Computer Society. Washington, USA, pp 329–338

  17. Elbaum S, Malishevsky AG, Rothermel G (2002) Test case prioritization: a family of empirical studies. IEEE Trans Softw Eng 28(2): 159–182. doi:10.1109/32.988497

    Article  Google Scholar 

  18. Hassan A, Goseva-Popstojanova K, Ammar H (2005) Uml based severity analysis methodology. In: Proceedings of annual reliability and maintainability symposium (RAMS 2005), Alexandria, pp 158–164

  19. Huang L, Boehm B (2006) How much software quality investment is enough: a value-based approach. IEEE Softw 23(5): 88–95. doi:10.1109/MS.2006.127

    Article  Google Scholar 

  20. Jeffrey D, Gupta N (2008) Experiments with test case prioritization using relevant slices. J Syst Softw 81(2): 196–221. doi:10.1016/j.jss.2007.05.006

    Article  Google Scholar 

  21. John SK, Clark JA, Mcdermid JA (2000) Class mutation: mutation testing for object-oriented programs. In: Proc. Net. ObjectDays, pp 9–12

  22. Larsen L, Harrold MJ (1996) Slicing object-oriented software. In: ICSE’96: Proceedings of the 18th international conference on Software engineering. IEEE Computer Society. Washington, USA, pp 495–505

  23. Li J, Weiss D, Yee H (2006) Code-coverage guided prioritized test generation. Inf Softw Technol 48(12): 1187–1198

    Article  Google Scholar 

  24. Li JJ (2005) Prioritize code for testing to improve code coverage of complex software. In: ISSRE’05: Proceedings of the 16th IEEE international symposium on software reliability engineering. IEEE Computer Society. Washington, USA, pp 75–84. doi:10.1109/ISSRE.2005.33

  25. Li Q, Li M, Yang Y, Wang Q, Tan T, Boehm B, Hu C (2009) Bridge the gap between software test process and business value: a case study. In: ICSP’09 Proceedings of the International Conference on Software Process: Trustworthy Software Development Processes

  26. Liang D, Harrold MJ (1998) Slicing objects using system dependence graphs. In: ICSM’98: Proceedings of the international conference on software maintenance. IEEE Computer Society. Washington, USA, p 358

  27. Littlewood B (1975) A reliability model for systems with markov structure. J Royal Stat Soc Series C (Applied Statistics) 24(2): 172–177

    Article  MathSciNet  Google Scholar 

  28. Lo JH, Kuo SY, Lyu MR, Huang CY (2002) Optimal resource allocation and reliability analysis for component-based software applications. In: Proceedings of the 26th Annual International Computer Software and Applications Conference (COMPSAC’02), pp 7–12

  29. Luke SR (1995) Failure mode, effects and criticality analysis (fmeca) for software. In: 5th Fleet maintenance symposium. Virginia Beach, VA (USA), pp 731–735

  30. Mall R (2009) Fundamentals of Software Engineering, 3rd edn. Prentice Hall of India

  31. Mall R (2009) Fundamentals of Software Engineering, 3rd edn. Prentice Hall of India

  32. Malloy BA, McGregor JD, Krishnaswamy A, Medikonda M (1994) An extensible program representation for object-oriented software. SIGPLAN Not 29(12): 38–47. doi:10.1145/193209.193218

    Article  Google Scholar 

  33. Musa JD (1993) Operational profiles in software-reliability engineering. IEEE Softw 10(2): 14–32. doi:10.1109/52.199724

    Article  Google Scholar 

  34. Musa JD (2004) Software Reliability Engineering: More Reliable Software Faster and Cheaper. AuthorHouse

  35. Point O (2008) http://sunset.usc.edu/csse/research/cocomoii/cocomomain.html

  36. Profiler http://www.gnu.org/manual/gprof-2.9.1

  37. Ramler R, Biffl S, Grunbacher P (2005) Value-Based Management of Software Testing. Springer, Berlin, pp 225–244. ISBN: 3-540-25993-7

  38. Ray M, Mohapatra DP (2010a) Reliability improvement based on prioritization of source code. In: Janowski T. Mohanty H (eds) ICDCIT. Lecture Notes in Computer Science, vol 5966. Springer, Berlin, pp 212–223

  39. Ray M, Mohapatra DP (2010b) A scheme to prioritize classes at the early stage for improving observable reliability. In: Proceedings of the 3rd India software engineering conference. ACM. New York. ISEC’10, pp 69–72

  40. Ray M, Kumawat K lal, Mohapatra D Prasad (2011) Source code prioritization using forward slicing for exposing critical elements in a program. J Comput Sci Technol 26(2): 314–327

    Article  Google Scholar 

  41. Rosenberg LH, Stapko R, Gallo A (1999) Risk-based object-oriented testing. In: Proceedings of 24th annual software engineering Workshop. NASA. Software Engineering Laboratory

  42. Rothermel G, Untch RH, Chu C, Harrold MJ (2001) Prioritizing test cases for regression testing. IEEE Trans Softw Eng 27(10):929–948. doi:10.1109/32.962562. http://doi.ieeecomputersociety.org/

    Google Scholar 

  43. Sommerville I (1995) Software Engineering. 5th edn. Pearson. chapter 18

  44. Srikant, YN, Shankar, P (eds) (2002) The Compiler Design Handbook: Optimizations and Machine Code Generation. CRC Press, Boca Raton

    Google Scholar 

  45. Weiser M (1984) Program slicing. IEEE Trans Software Eng 10(4): 352–357

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, National Institute of Technology, Rourkala, Orissa, 769008, India

    Mitrabinda Ray & Durga Prasad Mohapatra

Authors
  1. Mitrabinda Ray
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Durga Prasad Mohapatra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mitrabinda Ray.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ray, M., Mohapatra, D.P. Code-based prioritization: a pre-testing effort to minimize post-release failures. Innovations Syst Softw Eng 8, 279–292 (2012). https://doi.org/10.1007/s11334-012-0186-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11334-012-0186-3

Keywords

Search

Navigation