Abstract
Improving the efficiency of the testing process is a challenging goal. Prior work has shown that often a small number of errors account for the majority of software failures; and often, most errors are found in a small portion of a source code. We argue that prioritizing code elements before conducting testing can help testers focus their testing effort on the parts of the code most likely to expose errors. This can, in turn, promote more efficient testing of software. Keeping this in view, we propose a testing effort prioritization method to guide tester during software development life cycle. Our approach considers five factors of a component such as Influence value, Average execution time, Structural complexity, Severity and Value as inputs and produce the priority value of the component as an output. Once all components of a program have been prioritized, testing effort can be apportioned so that the components causing more frequent and/or more severe failures will be tested more thoroughly. Our proposed approach is effective in guiding testing effort as it is linked to external measure of defect severity and business value, internal measure of frequency and complexity. As a result, the failure rate is decreased and the chance of severe type of failures is also decreased in the operational environment. We have conducted experiments to compare our scheme with a related scheme. The results establish that our proposed approach that prioritizes the testing effort within the source code is able to minimize highly severed types of failures and also number of failures at the post-release time of a software system.
Similar content being viewed by others
References
Adams EN (1984) Optimizing preventive service of software products. IBM J Res Dev 28(01): 3–14
Andrews J, Briand L, Labiche Y (2005) Is mutation an appropriate tool for testing experiments? In: IEEE internal conference on software engineering, pp 402–411
Binder R (2000) Testing Object-Oriented Systems Models. Patterns and Tools. Addison-Wesley
Boehm B (2003) Value-based software engineering: rein- venting. SIGSOFT Softw Eng Notes 28(2): 3. doi:10.1145/638750.638775
Boehm B, Basili VR (2001) Software defect reduction top 10 list. Computer 34(1): 135–137. doi:10.1109/2.962984
Boehm B, Huang L (2003) Value based software engineering: a case study. IEEE Computer, pp 21–29
Boehm B, Clark B, Horowitz E, Westland C, Madachy R, Selby R (1995) Cost models for future software life cycle processes: Cocomo 2.0
Booth T (1979) Performance optimization of software systems processing information sequences modeled by probabilistic languages. IEEE Transactions Softw Eng 5(1): 31–44
Briand LC, Wüst J, Daly JW, Porter DV (2000) Exploring the relationship between design measures and software quality in object-oriented systems. J Syst Softw 51(3): 245–273. doi:10.1016/S0164-1212(99)00102-8
Briand LC, Labiche Y, Wang Y (2003) A comprehensive and systematic methodology for client-server class integration testing. In: ISSRE’03: proceedings of the 14th international symposium on software reliability engineering. IEEE Computer Society. Washington, USA, p 14
Cheung RC (1980) A user-oriented software reliability model. IEEE Trans Softw Eng 6(2): 118–125
Chidamber SR, Kemerer CF (1994) A metrics suite for object-oriented design. IEEE Trans Softw Eng 20(6): 476–493. doi:10.1109/32.295895
Cobb RH, Mills HD (1990) Engineering software under statistical quality control. IEEE Softw 7(6): 44–54
Delamaro ME, Maldonado JC, Pasquini A, Mathur AP (2001) Interface mutation test adequacy criterion: An empirical evaluation. Empir Softw Eng 6(2): 111–142. doi:10.1023/A:1011429104252
Department D (1984) Procedures for performing a failure mode effects. and criticality analysis. US MIL STD 1629A/Notice 2
Elbaum S, Malishevsky A, Rothermel G (2001) Incorporating varying test costs and fault severities into test case prioritization. In: ICSE’01: proceedings of the 23rd international conference on software engineering. IEEE Computer Society. Washington, USA, pp 329–338
Elbaum S, Malishevsky AG, Rothermel G (2002) Test case prioritization: a family of empirical studies. IEEE Trans Softw Eng 28(2): 159–182. doi:10.1109/32.988497
Hassan A, Goseva-Popstojanova K, Ammar H (2005) Uml based severity analysis methodology. In: Proceedings of annual reliability and maintainability symposium (RAMS 2005), Alexandria, pp 158–164
Huang L, Boehm B (2006) How much software quality investment is enough: a value-based approach. IEEE Softw 23(5): 88–95. doi:10.1109/MS.2006.127
Jeffrey D, Gupta N (2008) Experiments with test case prioritization using relevant slices. J Syst Softw 81(2): 196–221. doi:10.1016/j.jss.2007.05.006
John SK, Clark JA, Mcdermid JA (2000) Class mutation: mutation testing for object-oriented programs. In: Proc. Net. ObjectDays, pp 9–12
Larsen L, Harrold MJ (1996) Slicing object-oriented software. In: ICSE’96: Proceedings of the 18th international conference on Software engineering. IEEE Computer Society. Washington, USA, pp 495–505
Li J, Weiss D, Yee H (2006) Code-coverage guided prioritized test generation. Inf Softw Technol 48(12): 1187–1198
Li JJ (2005) Prioritize code for testing to improve code coverage of complex software. In: ISSRE’05: Proceedings of the 16th IEEE international symposium on software reliability engineering. IEEE Computer Society. Washington, USA, pp 75–84. doi:10.1109/ISSRE.2005.33
Li Q, Li M, Yang Y, Wang Q, Tan T, Boehm B, Hu C (2009) Bridge the gap between software test process and business value: a case study. In: ICSP’09 Proceedings of the International Conference on Software Process: Trustworthy Software Development Processes
Liang D, Harrold MJ (1998) Slicing objects using system dependence graphs. In: ICSM’98: Proceedings of the international conference on software maintenance. IEEE Computer Society. Washington, USA, p 358
Littlewood B (1975) A reliability model for systems with markov structure. J Royal Stat Soc Series C (Applied Statistics) 24(2): 172–177
Lo JH, Kuo SY, Lyu MR, Huang CY (2002) Optimal resource allocation and reliability analysis for component-based software applications. In: Proceedings of the 26th Annual International Computer Software and Applications Conference (COMPSAC’02), pp 7–12
Luke SR (1995) Failure mode, effects and criticality analysis (fmeca) for software. In: 5th Fleet maintenance symposium. Virginia Beach, VA (USA), pp 731–735
Mall R (2009) Fundamentals of Software Engineering, 3rd edn. Prentice Hall of India
Mall R (2009) Fundamentals of Software Engineering, 3rd edn. Prentice Hall of India
Malloy BA, McGregor JD, Krishnaswamy A, Medikonda M (1994) An extensible program representation for object-oriented software. SIGPLAN Not 29(12): 38–47. doi:10.1145/193209.193218
Musa JD (1993) Operational profiles in software-reliability engineering. IEEE Softw 10(2): 14–32. doi:10.1109/52.199724
Musa JD (2004) Software Reliability Engineering: More Reliable Software Faster and Cheaper. AuthorHouse
Point O (2008) http://sunset.usc.edu/csse/research/cocomoii/cocomomain.html
Ramler R, Biffl S, Grunbacher P (2005) Value-Based Management of Software Testing. Springer, Berlin, pp 225–244. ISBN: 3-540-25993-7
Ray M, Mohapatra DP (2010a) Reliability improvement based on prioritization of source code. In: Janowski T. Mohanty H (eds) ICDCIT. Lecture Notes in Computer Science, vol 5966. Springer, Berlin, pp 212–223
Ray M, Mohapatra DP (2010b) A scheme to prioritize classes at the early stage for improving observable reliability. In: Proceedings of the 3rd India software engineering conference. ACM. New York. ISEC’10, pp 69–72
Ray M, Kumawat K lal, Mohapatra D Prasad (2011) Source code prioritization using forward slicing for exposing critical elements in a program. J Comput Sci Technol 26(2): 314–327
Rosenberg LH, Stapko R, Gallo A (1999) Risk-based object-oriented testing. In: Proceedings of 24th annual software engineering Workshop. NASA. Software Engineering Laboratory
Rothermel G, Untch RH, Chu C, Harrold MJ (2001) Prioritizing test cases for regression testing. IEEE Trans Softw Eng 27(10):929–948. doi:10.1109/32.962562. http://doi.ieeecomputersociety.org/
Sommerville I (1995) Software Engineering. 5th edn. Pearson. chapter 18
Srikant, YN, Shankar, P (eds) (2002) The Compiler Design Handbook: Optimizations and Machine Code Generation. CRC Press, Boca Raton
Weiser M (1984) Program slicing. IEEE Trans Software Eng 10(4): 352–357
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ray, M., Mohapatra, D.P. Code-based prioritization: a pre-testing effort to minimize post-release failures. Innovations Syst Softw Eng 8, 279–292 (2012). https://doi.org/10.1007/s11334-012-0186-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11334-012-0186-3