Abstract
Regression testing is performed to provide confidence that changes in a part of software do not affect other parts of the software. An execution of all existing test cases is the best way to re-establish this confidence. However, regression testing is an expensive process—there might be insufficient resources (e.g., time, workforce) to allow for the re-execution of all test cases. Regression test prioritization techniques attempt to re-order a regression test suite based on some criteria so that highest priority test cases are executed earlier. In this study, we prioritize test cases for regression testing based on the dependency network of faults. In software testing, it is common that some faults are the consequences of other faults (leading faults). Dependent faults can be removed if and only if the leading faults have been removed. Our goal is to prioritize test cases so that test cases that have exposed the leading faults in the system testing phase, are executed first in regression testing. The leading faults are modeled as the most central faults in the fault dependency network. We present ComReg, a test-case prioritization technique based on the dependency network of faults. We model a fault dependency network as a directed graph and identify leading faults to prioritize test cases for regression testing. We use a centrality aggregation technique which considers six network representative centrality metrics to identify leading faults in the fault dependency network. We also discuss the use of fault communities to select an arbitrary percentage of the test cases from a prioritized regression test suite. We conduct a case study that evaluates the effectiveness and applicability of the proposed method. We obtain a fault dependency network from the development of a vocabulary learning software. We found that the fault network is a small-world graph with distinguishable community structure. The leading faults are common in all centralities and a re-ordering of test cases is feasible for regression testing based on those leading faults. Our method outperforms traditional regression testing prioritization techniques in detecting fault dependencies. Our modeling of the network of faults provides insights into the requirement of recognizing fault dependencies while re-ordering regression test suites for both research and practice. The dependency model needs further evaluation and improvement considering relevant resources (e.g., man-hours).
Similar content being viewed by others
References
Leung HKN, White L (1989) Insights into regression testing [software testing]. In: Proceedings of conference on software maintenance, pp 60–69
Yoo S, Harman M (2012) Regression testing minimization, selection and prioritization: a survey. Softw Test Verif Reliab 22:67–120
Kayes I (2011) Agile testing: introducing PRAT as a metric of testing quality in scrum. ACM SIGSOFT Softw Eng Notes 36:1–5
Kayes I, Sarker M, Chakareski J (2013) On measuring test quality in scrum: an empirical study. arXiv:1310.2545
Beizer B (2003) Software testing techniques. Dreamtech Press, India
Malishevsky AG, Ruthruff JR, Rothermel G, Elbaum S (2006) Cost-cognizant test case prioritization. Department of Computer Science and Engineering, University of Nebraska-Lincoln, Technical Report
Hamlet D, Maybee J (2000) The engineering of software: a technical guide for the individual, 1st edn. Addison-Wesley Longman Publishing Co., Inc., Boston In:
Elbaum S, Malishevsky A, Rothermel G (2002) Test case prioritization: a family of empirical studies. IEEE Trans Softw Eng 28:159–182
Rothermel G, Untch R, Chu C, Harrold M (1999) Test case prioritization: an empirical study. In: Proceedings of IEEE international conference on software maintenance (ICSM’99), pp 179–188
Rothermel G, Untch R, Chu C, Harrold M (2001) Prioritizing test cases for regression testing. IEEE Trans Softw Eng 27:929–948
Elbaum S, Malishevsky AG, Rothermel G (2000) Prioritizing test cases for regression testing. In: Proceedings of the 2000 ACM SIGSOFT international symposium on software testing and analysis (ISSTA’00). ACM, New York, pp 102–112
Srikanth H, Williams L, Osborne J (2005) System test case prioritization of new and regression test cases. In: Proceedings of international symposium on empirical software engineering
Krishnamoorthi R, Sahaaya Arul Mary S (2009) Factor oriented requirement coverage based system test case prioritization of new and regression test cases. Inf Softw Technol 51:799–808
Zhang L, Hou SS, Guo C, Xie T, Mei H (2009) Time-aware test-case prioritization using integer linear programming. In: Proceedings of the eighteenth international symposium on software testing and analysis (ISSTA’09). ACM, New York, pp 213–224
Walcott KR, Soffa ML, Kapfhammer GM, Roos RS (2006) Timeaware test suite prioritization. In: Proceedings of the international symposium on software testing and analysis (ISSTA’06). ACM, New York, pp 1–12
Alspaugh S, Walcott KR, Belanich M, Kapfhammer GM, Soffa ML (2007) Efficient time-aware prioritization with knapsack solvers. In: Proceedings of the 1st ACM international workshop on empirical assessment of software engineering languages and technologies (WEASELTech’07): held in conjunction with the 22nd IEEE/ACM international conference on automated software engineering (ASE’07). ACM, New York, pp 13–18
Huang CY, Lin CT (2006) Software reliability analysis by considering fault dependency and debugging time lag. IEEE Trans Reliab 55:436–450
Neumann PG (1990) Cause of AT&T network failure. Risks Dig 9:62
Slabodkin G (1998) Software glitches leave navy smart ship dead in the water. Gov Comput News 13:33727-1
Oppenheimer D, Ganapathi A, Patterson DA (2003) Why do internet services fail, and what can be done about it? In: USENIX symposium on internet technologies and systems, vol 67, Seattle
Kayes M (2011) Test case prioritization for regression testing based on fault dependency. In: Proceedings of 3rd international conference on electronics computer technology (ICECT’11), vol 5, pp 48–52
Rooney P (2002) Microsoft’s CEO: 80-20 rule applies to bugs, not just features. http://www.crn.com/news/security/18821726/microsofts-ceo-80-20-rule-applies-to-bugs-not-just-features.htm. Accessed 25 Nov 2014
Kayes I, Qian X, Skvoretz J, Iamnitchi A (2012) How influential are you: detecting influential bloggers in a blogging community. In: Proceedings of the 4th international conference on social informatics. Springer, Berlin, pp 29–42
Kayes I, Zuo X, Wang D, Chakareski J (2014) To blog or not to blog: characterizing and predicting retention in community blogs. In: Proceedings of the international conference on social computing (SocialCom’14), vol 7. ACM, New York, pp 1–7:8
Kayes I, Zuo X, Wang D, Chakareski J (2014) Did you blog yesterday? Retention in community blogs. In: Proceedings of the international conference on social computing (SocialCom’14). ACM, New York, pp 16:1–16:2
Freeman L (1977) A set of measures of centrality based upon betweenness. Sociometry 40:35–41
Newman MEJ (2010) Networks: an introduction. Oxford University Press, USA
Brandes U (2008) On variants of shortest-path betweenness centrality and their generic computation. Soc Netw 30:136–145
Bonacich P (1972) Factoring and weighting approaches to status scores and clique identification. J Math Sociol 2:113–120
Page L, Brin S, Motwani R, Winograd T (1999) The pagerank citation ranking: bringing order to the web. Stanford InfoLab
Girvan M, Newman ME (2002) Community structure in social and biological networks. Proc Natl Acad Sci 99:7821–7826
Tyler JR, Wilkinson DM, Huberman BA (2005) E-mail as spectroscopy: automated discovery of community structure within organizations. Inf Soc 21:143–153
Nazir A, Raza S, Chuah CN (2008) Unveiling facebook: a measurement study of social network based applications. In: Proceedings of the 8th ACM SIGCOMM conference on internet measurement (IMC’08). ACM, New York, pp 43–56
Onnela JP, Saramäki J, Hyvönen J, Szabó G, Lazer D, Kaski K, Kertész J, Barabási AL (2007) Structure and tie strengths in mobile communication networks. Proc Natl Acad Sci 104:7332–7336
Kumar R, Novak J, Raghavan P, Tomkins A (2004) Structure and evolution of blogspace. Commun ACM 47:35–39
Chen J, Yuan B (2006) Detecting functional modules in the yeast protein–protein interaction network. Bioinformatics 22:2283–2290
Newman ME, Girvan M (2004) Finding and evaluating community structure in networks. Phys Rev E 69:026113
Pareto V (1971) Manual of political economy. Macmillan, London
Duong L (2009) Applying the “80–20 rule” with the standish group’s statistics on software usage. http://bit.ly/beBOf1. Accessed 25 Nov 2014
WordHacker (2006) Gre word list. http://www.wordhacker.com/en/article/Barron_gre_list_a.htm. Accessed 25 Nov 2014
Watts DJ, Strogatz S (1998) Collective dynamics of ‘small-world’ networks. Nature 393:440–442
Jeffrey D, Gupta R (2006) Test case prioritization using relevant slices. In: Proceedings of 30th annual international computer software and applications conference (COMPSAC’06), vol 1, pp 411–420
Zhi-hua Z, Yong-min M, Ying-ai T (2012) Test case prioritization for regression testing based on function call path. In: Fourth international conference on computational and information sciences (ICCIS), pp 1372–1375
Blondel VD, Guillaume JL, Lambiotte R, Lefebvre E (2008) Fast unfolding of communities in large networks. J Stat Mech Theory Exp 2008:P10008
Do H, Rothermel G, Kinneer A (2004) Empirical studies of test case prioritization in a JUnit testing environment. In: 15th international symposium on software reliability engineering (ISSRE’04), pp 113–124
Korel B, Tahat LH, Harman M (2005) Test prioritization using system models. In: Proceedings of the 21st IEEE international conference on software maintenance (ICSM’05). IEEE, pp 559–568
Korel B, Koutsogiannakis G, Tahat L (2008) Application of system models in regression test suite prioritization. In: IEEE international conference on software maintenance (ICSM’08), pp 247–256
Jones J, Harrold M (2003) Test-suite reduction and prioritization for modified condition/decision coverage. IEEE Trans Softw Eng 29:195–209
Kim JM, Porter A (2002) A history-based test prioritization technique for regression testing in resource constrained environments. In: Proceedings of the 24th international conference on software engineering (ICSE’02), pp 119–129
Sherriff M, Lake M, Williams L (2007) Prioritization of regression tests using singular value decomposition with empirical change records. In: The 18th IEEE international symposium on software reliability (ISSRE’07), pp 81–90
Leon D, Podgurski A (2003) A comparison of coverage-based and distribution-based techniques for filtering and prioritizing test cases. In: Proceedings of 14th international symposium on software reliability engineering (ISSRE’03), pp 442–453
Sampath S, Bryce R, Viswanath G, Kandimalla V, Koru A (2008) Prioritizing user-session-based test cases for web applications testing. In: Proceedings of 1st international conference on software testing, verification, and validation, pp141–150
Rummel MJ, Kapfhammer GM, Thall A (2005) Towards the prioritization of regression test suites with data flow information. In: Proceedings of the ACM symposium on applied computing (SAC’05). ACM, New York, pp 1499–1504
Qu B, Nie C, Xu B, Zhang X (2007) Test case prioritization for black box testing. In: 31st annual international of computer software and applications conference (COMPSAC’07), vol 1, pp 465–474
Faloutsos M, Faloutsos P, Faloutsos C (1999) On power-law relationships of the internet topology. SIGCOMM Comput. Commun. Rev. 29:251–262
Aiello W, Chung F, Lu L (2001) Random evolution in massive graphs. In: Proceedings of 42nd IEEE symposium on foundations of computer science, pp 510–519
Redner S (1998) How popular is your paper? An empirical study of the citation distribution. Eur Phys J B-Condens Matter Complex Syst 4:131–134
Kayes I, Kourtellis N, Quercia D, Iamnitchi A, Bonchi F (2015) The social world of content abusers in community question answering. In: Proceedings of the 24th international conference on world wide web (WWW’15). International world wide web conferences steering committee, Republic and Canton of Geneva, pp 570–580
Newman ME, Strogatz SH, Watts DJ (2001) Random graphs with arbitrary degree distributions and their applications. Phys Rev E 64:026118
Jiang K, Ediger D, Bader D (2009) Generalizing \(k\)-betweenness centrality using short paths and a parallel multithreaded implementation. In: international conference on parallel processing (ICPP’09), pp 542–549
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kayes, I., Islam, S. & Chakareski, J. The network of faults: a complex network approach to prioritize test cases for regression testing. Innovations Syst Softw Eng 11, 261–275 (2015). https://doi.org/10.1007/s11334-015-0255-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11334-015-0255-5