Amol Saxena
ABSTRACT
Automated software fault localization techniques aid developers in program debugging by identifying the probable locations of faults in a program with minimum human intervention. As software is growing in complexity and scale today, increasing the efficiency of fault localization techniques is very essential in order to reduce the overall software development cost. The effectiveness of the test suites used in the fault localization process has a significant impact on the efficiency of the process. Previous studies, on the other hand, have placed less focus on the adequacy of test suites for the fault localization process. We apply optimized test suites in this paper to improve the performance of software fault localization in a single-fault scenario. For our experiments, we use spectrum-based fault localization (SBFL) techniques. Because of its minimal computing overhead and scalability, spectrum-based fault localization is a popular, efficient, and yet lightweight fault localization technique. To optimize the test suite, we employ a heuristic that asserts that if a faulty statement is executed by a passing test case, that test case will have a negative impact on fault localization performance. In contrast, if a passing test case does not execute the faulty statement, the faulty statement's suspiciousness increases, which has a positive impact on fault localization performance. The test suite optimization approach used in this paper significantly improves fault localization performance, as demonstrated by our experiments. The results show that the proposed method efficiently reduces the number of statements examined by about 84.94 percent on average.
- W. Eric Wong, Ruizhi Gao, Yihao Li, Rui Abreu, and Franz Wotawa. 2016. A Survey on Software Fault Localization. IEEE Transactions on Software Engineering 42 (8): 707–40. https://doi.org/10.1109/tse.2016.2521368.Google Scholar
Digital Library
- Higor A. de Souza, Marcos L. Chaim, and Fabio Kon. 2016. Spectrum-Based Software Fault Localization: A Survey of Techniques, Advances, and Challenges. ArXiv [Cs.SE]. http://arxiv.org/abs/1607.04347.Google Scholar
- Amol Saxena, Roheet Bhatnagar, and Devesh Kumar Srivastava. 2021. Improving Effectiveness of Spectrum-Based Software Fault Localization Using Mutation Testing. In 2021 2nd International Conference for Emerging Technology (INCET). IEEE. https://doi.org/10.1109/incet51464.2021.9456109.Google ScholarCross Ref
- Yan Lei, Chengnian Sun, Xiaoguang Mao, and Zhendong Su. 2018. How Test Suites Impact Fault Localisation Starting from the Size. IET Software 12 (3): 190–205. https://doi.org/10.1049/iet-sen.2017.0026.Google ScholarDigital Library
- Alexandre Perez, Rui Abreu, and Arie van Deursen. 2017. “A Test-Suite Diagnosability Metric for Spectrum-Based Fault Localization Approaches.” In 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE). IEEE. https://doi.org/10.1109/icse.2017.66.Google ScholarDigital Library
- Laura Inozemtseva, and Reid Holmes. 2014. “Coverage Is Not Strongly Correlated with Test Suite Effectiveness.” In Proceedings of the 36th International Conference on Software Engineering. New York, NY, USA: ACM. https://doi.org/10.1145/2568225.2568271.Google ScholarDigital Library
- Jifeng Xuan, and Martin Monperrus. 2014. “Test Case Purification for Improving Fault Localization.” In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. New York, NY, USA: ACM. https://doi.org/10.1145/2635868.2635906.Google ScholarDigital Library
- Rui Abreu, Peter Zoeteweij, and Arjan J. C. van Gemund. 2007. On the Accuracy of Spectrum-Based Fault Localization. In Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION 2007). IEEE. https://doi.org/10.1109/taicpart.2007.4344104.Google ScholarCross Ref
- Yong Wang, Zhiqiu Huang, Yong Li, and Bingwu Fang. 2017. Lightweight Fault Localization Combined with Fault Context to Improve Fault Absolute Rank. Science China Information Sciences 60 (9). https://doi.org/10.1007/s11432-017-9112-2.Google ScholarCross Ref
- Spencer Pearson, Jose Campos, Rene Just, Gordon Fraser, Rui Abreu, Michael D. Ernst, Deric Pang, and Benjamin Keller. 2017. Evaluating and Improving Fault Localization. In 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE). IEEE. https://doi.org/10.1109/icse.2017.62.Google ScholarDigital Library
- James A. Jones, and Mary Jean Harrold. 2005. Empirical Evaluation of the Tarantula Automatic Fault-Localization Technique. In Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering - ASE ’05. New York, New York, USA: ACM Press. https://doi.org/10.1145/1101908.1101949.Google ScholarDigital Library
- Rui Abreu, Peter Zoeteweij, and Arjan Van Gemund. 2006. An Evaluation of Similarity Coefficients for Software Fault Localization. In 2006 12th Pacific Rim International Symposium on Dependable Computing (PRDC’06). IEEE. https://doi.org/10.1109/prdc.2006.18.Google ScholarDigital Library
- Rui Abreu, Peter Zoeteweij, Arjan J.C.van Gemund. 2011. Simultaneous Debugging of Software Faults. The Journal of Systems and Software 84 (4): 573–86. https://doi.org/10.1016/j.jss.2010.11.915.Google ScholarDigital Library
- W. Eric Wong, Vidroha Debroy, Ruizhi Gao, and Yihao Li. 2014. The DStar Method for Effective Software Fault Localization. IEEE Transactions on Reliability 63 (1): 290–308. https://doi.org/10.1109/tr.2013.2285319.Google ScholarCross Ref
- Rui Abreu, Peter Zoeteweij, Rob Golsteijn, and Arjan J. C. van Gemund. 2009. A Practical Evaluation of Spectrum-Based Fault Localization. The Journal of Systems and Software 82 (11): 1780–92. https://doi.org/10.1016/j.jss.2009.06.035.Google ScholarDigital Library
- Hyunsook Do, Sebastian Elbaum, and Gregg Rothermel. 2005. Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and Its Potential Impact. Empirical Software Engineer 10 (4): 405–35. https://doi.org/10.1007/s10664-005-3861-2.Google ScholarDigital Library
- Yanbing Yu, James A. Jones, and Mary Jean Harrold. 2008. An Empirical Study of the Effects of Test-Suite Reduction on Fault Localization. In Proceedings of the 13th International Conference on Software Engineering - ICSE ’08. New York, New York, USA: ACM Press. https://doi.org/10.1145/1368088.1368116.Google ScholarDigital Library
- Chris Parnin, and Alessandro Orso. 2011. Are Automated Debugging Techniques Actually Helping Programmers? In Proceedings of the 2011 International Symposium on Software Testing and Analysis - ISSTA ’11. New York, New York, USA: ACM Press. https://doi.org/10.1145/2001420.2001445.Google ScholarDigital Library
- James A. Jones, Mary Jean Harrold, and John Stasko. 2002. “Visualization of Test Information to Assist Fault Localization.” In Proceedings of the 24th International Conference on Software Engineering - ICSE ’02. New York, New York, USA: ACM Press. https://doi.org/10.1145/581339.581397.Google ScholarDigital Library
- Wes Masri, and Rawad Abou Assi. 2014. “Prevalence of Coincidental Correctness and Mitigation of Its Impact on Fault Localization.” ACM Transactions on Software Engineering and Methodology 23 (1): 1–28. https://doi.org/10.1145/2559932.Google ScholarDigital Library
- Yong Wang, Zhiqiu Huang, Rongcun Wang, and Qiao Yu. 2018. “Spectrum-Based Fault Localization Using Fault Triggering Model to Refine Fault Ranking List.” IEICE Transactions on Information and Systems E101.D (10): 2436–46. https://doi.org/10.1587/transinf.2017edp7386.Google ScholarCross Ref
- Shounak Roychowdhury, and Sarfraz Khurshid. 2011. “Software Fault Localization Using Feature Selection.” In Proceedings of the International Workshop on Machine Learning Technologies in Software Engineering - MALETS ’11. New York, New York, USA: ACM Press. https://doi.org/10.1145/2070821.2070823.Google ScholarDigital Library
- Ting Shu, Lei Wang, and Jinsong Xia. 2017. “Fault Localization Using a Failed Execution Slice.” In 2017 International Conference on Software Analysis, Testing and Evolution (SATE). IEEE. https://doi.org/10.1109/sate.2017.13.Google ScholarCross Ref
- Jeongho Kim, Jonghee Park and Eunseok Lee. 2016. “A New Spectrum-based Fault Localization With the Technique of Test Case Optimization.” J. Inf. Sci. Eng. 32 (2016): 177-196.Google Scholar
- Horvath, Ferenc, Victor Schnepper Lacerda, Arpad Beszedes, Laszlo Vidacs, and Tibor Gyimothy. 2019. “A New Interactive Fault Localization Method with Context Aware User Feedback.” In 2019 IEEE 1st International Workshop on Intelligent Bug Fixing (IBF). IEEE. https://doi.org/10.1109/ibf.2019.8665415.Google ScholarCross Ref
- A. Zakari, S. P. Lee, and I. A. T. Hashem, “A single fault localization technique based on failed test input,” Array, vol. 3–4, no. 100008, p. 100008, 2019.Google Scholar
- D. Ghosh and J. Singh, “Spectrum-based multi-fault localization using Chaotic Genetic Algorithm,” Inf. Softw. Technol., vol. 133, no. 106512, p. 106512, 2021.Google Scholar
- Y. Li, S. Wang, and T. Nguyen, “Fault localization with code coverage representation learning,” in 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), 2021.Google ScholarDigital Library
Index Terms
- Effective Lightweight Software Fault Localization based on Test Suite Optimization
Recommendations
Impacts of Test Suite's Class Imbalance on Spectrum-Based Fault Localization Techniques
QSIC '13: Proceedings of the 2013 13th International Conference on Quality SoftwareSpectrum-based fault localization (SBFL) uses the execution results of test cases to debug. There are two types of SBFL techniques: one using conventional slices, and the other using metamorphic slices. This paper investigates the ratio between non-...
Test case selection using multi-criteria optimization for effective fault localization
As spectra-based fault localization techniques report suspicious statements by analyzing the coverage of test cases, the effectiveness of the results is highly dependent on the composition of test suites. This paper proposes an approach for selecting a ...
A test-suite diagnosability metric for spectrum-based fault localization approaches
ICSE '17: Proceedings of the 39th International Conference on Software EngineeringCurrent metrics for assessing the adequacy of a test-suite plainly focus on the number of components (be it lines, branches, paths) covered by the suite, but do not explicitly check how the tests actually exercise these components and whether they ...
Comments