Skip to main content
SpringerLink
Log in

Cost-effective testing based fault localization with distance based test-suite reduction

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

The aim of testing based fault localization (TBFL) involves improving the efficiency of program debugging by providing developers with a guide of ranked list of suspicious statements. However, collection of testing information of the whole original test-suite is excessively expensive or even infeasible for developers to conduct TBFL. Traditional test-suite reduction (TSR) techniques are utilized to reduce the size of test-suite. However, they entail a time-consuming process of whole testing information collection. In this study, the distance based test-suite reduction (DTSR) technique is proposed. As opposed to the whole testing information, the distances among the test cases are used to guide the process of test-suite reduction in DTSR. Hence, it is only necessary to collect the testing information for a portion of the test cases for TSR and TBFL. The investigation on the Siemens and SIR benchmarks reveals that DTSR can effectively reduce the size of the given test-suite as well as the time cost of TBFL. Additionally, the fault locating effectiveness of DTSR results is close to that when the whole test-suite is used.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Wong W E, Gao R Z, Li Y H, et al. A survey on software fault localization. IEEE Trans Softw Eng, 2016, 42: 707–740

    Article  Google Scholar 

  2. Xie X Y, Chen T Y, Kuo F C, et al. A theoretical analysis of the risk evaluation formulas for spectrum-based fault localization. ACM Trans Softw Eng Meth, 2013, 22: 402–418

    Article  Google Scholar 

  3. Ledru Y, Petrenko A, Boroday S, et al. Prioritizing test cases with string distances. Automat Softw Eng, 2012, 19: 65–95

    Article  Google Scholar 

  4. Hao D, Xie T, Zhang L, et al. Test input reduction for result inspection to facilitate fault localization. Automat Softw Eng, 2010, 17: 5–31

    Article  Google Scholar 

  5. Gong L, Lo D, Jiang L X, et al. Diversity maximization speedup for fault localization. In: Proceedings of the 27th International Conference on Automated Software Engineering, Essen, 2012. 30–39

    Google Scholar 

  6. Yu Y B, Jones J A, Harrold M J. An empirical study of the effects of test-suite reduction on fault localization. In: Proceedings of the 30th International Conference on Software Engineering, Leipzig, 2008. 201–210

    Google Scholar 

  7. Shi A, Yung T, Gyori A, et al. Comparing and combining test-suite reduction and regression test selection. In: Proceedings of the 10th Joint Meeting on Foundations of Software Engineering, Bergamo, 2015. 237–247

    Google Scholar 

  8. Jones J A, Harrold M J, Stasko J. Visualization of test information to assist fault localization. In: Proceedings of the 24th International Conference on Software Engineering, Orlando, 2002. 467–477

    Google Scholar 

  9. Ju X L, Jiang S J, Chen X, et al. HSFal: effective fault localization using hybrid spectrum of full slices and execution slices. J Syst Softw, 2014, 90: 3–17

    Article  Google Scholar 

  10. Baudry B. Improving test suites for efficient fault localization. In: Proceedings of the 28th International Conference on Software Engineering, Shanghai, 2006. 82–91

    Google Scholar 

  11. Rothermel G, Harrold M J. Empirical studies of a safe regression test selection technique. IEEE Trans Softw Eng, 1998, 24: 401–419

    Article  Google Scholar 

  12. Wang R C, Qu B B, Lu Y S. Empirical study of the effects of different profiles on regression test case reduction. IET Softw, 2015, 9: 29–38

    Article  Google Scholar 

  13. Jiang B, Zhai K, Chan W K, et al. On the adoption of MC/DC and control-flow adequacy for a tight integration of program testing and statistical fault localization. Inform Softw Tech, 2013, 55: 897–917

    Article  Google Scholar 

  14. Hamming R W. Error detecting and error correcting codes. BELL Labs Tech J, 1950, 29: 147–160

    Article  MathSciNet  Google Scholar 

  15. Levenshtein V I. Binary codes capable of correcting deletions, insertions and reversals. Dokl Akad Nauk Sssr, 1965, 10: 707–710

    MathSciNet  MATH  Google Scholar 

  16. Chen T Y, Poon P L, Tang S F, et al. On the identification of categories and choices for specification-based test case generation. Inform Softw Tech, 2004, 46: 887–898

    Article  Google Scholar 

  17. Zhang X Y, Towey D, Chen T Y, et al. Using partition information to prioritize test cases for fault localization. In: Proceedings of the 39th Annual Computer Software and Applications Conference, Taichung, 2002. 121–126

    Google Scholar 

  18. Ostrand T J, Balcer M J. The category-partition method for specifying and generating fuctional tests. Commun ACM, 1988, 31: 676–686

    Article  Google Scholar 

  19. Santelices R, Jones J A, Yu Y B, et al. Lightweight fault-localization using multiple coverage types. In: Proceedings of the 31st International Conference on Software Engineering, Vancouver, 2009. 56–66

    Google Scholar 

  20. Do H, Elbaum S, Rothermel G. Supporting controlled experimentation with testing techniques: an infrastructure and its potential impact. Empir Softw Eng, 2005, 10: 405–435

    Article  Google Scholar 

  21. Masri W, Assi R A. Prevalence of coincidental correctness and mitigation of its impact on fault localization. ACM Trans Softw Eng Meth, 2014, 23: 294–304

    Article  Google Scholar 

  22. Zhang Z Y, Chen Z Y, Gao R Z, et al. An empirical study on constraint optimization techniques for test generation. Sci China Inf Sci, 2017, 60: 012105

    Article  Google Scholar 

  23. Fang C R, Chen Z Y, Xu B W. Comparing logic coverage criteria on test case prioritization. Sci China Inf Sci, 2012, 55: 2826–2840

    Article  Google Scholar 

  24. Jiang B, Zhang Z Y, Chan W K, et al. Adaptive random test case prioritization. In: Proceedings of the 24th International Conference on Automated Software Engineering, Auckland, 2009. 233–244

    Google Scholar 

  25. Zhang X F, Xie X Y, Chen T Y. Test case prioritization using adaptive random sequence with category-partition-based distance. In: Proceedings of the 2016 International Conference on Software Quality, Reliability and Security, Vienna, 2016. 374–385

    Chapter  Google Scholar 

  26. Chen L, Wang Z Y, Xu L, et al. Test case prioritization for web service regression testing. In: Proceedings of the 5th International Symposium on Service-Oriented System Engineering, Nanjing, 2010. 173–178

    Google Scholar 

  27. Rothermel G, Harrold M J. Analyzing regression test selection techniques. IEEE Trans Softw Eng, 1996, 22: 529–551

    Article  Google Scholar 

  28. Wong WE, Horgan J R, London S, et al. Effect of test set minimization on fault detection effectiveness. In: Proceedings of the 17th International Conference on Software Engineering, Washington, 1995. 41–50

    Google Scholar 

  29. Rothermel G, Harrold M J, Ronne J V. Empirical studies of test-suite reduction. Softw Test Verif Rel, 2002, 12: 219–249

    Article  Google Scholar 

  30. Wang X Y, Jiang S J, Ju X L, et al. Mitigating the dependence confounding effect for effective predicate-based statistical fault localization. In: Proceedings of the 39th Annual Computer Software and Applications Conference, Taichung, 2015. 105–114

    Google Scholar 

  31. Vidacs L, Beszedes A, Tengeri D, et al. Test suite reduction for fault detection and localization: a combined approach. In: Proceedings of the 2014 IEEE Conference on Software Maintenance, Reengineering and Reverse Engineering, Antwerp, 2014. 204–213

    Google Scholar 

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Grant Nos. 61673384, 61502497, 61562015), Guangxi Key Laboratory of Trusted Software (Grant Nos. kx201609, kx201532), Scientific Research Innovation Project for Graduate Students of Jiangsu Province (Grant No. KYLX 1390), Science and Technology Program of Xuzhou (Grant No. KC15SM051), and China Postdoctoral Science Foundation (Grant No. 2015M581887).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, 221116, China

    Xingya Wang, Shujuan Jiang, Pengfei Gao, Rongcun Wang & Yanmei Zhang

  2. School of Computer Science and Technology, Nantong University, Nantong, 226019, China

    Xiaolin Ju

  3. Guangxi Key Laboratory of Trusted Software, Guilin University of Electronic Technology, Guilin, 541004, China

    Xiaolin Ju & Yanmei Zhang

Authors
  1. Xingya Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shujuan Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pengfei Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaolin Ju
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rongcun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yanmei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shujuan Jiang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Jiang, S., Gao, P. et al. Cost-effective testing based fault localization with distance based test-suite reduction. Sci. China Inf. Sci. 60, 092112 (2017). https://doi.org/10.1007/s11432-016-9057-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-016-9057-8

Keywords

Search

Navigation