Skip to main content

Searching for Multi-fault Programs in Defects4J

  • Conference paper
  • First Online:
Search-Based Software Engineering (SSBSE 2021)

Abstract

Defects4J has enabled numerous software testing and debugging research work since its introduction. A large part of its contribution, and the resulting popularity, lies in the clear separation and distillation of the root cause of each individual test failure based on careful manual analysis, which in turn allowed researchers to easily study individual faults in isolation. However, in a realistic debugging scenario, multiple faults can coexist and affect test results collectively. Study of automated debugging techniques for these situations, such as failure clustering or fault localisation for multiple faults, would significantly benefit from a reliable benchmark of multiple, coexisting faults. We search for versions of Defects4J subjects that contain multiple faults, by iteratively transplanting fault-revealing test cases across Defects4J versions. Out of 326 studied versions of Defects4J subjects, we report that over 95% (311 versions) actually contain from two to 24 faults. We hope that the extended, multi-fault Defects4J can provide a platform for future research of testing and debugging techniques for multi-fault programs.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/coinse/Defects4J-multifault.

  2. 2.

    http://program-repair.org/defects4j-dissection/#!/bug/Math/5.

  3. 3.

    defects4j checkout -p Math -v 6b -w \(\texttt {{<}dir{>}}\) checks out \(B_{Math-6}\) into \(\texttt {{<}dir{>}}\).

  4. 4.

    https://github.com/javaparser/javaparser.

  5. 5.

    Defects4J Bug IDs: Lang 1-65, Chart 1-26, Math 1-106, Time 1-27, and Closure 1-106. Note that Lang-2, Time-21, Closure-63 and -93 are excluded since they are either no longer reproducible under Java 8 or the duplicate bugs.

References

  1. Abreu, R., Zoeteweij, P., Van Gemund, A.J.: Spectrum-based multiple fault localization. In: 2009 IEEE/ACM International Conference on Automated Software Engineering, pp. 88–99. IEEE (2009)

    Google Scholar 

  2. Le, T.D.B., Lo, D., Le Goues, C., Grunske, L.: A learning-to-rank based fault localization approach using likely invariants. In: Proceedings of the 25th International Symposium on Software Testing and Analysis, pp. 177–188 (2016)

    Google Scholar 

  3. Canfora, G., Ceccarelli, M., Cerulo, L., Di Penta, M.: How long does a bug survive? An empirical study. In: Proceedings of Working Conference on Reverse Engineering, pp. 191–200. IEEE (2011)

    Google Scholar 

  4. Chen, Z., Kommrusch, S.J., Tufano, M., Pouchet, L.N., Poshyvanyk, D., Monperrus, M.: Sequencer: sequence-to-sequence learning for end-to-end program repair. IEEE Trans. Softw. Eng. (2019)

    Google Scholar 

  5. Dang, Y., Wu, R., Zhang, H., Zhang, D., Nobel, P.: ReBucket: a method for clustering duplicate crash reports based on call stack similarity. In: 2012 34th International Conference on Software Engineering (ICSE), pp. 1084–1093. IEEE (2012)

    Google Scholar 

  6. Ghosh, D., Singh, J.: Spectrum-based multi-fault localization using chaotic genetic algorithm. Inf. Softw. Technol. 133, 106512 (2021)

    Article  Google Scholar 

  7. Golagha, M., Lehnhoff, C., Pretschner, A., Ilmberger, H.: Failure clustering without coverage. In: Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 134–145 (2019)

    Google Scholar 

  8. Jones, J.A., Bowring, J.F., Harrold, M.J.: Debugging in parallel. In: Proceedings of the International Symposium on Software Testing and Analysis, pp. 16–26 (2007)

    Google Scholar 

  9. Just, R., Jalali, D., Ernst, M.D.: Defects4J: a database of existing faults to enable controlled testing studies for Java programs. In: Proceedings of the 2014 International Symposium on Software Testing and Analysis, pp. 437–440 (2014)

    Google Scholar 

  10. Kim, S., Whitehead Jr., E.J.: How long did it take to fix bugs? In: Proceedings of the International Workshop on Mining Software Repositories, pp. 173–174 (2006)

    Google Scholar 

  11. Koyuncu, A., et al.: FixMiner: mining relevant fix patterns for automated program repair. Empirical Softw. Eng. 25, 1–45 (2020)

    Article  Google Scholar 

  12. Laghari, G., Murgia, A., Demeyer, S.: Fine-tuning spectrum based fault localisation with frequent method item sets. In: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, pp. 274–285 (2016)

    Google Scholar 

  13. Li, X., Li, W., Zhang, Y., Zhang, L.: DeepFL: integrating multiple fault diagnosis dimensions for deep fault localization. In: Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 169–180 (2019)

    Google Scholar 

  14. Li, X., d’Amorim, M., Orso, A.: Iterative user-driven fault localization. In: Bloem, R., Arbel, E. (eds.) HVC 2016. LNCS, vol. 10028, pp. 82–98. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-49052-6_6

    Chapter  Google Scholar 

  15. Liu, K., Koyuncu, A., Kim, D., Bissyandé, T.F.: TBar: revisiting template-based automated program repair. In: Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 31–42 (2019)

    Google Scholar 

  16. Saha, S., et al.: Harnessing evolution for multi-hunk program repair. In: International Conference on Software Engineering, pp. 13–24 (2019)

    Google Scholar 

  17. Sohn, J., Yoo, S.: FLUCCS: using code and change metrics to improve fault localization. In: Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 273–283 (2017)

    Google Scholar 

  18. Xia, X., Bao, L., Lo, D., Li, S.: “Automated debugging considered harmful” considered harmful: a user study revisiting the usefulness of spectra-based fault localization techniques with professionals using real bugs from large systems. In: International Conference on Software Maintenance and Evolution, pp. 267–278 (2016)

    Google Scholar 

  19. Zakari, A., Lee, S.P., Abreu, R., Ahmed, B.H., Rasheed, R.A.: Multiple fault localization of software programs: a systematic literature review. Inf. Softw. Technol. 124, 106312 (2020)

    Article  Google Scholar 

  20. Zhang, M., Li, X., Zhang, L., Khurshid, S.: Boosting spectrum-based fault localization using pagerank. In: Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 261–272 (2017)

    Google Scholar 

  21. Zheng, Y., Wang, Z., Fan, X., Chen, X., Yang, Z.: Localizing multiple software faults based on evolution algorithm. J. Syst. Softw. 139, 107–123 (2018)

    Article  Google Scholar 

Download references

Acknowledgement

This work is supported by National Research Foundation of Korea (NRF) Grant (NRF-2020R1A2C1013629), Institute for Information & communications Technology Promotion grant funded by the Korean government (MSIT) (No.2021-0-01001), and Samsung Electronics (Grant No. IO201210-07969-01).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Shin Yoo .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

An, G., Yoon, J., Yoo, S. (2021). Searching for Multi-fault Programs in Defects4J. In: O'Reilly, UM., Devroey, X. (eds) Search-Based Software Engineering. SSBSE 2021. Lecture Notes in Computer Science(), vol 12914. Springer, Cham. https://doi.org/10.1007/978-3-030-88106-1_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-88106-1_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-88105-4

  • Online ISBN: 978-3-030-88106-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics