Abstract
Debugging is a heavy task in software development. Computer-assisted debugging is expected to reduce these costs. Spectrum-based Fault Localization (SBFL) is one of the most actively studied computer-assisted debugging techniques. SBFL aims to identify the location of faulty code elements based on the execution paths of tests. Previous research reports that the accuracy of SBFL is affected by test types, such as flaky tests. Our research focuses on exceptional behavior tests to reveal the impact of such tests on SBFL. Since separating exceptional handling from normal control flow enables developers to increase program robustness, we think the execution paths of exceptional behavior tests are different from the ones of normal control flow tests, which means that the differences significantly affect the accuracy of SBFL. In this study, we investigated the accuracy of SBFL on two types of faults: faults that occurred in the real software development process and artificially generated faults. As a result, our study reveals that SBFL tends to be more accurate when all failing tests are exceptional behavior tests than when failing tests include no exceptional behavior tests.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abreu, R., Zoeteweij, P., Golsteijn, R., van Gemund, A.J.: A practical evaluation of spectrum-based fault localization. J. Syst. Softw. 82(11), 1780–1792 (2009)
Ali, S., Andrews, J.H., Dhandapani, T., Wang, W.: Evaluating the accuracy of fault localization techniques. In: Proceedings of International Conference on Automated Software Engineering, pp. 76–87 (2009)
Dalton, F., Ribeiro, M., Pinto, G., Fernandes, L., Gheyi, R., Fonseca, B.: Is exceptional behavior testing an exception? an empirical assessment using java automated tests. In: Proceedings of International Conference on Evaluation and Assessment in Software Engineering, pp. 170–179 (2020)
Garcia, A.F., Rubira, C.M., Romanovsky, A., Xu, J.: A comparative study of exception handling mechanisms for building dependable object-oriented software. J. Syst. Softw. 59(2), 197–222 (2001)
Higo, Y., et al.: kGenProg: a high-performance, high-extensibility and high-portability APR system. In: Proceedings of Asia-Pacific Software Engineering Conference, pp. 697–698 (2018)
Jin, W., Orso, A.: BugRedux: reproducing field failures for in-house debugging. In: Proceedings of International Conference on Software Engineering, pp. 474–484 (2012)
Just, R., Jalali, D., Ernst, M.D.: Defects4j: a database of existing faults to enable controlled testing studies for java programs. In: Proceedings of International Symposium on Software Testing and Analysis, pp. 437–440 (2014)
Kochhar, P.S., Xia, X., Lo, D., Li, S.: Practitioners’ expectations on automated fault localization. In: Proceedings of International Symposium on Software Testing and Analysis, pp. 165–176 (2016)
Lei, Y., Xie, H., Zhang, T., Yan, M., Xu, Z., Sun, C.: Feature-FL: feature-based fault localization. IEEE Trans. Reliab. 71(1), 264–283 (2022)
Li, X., Li, W., Zhang, Y., Zhang, L.: Deepfl: integrating multiple fault diagnosis dimensions for deep fault localization. In: Proceedings of International Symposium on Software Testing and Analysis, pp. 169–180 (2019)
Liu, C., Yan, X., Fei, L., Han, J., Midkiff, S.P.: SOBER: statistical model-based bug localization. In: Proceedings of European Software Engineering Conference Held Jointly with International Symposium on Foundations of Software Engineering, pp. 286–295 (2005)
Lou, Y., et al.: Can automated program repair refine fault localization? a unified debugging approach. In: Proceedings of International Symposium on Software Testing and Analysis, pp. 75–87 (2020)
Martinez, M., Martin, M.: Ultra-large repair search space with automatically mined templates: the cardumen mode of astor. In: Proceedings of International Symposium on Search Based Software Engineering, pp. 65–86 (2017)
Pearson, S., et al.: Evaluating and improving fault localization. In: Proceedings of International Conference on Software Engineering, pp. 609–620 (2017)
Qi, Y., Mao, X., Lei, Y., Wang, C.: Using automated program repair for evaluating the effectiveness of fault localization techniques. In: Proceedings of International Symposium on Software Testing and Analysis, pp. 191–201 (2013)
Qin, Y., Wang, S., Liu, K., Mao, X., Bissyandé, T.F.: On the impact of flaky tests in automated program repair. In: International Conference on Software Analysis, Evolution and Reengineering, pp. 295–306 (2021)
Tassey, G.: The economic impacts of inadequate infrastructure for software testing (2002)
Wong, W.E., Gao, R., Li, Y., Abreu, R., Wotawa, F.: A survey on software fault localization. IEEE Trans. Softw. Eng. 42(8), 707–740 (2016)
Yuan, Y., Banzhaf, W.: Arja: automated repair of java programs via multi-objective genetic programming. Trans. Softw. Eng. 46(10), 1040–1067 (2020)
Acknowledgements
This research was supported by JSPS KAKENHI Japan (JP20H04166, JP21K18302, JP21K11829, JP21H04877, JP22H03567, JP22K11985)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yoshioka, H., Higo, Y., Matsumoto, S., Kusumoto, S., Itoh, S., Huyen, P.T.T. (2024). Do Exceptional Behavior Tests Matter on Spectrum-Based Fault Localization?. In: Kadgien, R., Jedlitschka, A., Janes, A., Lenarduzzi, V., Li, X. (eds) Product-Focused Software Process Improvement. PROFES 2023. Lecture Notes in Computer Science, vol 14483. Springer, Cham. https://doi.org/10.1007/978-3-031-49266-2_28
Download citation
DOI: https://doi.org/10.1007/978-3-031-49266-2_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-49265-5
Online ISBN: 978-3-031-49266-2
eBook Packages: Computer ScienceComputer Science (R0)