skip to main content
10.1145/3167132.3167337acmconferencesArticle/Chapter ViewAbstractPublication PagessacConference Proceedingsconference-collections
research-article

On the use of sequence mining within spectrum based fault localisation

Published: 09 April 2018 Publication History

Abstract

Spectrum based fault localisation is a widely studied class of heuristics for locating faults within a software program. Unfortunately, the current state of the art ignores the inherent dependencies between the methods leading up to the fault, hence having a limited diagnostic accuracy. In this paper we present a variant of spectrum based fault localisation, which leverages series of method calls by means of sequence mining. We validate our variant (we refer to it as sequenced spectrum analysis) on the Defects4J benchmark, demonstrating that sequenced spectrum analysis gains a 12% points improvement against the state of the art.

References

[1]
Rui Abreu, Peter Zoeteweij, and Arjan J. C. van Gemund. 2009. Spectrum-Based Multiple Fault Localization. In Proceedings of the 2009 IEEE/ACM International Conference on Automated Software Engineering (ASE '09). IEEE Computer Society, Washington, DC, USA, 88--99.
[2]
Rui Abreu, Peter Zoeteweij, Rob Golsteijn, and Arjan J. C. van Gemund. 2009. A Practical Evaluation of Spectrum-based Fault Localization. Journal of Systems and Software 82, 11 (Nov. 2009), 1780--1792.
[3]
Rui Abreu, Peter Zoeteweij, and Arjan J. C. van Gemund. 2007. On the Accuracy of Spectrum-based Fault Localization. In Proceedings of the Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION '07). IEEE Computer Society, Washington, DC, USA, 89--98. http://dl.acm.org/citation.cfm?id=1308173.1308264
[4]
Tien-Duy B. Le, David Lo, Claire Le Goues, and Lars Grunske. 2016. A Learning-to-rank Based Fault Localization Approach Using Likely Invariants. In Proceedings of the 25th International Symposium on Software Testing and Analysis (ISSTA 2016). ACM, New York, NY, USA, 177--188.
[5]
Robert V. Binder. 1999. Testing Object-oriented Systems: Models, Patterns, and Tools. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA.
[6]
José Campos, André Riboira, Alexandre Perez, and Rui Abreu. 2012. GZoltar: An Eclipse Plug-in for Testing and Debugging. In Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering (ASE 2012). ACM, New York, NY, USA, 378--381.
[7]
Mike Y. Chen, Emre Kiciman, Eugene Fratkin, Armando Fox, and Eric Brewer. 2002. Pinpoint: Problem Determination in Large, Dynamic Internet Services. In Proceedings of the 2002 International Conference on Dependable Systems and Networks (DSN '02). IEEE Computer Society, Washington, DC, USA, 595--604. http://dl.acm.org/citation.cfm?id=647883.738238
[8]
Boris Cule, Nikolaj Tatti, and Bart Goethals. 2014. Marbles: Mining association rules buried in long event sequences. Statistical Analysis and Data Mining: The ASA Data Science Journal 7, 2 (2014), 93--110.
[9]
Valentin Dallmeier, Christian Lindig, and Andreas Zeller. 2005. Lightweight Defect Localization for Java. In Proceedings of the 19th European Conference on Object-Oriented Programming (ECOOP'05). Springer-Verlag, Berlin, Heidelberg, 528--550.
[10]
Mark Gabel and Zhendong Su. 2010. Online Inference and Enforcement of Temporal Properties. In Proceedings of the 32Nd ACM/IEEE International Conference on Software Engineering - Volume 1 (ICSE '10). ACM, New York, NY, USA, 15--24.
[11]
James A. Jones, Mary Jean Harrold, and John Stasko. 2002. Visualization of Test Information to Assist Fault Localization. In Proceedings of the 24th International Conference on Software Engineering (ICSE '02). ACM, New York, NY, USA, 467--477.
[12]
Jonathan Aldrich Joshua Sushine, James D. Herbsleb. 2015. Searching the State Space: A Qualitative Study of API Protocol Usability. In Proceedings of the International Conference on Program Comprehension (ICPC).
[13]
René Just, Darioush Jalali, and Michael D. Ernst. 2014. Defects4J: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs. In Proceedings of the 2014 International Symposium on Software Testing and Analysis (ISSTA 2014). ACM, New York, NY, USA, 437--440.
[14]
Pavneet Singh Kochhar, Xin Xia, David Lo, and Shanping Li. 2016. Practitioners' Expectations on Automated Fault Localization. In Proceedings of the 25th International Symposium on Software Testing and Analysis (ISSTA 2016). ACM, New York, NY, USA, 165--176.
[15]
Gulsher Laghari, Alessandro Murgia, and Serge Demeyer. 2015. Localising Faults in Test Execution Traces. In Proceedings of the 14th International Workshop on Principles of Software Evolution (IWPSE 2015). ACM, New York, NY, USA, 1--8.
[16]
Gulsher Laghari, Alessandro Murgia, and Serge Demeyer. 2016. Fine-tuning Spectrum Based Fault Localisation with Frequent Method Item Sets. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering (ASE 2016). ACM, New York, NY, USA, 274--285.
[17]
Tien-Duy B. Le, David Lo, and Ferdian Thung. 2015. Should I Follow This Fault Localization Tool's Output? Empirical Softw. Engg. 20, 5 (Oct. 2015), 1237--1274.
[18]
Choonghwan Lee, Feng Chen, and Grigore Roşu. 2011. Mining Parametric Specifications. In Proceedings of the 33rd International Conference on Software Engineering (ICSE '11). ACM, New York, NY, USA, 591--600.
[19]
Lucia, D. Lo, Lingxiao Jiang, and A. Budi. 2010. Comprehensive evaluation of association measures for fault localization. In Software Maintenance (ICSM), 2010 IEEE International Conference on. 1--10.
[20]
Lucia, David Lo, and Xin Xia. 2014. Fusion Fault Localizers. In Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering (ASE '14). ACM, New York, NY, USA, 127--138.
[21]
Lee Naish, Hua Jie Lee, and Kotagiri Ramamohanarao. 2011. A Model for Spectra-based Software Diagnosis. ACM Trans. Softw. Eng. Methodol. 20, 3, Article 11 (Aug. 2011), 32 pages.
[22]
Chris Parnin and Alessandro Orso. 2011. Are Automated Debugging Techniques Actually Helping Programmers?. In Proceedings of the 2011 International Symposium on Software Testing and Analysis (ISSTA '11). ACM, New York, NY, USA, 199--209.
[23]
Spencer Pearson, José Campos, René Just, Gordon Fraser, Rui Abreu, Michael D. Ernst, Deric Pang, and Benjamin Keller. 2017. Evaluating and improving fault localization. In ICSE 2017, Proceedings of the 39th International Conference on Software Engineering. Buenos Aires, Argentina.
[24]
Michael Pradel and Thomas R. Gross. 2009. Automatic Generation of Object Usage Specifications from Large Method Traces. In Proceedings of the 2009 IEEE/ACM International Conference on Automated Software Engineering (ASE '09). IEEE Computer Society, Washington, DC, USA, 371--382.
[25]
Martin P. Robillard, Eric Bodden, David Kawrykow, Mira Mezini, and Tristan Ratchford. 2013. Automated API Property Inference Techniques. IEEE Trans. Softw. Eng. 39, 5 (May 2013), 613--637.
[26]
Per Runeson and Martin Höst. 2009. Guidelines for conducting and reporting case study research in software engineering. Empirical Softw. Engineering 14, 2 (2009), 131--164.
[27]
R.K. Saha, M. Lease, S. Khurshid, and D.E. Perry. 2013. Improving bug localization using structured information retrieval. In Automated Software Engineering (ASE), 2013 IEEE/ACM 28th International Conference on. 345--355.
[28]
Alberto González Sánchez. 2007. Automatic Error Detection Techniques Based on Dynamic Invariants. Master's thesis. Delft University of Technology, the Netherlands. http://swerl.tudelft.nl/twiki/pub/Main/AlbertoGonzalezSanchez/thesis_gonzalez.pdf
[29]
F. Steimann and M. Frenkel. 2012. Improving Coverage-Based Localization of Multiple Faults Using Algorithms from Integer Linear Programming. In Software Reliability Engineering (ISSRE), 2012 IEEE 23rd International Symposium on. 121--130.
[30]
Friedrich Steimann, Marcus Frenkel, and Rui Abreu. 2013. Threats to the Validity and Value of Empirical Assessments of the Accuracy of Coverage-based Fault Locators. In Proceedings of the 2013 International Symposium on Software Testing and Analysis (ISSTA 2013). ACM, New York, NY, USA, 314--324.
[31]
Jingxuan Tu, Lin Chen, Yuming Zhou, Jianjun Zhao, and Baowen Xu. 2012. Leveraging Method Call Anomalies to Improve the Effectiveness of Spectrum-Based Fault Localization Techniques for Object-Oriented Programs. In Proceedings of the 2012 12th International Conference on Quality Software (QSIC '12). IEEE Computer Society, Washington, DC, USA, 1--8.
[32]
Andrzej Wasylkowski, Andreas Zeller, and Christian Lindig. 2007. Detecting Object Usage Anomalies. In Proceedings of the the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on The Foundations of Software Engineering (ESEC-FSE '07). ACM, New York, NY, USA, 35--44.
[33]
W. Eric Wong, Ruizhi Gao, Yihao Li, Rui Abreu, and Franz Wotawa. 2016. A Survey on Software Fault Localization. IEEE Transactions on Software Engineering 42, 8 (Aug. 2016), 707--740.
[34]
W. Eric Wong, Yu Qi, Lei Zhao, and Kai-Yuan Cai. 2007. Effective Fault Localization Using Code Coverage. In Proceedings of the 31st Annual International Computer Software and Applications Conference - Volume 01 (COMPSAC '07). IEEE Computer Society, Washington, DC, USA, 449--456.
[35]
J. Xuan and M. Monperrus. 2014. Learning to Combine Multiple Ranking Metrics for Fault Localization. In Software Maintenance and Evolution (ICSME), 2014 IEEE International Conference on. 191--200.
[36]
Cemal Yilmaz, Amit Paradkar, and Clay Williams. 2008. Time will tell: fault localization using time spectra. In In ICSE Conference Proceedings. 81--90.
[37]
Robert K. Yin. 2002. Case Study Research: Design and Methods, 3 edition. Sage Publications.
[38]
Shin Yoo. 2012. Evolving Human Competitive Spectra-based Fault Localisation Techniques. In Proceedings of the 4th International Conference on Search Based Software Engineering (SSBSE'12). Springer-Verlag, Berlin, Heidelberg, 244--258.
[39]
Andreas Zeller. 2005. Why Programs Fail: A Guide to Systematic Debugging. Morgan Kaufmann.
[40]
Andreas Zeller, Thomas Zimmermann, and Christian Bird. 2011. Failure is a Four-letter Word: A Parody in Empirical Research. In Proceedings of the 7th International Conference on Predictive Models in Software Engineering (Promise '11). ACM, New York, NY, USA, Article 5, 7 pages.

Cited By

View all
  • (2024)Demystifying Faulty Code: Step-by-Step Reasoning for Explainable Fault Localization2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER60148.2024.00064(568-579)Online publication date: 12-Mar-2024
  • (2023)Fault Localization Using TrustRank AlgorithmApplied Sciences10.3390/app13221234413:22(12344)Online publication date: 15-Nov-2023
  • (2022)XAI4FLProceedings of the 30th IEEE/ACM International Conference on Program Comprehension10.1145/3524610.3527902(499-510)Online publication date: 16-May-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SAC '18: Proceedings of the 33rd Annual ACM Symposium on Applied Computing
April 2018
2327 pages
ISBN:9781450351911
DOI:10.1145/3167132
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 April 2018

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. debugging
  2. method call sequences
  3. spectrum based fault localisation

Qualifiers

  • Research-article

Conference

SAC 2018
Sponsor:
SAC 2018: Symposium on Applied Computing
April 9 - 13, 2018
Pau, France

Acceptance Rates

Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Upcoming Conference

SAC '25
The 40th ACM/SIGAPP Symposium on Applied Computing
March 31 - April 4, 2025
Catania , Italy

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)4
  • Downloads (Last 6 weeks)1
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Demystifying Faulty Code: Step-by-Step Reasoning for Explainable Fault Localization2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER60148.2024.00064(568-579)Online publication date: 12-Mar-2024
  • (2023)Fault Localization Using TrustRank AlgorithmApplied Sciences10.3390/app13221234413:22(12344)Online publication date: 15-Nov-2023
  • (2022)XAI4FLProceedings of the 30th IEEE/ACM International Conference on Program Comprehension10.1145/3524610.3527902(499-510)Online publication date: 16-May-2022
  • (2022)A Survey of Challenges in Spectrum-Based Software Fault LocalizationIEEE Access10.1109/ACCESS.2022.314407910(10618-10639)Online publication date: 2022
  • (2021)A Framework for Improving Fault Localization Effectiveness Based on Fuzzy Expert SystemIEEE Access10.1109/ACCESS.2021.30868789(82577-82596)Online publication date: 2021
  • (2021)Fault centrality: boosting spectrum-based fault localization via local influence calculationApplied Intelligence10.1007/s10489-021-02822-452:7(7113-7135)Online publication date: 20-Sep-2021
  • (2020)Enhancing Spectrum-Based Fault Localization Using Fault Influence PropagationIEEE Access10.1109/ACCESS.2020.29651398(18497-18513)Online publication date: 2020

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media