research-article

Influential Global and Local Contexts Guided Trace Representation for Fault Localization

Authors:
Zhuo Zhang

Guangzhou College of Commerce, Guangzhou, China

Guangzhou College of Commerce, Guangzhou, China

0000-0003-3243-1019
View Profile

,
Yan Lei

School of Big Data & Software Engineering, Chongqing University, Chongqing, China

School of Big Data & Software Engineering, Chongqing University, Chongqing, China

0000-0003-4504-6806
View Profile

,
Ting Su

Software Engineering Institute, East China Normal University, Shanghai, China

Software Engineering Institute, East China Normal University, Shanghai, China

0000-0003-1628-9796
View Profile

,
Meng Yan

School of Big Data & Software Engineering, Chongqing University, Chongqing, China

School of Big Data & Software Engineering, Chongqing University, Chongqing, China

0000-0002-9538-9121
View Profile

,
Xiaoguang Mao

College of Computer, National University of Defense Technology, Changsha, China

College of Computer, National University of Defense Technology, Changsha, China

0000-0003-4204-7424
View Profile

,
Yue Yu

College of Computer, National University of Defense Technology, Changsha, China

College of Computer, National University of Defense Technology, Changsha, China

0000-0002-9865-2212
View Profile

ACM Transactions on Software Engineering and Methodology Volume 32 Issue 3Article No.: 78pp 1–27https://doi.org/10.1145/3576043

Published:26 April 2023Publication History

ACM Transactions on Software Engineering and Methodology

Abstract

Trace data is critical for fault localization (FL) to analyze suspicious statements potentially responsible for a failure. However, existing trace representation meets its bottleneck mainly in two aspects: (1) the trace information of a statement is restricted to a local context (i.e., a test case) without the consideration of a global context (i.e., all test cases of a test suite); (2) it just uses the ‘occurrence’ for representation without strong FL semantics.

Thus, we propose UNITE: an inflUential coNtext-GuIded Trace rEpresentation, representing the trace from both global and local contexts with influential semantics for FL. UNITE embodies and implements two key ideas: (1) UNITE leverages the widely used weighting capability from local and global contexts of information retrieval to reflect how important a statement (a word) is to a test case (a document) in all test cases of a test suite (a collection), where a test case (a document) and all test cases of a test suite (a collection) represent local and global contexts respectively; (2) UNITE further elaborates the trace representation from ‘occurrence’ (weak semantics) to ‘influence’ (strong semantics) by combing program dependencies. The large-scale experiments on 12 FL techniques and 20 programs show that UNITE significantly improves FL effectiveness.

REFERENCES

[1] Abdi Hervé. 2007. The Bonferonni and Šidák corrections for multiple comparisons. Encyclopedia of Measurement and Statistics 3 (2007), 103–107.Google Scholar
[2] Agrawal Hiralal and Horgan Joseph R.. 1990. Dynamic program slicing. ACM SIGPLAN Notices 25, 6 (1990), 246–256.Google ScholarDigital Library
[3] Arcuri Andrea and Briand Lionel. 2011. A practical guide for using statistical tests to assess randomized algorithms in software engineering. In Proceedings of the 33rd International Conference on Software Engineering (ICSE’11). IEEE, 1–10.Google ScholarDigital Library
[4] Briand L. C, Labiche Y., and Liu Xuetao. 2007. Using machine learning to support debugging with tarantula. In Proceedings of the IEEE International Symposium on Software Reliability (ISSRE’07). 137–146.Google ScholarDigital Library
[5] Corder Gregory W. and Foreman Dale I.. 2010. Nonparametric Statistics for Non-Statisticians: A Step-by-Step Approach. Vol. 78. International Statistical Review, 451–452.Google Scholar
[6] Debroy Vidroha and Wong W. Eric. 2009. Insights on fault interference for programs with multiple bugs. In Proceedings of the 20th International Symposium on Software Reliability Engineering (ISSRE’09). IEEE, 165–174.Google ScholarDigital Library
[7] Debroy Vidroha, Wong W. Eric, Xu Xiaofeng, and Choi Byoungju. 2010. A grouping-based strategy to improve the effectiveness of fault localization techniques. In Proceedings of the 10th International Conference on Quality Software (QSIC’10). 13–22.Google ScholarDigital Library
[8] Fang Chunrong, Feng Yang, Shi Qingkai, Liu Zicong, Li Shuying, and Xu Baowen. 2017. Fault interference and coupling effect. In Proceedings of the 29th International Conference on Software Engineering and Knowledge Engineering (SEKE’17). 501–506.Google ScholarCross Ref
[9] Grosse R. Ranganath H. Lee, R., and Ng A. Y.. 2009. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In Proceedings of the 26th Annual International Conference on Machine Learning (ICML’09). 609–616.Google Scholar
[10] Heiden Simon, Grunske Lars, Kehrer Timo, Keller Fabian, Hoorn Andre Van, Filieri Antonio, and Lo David. 2019. An evaluation of pure spectrum-based fault localization techniques for large-scale software systems. Software: Practice and Experience 49, 8 (2019), 1197–1224.Google ScholarCross Ref
[11] Hua Jie Lee, Naish Lee, and Ramamohanarao Kotagiri. 2010. Effective software bug localization using spectral frequency weighting function. In Proceedings of the 34th Annual Computer Software and Applications Conference (COMPSAC’10). 449–456.Google Scholar
[12] Jiang Jiajun, Xiong Yingfei, Zhang Hongyu, Gao Qing, and Chen Xiangqun. 2018. Shaping program repair space with existing patches and similar code. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’18). 298–309.Google ScholarDigital Library
[13] Jones James A.. 2004. Fault localization using visualization of test information. In Proceedings of the International Conference on Software Engineering (ICSE’04). 54–56.Google ScholarCross Ref
[14] Jones James A., Bowring James F., and Harrold Mary Jean. 2007. Debugging in parallel. In Proceedings of the 2007 International Symposium on Software Testing and Analysis (ISSTA’07). ACM, 16–26.Google ScholarDigital Library
[15] Jones James A. and Harrold Mary Jean. 2005. Empirical evaluation of the tarantula automatic fault-localization technique. In Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering (ASE’05). 273–282.Google ScholarDigital Library
[16] Keller Fabian, Grunske Lars, Heiden Simon, Filieri Antonio, van Hoorn Andre, and Lo David. 2017. A critical evaluation of spectrum-based fault localization techniques on a large-scale software system. In Proceedings of the 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS’17). IEEE, 114–125.Google ScholarCross Ref
[17] Kochhar Pavneet Singh, Xia Xin, Lo David, and Li Shanping. 2016. Practitioners’ expectations on automated fault localization. In Proceedings of the 25th International Symposium on Software Testing and Analysis (ISSTA’16). 165–176.Google ScholarDigital Library
[18] Le Tien Duy B., Oentaryo Richard J., and Lo David. 2015. Information retrieval and spectrum based bug localization: Better together. In Proceedings of the10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE’15). 579–590.Google ScholarDigital Library
[19] Goues Claire Le, Nguyen ThanhVu, Forrest Stephanie, and Weimer Westley. 2011. Genprog: A generic method for automatic software repair. IEEE Transactions on Software Engineering (TSE) 38, 1 (2011), 54–72.Google ScholarDigital Library
[20] Lecun Yann, Bengio Yoshua, and Hinton Geoffrey. 2015. Deep learning. Nature 521, 7553 (2015), 436.Google ScholarCross Ref
[21] Lei Yan, Mao Xiaoguang, Dai Ziying, and Wang Chengsong. 2012. Effective statistical fault localization using program slices. In Proceedings of the 36th Annual Computer Software and Applications Conference (COMPSAC’12). 1–10.Google ScholarDigital Library
[22] Lei Yan, Mao Xiaoguang, Zhang Min, Ren Jingan, and Jiang Yinhua. 2017. Toward understanding information models of fault localization: Elaborate is not always better. In Proceedings of the 41st Annual Computer Software and Applications Conference (COMPSAC’17). IEEE, 57–66.Google ScholarCross Ref
[23] Lei Yan, Sun Chengnian, Mao Xiaoguang, and Su Zhendong. 2018. How test suites impact fault localisation starting from the size. IET Software 12, 3 (2018), 190–205.Google ScholarDigital Library
[24] Li Xia, Li Wei, Zhang Yuqun, and Zhang Lingming. 2019. DeepFL: Integrating multiple fault diagnosis dimensions for deep fault localization. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’19). 169–180.Google ScholarDigital Library
[25] Li Xia and Zhang Lingming. 2017. Transforming programs and tests in tandem for fault localization. In Proceedings of the ACM on Programming Languages (OOPSLA) (2017), 1–30.Google Scholar
[26] Li Yi, Wang Shaohua, and Nguyen Tien N.. 2021. Fault localization with code coverage representation learning. In Proceedings of the International Conference on Software Engineering (ICSE’21).Google ScholarDigital Library
[27] Liu Kui, Koyuncu Anil, Kim Dongsun, and Bissyandé Tegawendé F. 2019. TBar: Revisiting template-based automated program repair. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’19). 31–42.Google ScholarDigital Library
[28] Lou Yiling, Ghanbari Ali, Li Xia, Zhang Lingming, Zhang Haotian, Hao Dan, and Zhang Lu. 2020. Can automated program repair refine fault localization? A unified debugging approach. In Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’20). 75–87.Google ScholarDigital Library
[29] Madeiral Fernanda, Urli Simon, Maia Marcelo, and Monperrus Martin. 2019. Bears: An extensible java bug benchmark for automatic program repair studies. In Proceedings of the 26th International Conference on Software Analysis, Evolution and Reengineering (SANER’19). IEEE, 468–478.Google ScholarCross Ref
[30] Mao Xiaoguang, Lei Yan, Dai Ziying, Qi Yuhua, and Wang Chengsong. 2014. Slice-based statistical fault localization. Journal of Systems and Software 89, 1 (2014), 51–62.Google ScholarDigital Library
[31] Masri Wes and Assi Rawad Abou. 2010. Cleansing test suites from coincidental correctness to enhance fault localization. In Proceedings of the 3rd International Conference on Software Testing, Verification and Validation (ICST’10).Google ScholarDigital Library
[32] Masri Wes and Assi Rawad Abou. 2014. Prevalence of coincidental correctness and mitigation of its impact on fault localization. ACM Transactions on Software Engineering and Methodology (TOSEM) 23, 1 (2014), 1–28.Google ScholarDigital Library
[33] Moon Seokhyeon, Kim Yunho, Kim Moonzoo, and Yoo Shin. 2014. Ask the mutants: Mutating faulty programs for fault localization. In Proceedings of the 7th International Conference on Software Testing, Verification and Validation (ICST’14). 153–162.Google ScholarDigital Library
[34] Doukhan J. J. DiCarlo N. Pinto, D., and Cox D. D.. 2009. A high-throughput screening approach to discovering good forms of biologically inspired visual representation. In PLoS Computational Biology. vol. 5.Google Scholar
[35] Naish Lee and Hua. 2011. A model for spectra-based software diagnosis. ACM Transactions on Software Engineering and Methodology (TOSEM) 20, 3 (2011), 1–32.Google ScholarDigital Library
[36] Papadakis Mike and Traon Yves Le. 2012. Using mutants to locate “Unknown” faults. In Proceedings of the 5th International Conference on Software Testing, Verification and Validation (ICST’12). 691–700.Google ScholarDigital Library
[37] Papadakis Mike and Traon Yves Le. 2015. Metallaxis-FL: Mutation-based Fault Localization. John Wiley and Sons Ltd. 605–628.Google Scholar
[38] Parnin Chris and Orso Alessandro. 2011. Are automated debugging techniques actually helping programmers?. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA’11). 199–209.Google ScholarDigital Library
[39] Spencer Pearson, Jose Campos, Rene Just, Gordon Fraser, Rui Abreu, Michael D. Ernst, Deric Pang, and Benjamin Keller. 2017. Evaluating and improving fault localization. In Proceedings of the International Conference on Software Engineering (ICSE’17).Google Scholar
[40] Zoeteweij A. J. C. van Gemund, R. Abreu, and P.. 2006. An evaluation of similarity coeffcients for software fault localization. In Proceedings of the 12th Pacific Rim International Symposium on Dependable Computing. 39–46.Google Scholar
[41] Rajaraman Anand and Ullman Jeffrey David. 2011. Mining of massive datasets: Data mining. Min. Massive Datasets (2011), 1–17.Google Scholar
[42] Rui Abreu, Zoeteweij Peter, and van Gemund Arjan J. C.. 2009. Spectrum-based multiple fault localization. In Proceedings of the International Conference on Automated Software Engineering (ASE’09). 88–99.Google Scholar
[43] Rui Abreu, Zoeteweij Peter, Golsteijn Rob, and van Gemund Arjan J. C.. 2009. A practical evaluation of spectrum-based fault localization. Journal of Systems and Software 82, 11 (2009), 1780–1792.Google ScholarDigital Library
[44] Shi Cong Ying, ChaoJun X. U., and Yang Xiao Jiang. 2009. Study of TFIDF algorithm. Journal of Computer Applications (2009).Google Scholar
[45] Shu Ting, Ye Tiantian, Ding Zuohua, and Xia Jinsong. 2016. Fault localization based on statement frequency. Information Sciences 360 (2016), 43–56.Google ScholarDigital Library
[46] Sohn Jeongju and Yoo Shin. 2017. FLUCCS: Using code and change metrics to improve fault localization. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’17). ACM, New York, 273–283. Google ScholarDigital Library
[47] Soremekun Ezekiel, Kirschner Lukas, Böhme Marcel, and Zeller Andreas. 2021. Locating faults with program slicing: An empirical analysis. Empirical Software Engineering 26, 3 (2021), 1–45.Google ScholarDigital Library
[48] Sun Chengnian and Khoo Siau Cheng. 2013. Mining succinct predicated bug signatures. In Proceedings of the 9th Joint Meeting on Foundations of Software Engineering (ESEC/FSE’13). 576–586.Google ScholarDigital Library
[49] Turaga S. C., Murray J. F., Jain V., Roth F., Helmstaedter M., Briggman K., Denk W., and Seung H. S.. 2010. Convolutional networks can learn to generate affinity graphs for image segmentation. Neural Computation 22, 2 (2010), 511–538.Google ScholarDigital Library
[50] Vargha András and Delaney Harold D. 2000. A critique and improvement of the CL common language effect size statistics of McGraw and Wong. Journal of Educational and Behavioral Statistics 25, 2 (2000), 101–132.Google Scholar
[51] Wang Xinming, Cheung Shing Chi, Chan Wing Kwong, and Zhang Zhenyu. 2009. Taming coincidental correctness: Coverage refinement with context patterns to improve fault localization. In Proceedings of the IEEE International Conference on Software Engineering (ICSE’09).Google ScholarDigital Library
[52] Wong W. Eric, Debroy Vidroha, and Choi Byoungju. 2010. A family of code coverage-based heuristics for effective fault localization. Journal of Systems and Software (2010), 188–208.Google ScholarDigital Library
[53] Wong W. Eric, Debroy Vidroha, Golden Richard, Xu Xiaofeng, and Thuraisingham Bhavani. 2012. Effective software fault localization using an RBF neural network. IEEE Transactions on Reliability 61, 1 (2012), 149–169.Google ScholarCross Ref
[54] Wong W. Eric, Debroy Vidroha, Li Yihao, and Gao Ruizhi. 2012. Software fault localization using DStar (D*). In IEEE Sixth International Conference on Software Security and Reliability. 21–30.Google ScholarDigital Library
[55] Wong W. Eric, Gao Ruizhi, Li Yihao, Rui Abreu, and Wotawa Franz. 2016. A survey on software fault localization. IEEE Transactions on Software Engineering (TSE) 42, 8 (2016), 707–740.Google ScholarDigital Library
[56] Wong W. Eric and Qi Yu. 2009. BP neural network-based effective fault localization. International Journal of Software Engineering and Knowledge Engineering 19, 04 (2009), 573–597.Google ScholarCross Ref
[57] Wong W. Eric, Qi Yu, Zhao Lei, and Cai Kai Yuan. 2007. Effective fault localization using code coverage. In Proceedings of the 31st Annual International Computer Software and Applications Conference (COMPSAC’07). 449–456.Google ScholarDigital Library
[58] Xia Xin, Bao Lingfeng, Lo David, and Li Shanping. 2016. “Automated debugging considered harmful” considered harmful: A user study revisiting the usefulness of spectra-based fault localization techniques with professionals using real bugs from large systems. In Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME’16). 267–278.Google ScholarCross Ref
[59] Xie Xiaoyuan, Chen Tsong Yueh, Kuo Fei Ching, and Xu Baowen. 2013. A theoretical analysis of the risk evaluation formulas for spectrum-based fault localization. ACM Transactions on Software Engineering and Methodology (TOSEM) 22, 4 (2013), 31.Google ScholarDigital Library
[60] Xie Xiaoyuan, Kuo Fei Ching, Chen Tsong Yueh, Yoo Shin, and Harman Mark. 2013. Provably optimal and human-competitive results in SBSE for spectrum based fault localisation. In Proceedings of the 5th International Symposium on Search Based Software Engineering (SSBSE’13). Springer, Berlin, 224–238.Google ScholarDigital Library
[61] Xu Baowen, Qian Ju, Zhang Xiaofang, Wu Zhongqiang, and Chen Lin. 2005. A brief survey of program slicing. ACM SIGSOFT Software Engineering Notes 30, 2 (2005), 1–36.Google ScholarDigital Library
[62] Xuan Jifeng, Martinez Matias, Demarco Favio, Clement Maxime, Marcote Sebastian Lamelas, Durieux Thomas, Berre Daniel Le, and Monperrus Martin. 2016. Nopol: Automatic repair of conditional statement bugs in Java programs. IEEE Transactions on Software Engineering (TSE) 43, 1 (2016), 34–55.Google ScholarDigital Library
[63] Zakari Abubakar, Lee Sai Peck, Abreu Rui, Ahmed Babiker Hussien, and Rasheed Rasheed Abubakar. 2020. Multiple fault localization of software programs: A systematic literature review. Information and Software Technology 124 (2020), 106312.Google ScholarCross Ref
[64] Zhang Zhuo, Lei Yan, Mao Xiaoguang, Chang Xi, Xue Jianxin, and Xiong Qingyu. 2020. Fault localization approach using term frequency and inverse document frequency. Journal of Software 31, 11 (2020), 132–144.Google Scholar
[65] Zhang Zhuo, Lei Yan, Mao Xiaoguang, and Li Panpan. 2019. CNN-FL: An effective approach for localizing faults using convolutional neural networks. In the 26th International Conference on Software Analysis, Evolution and Reengineering (SANER’19). IEEE, 445–455.Google ScholarCross Ref
[66] Zhang Zhuo, Lei Yan, Mao Xiaoguang, Yan Meng, Xu Ling, and Wen Junhao. 2020. Improving deep-learning-based fault localization with resampling. Journal of Software: Evolution and Process (2020), 1–22.Google Scholar
[67] Zhang Zhuo, Lei Yan, Mao Xiaoguang, Yan Meng, Xu Ling, and Zhang Xiaohong. 2021. A study of effectiveness of deep learning in locating real faults. Information and Software Technology 131 (2021), 106486.Google ScholarCross Ref
[68] Zhang Zhuo, Lei Yan, Tan Qingping, Mao Xiaoguang, Zeng Ping, and Chang Xi. 2017. Deep learning-based fault localization with contextual information. IEICE Transactions on Information and Systems E100.D, 12 (2017), 3027–3031.Google ScholarCross Ref
[69] Zhang Zhuo, Lei Yan, Xu Jianjun, Mao Xiaoguang, and Chang Xi. 2019. TFIDF-FL: Localizing faults using term frequency-inverse document frequency and deep learning. IEICE Transactions on Information and Systems 102, 9 (2019), 1860–1864.Google ScholarCross Ref
[70] Zheng Wei, Hu Desheng, and Wang Jing. 2016. Fault localization analysis based on deep neural network. Mathematical Problems in Engineering, 2016, (2016-4-24) 2016 (2016), 1–11.Google Scholar
[71] Zou Daming, Liang Jingjing, Xiong Yingfei, Ernst Michael D., and Zhang Lu. 2019. An empirical study of fault localization families and their combinations. IEEE Transactions on Software Engineering (TSE) (2019), 1–1.Google Scholar

Index Terms

Influential Global and Local Contexts Guided Trace Representation for Fault Localization
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

An effective fault aware test case prioritization by incorporating a fault localization technique
ESEM '10: Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement

Prior coverage-based test case prioritization techniques aim to increase fault detection rates by ordering the test cases according to some coverage criteria. However, in practice, since detected faults are typically removed, test cases that already ...
Read More
Tester Feedback Driven Fault Localization
ICST '12: Proceedings of the 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation

Coincidentally correct test cases are those that execute faulty statements but do not cause failures. Such test cases reduce the effectiveness of spectrum-based fault localization techniques, such as Ochiai, because the correlation of failure with the ...
Read More
Test case selection using multi-criteria optimization for effective fault localization

As spectra-based fault localization techniques report suspicious statements by analyzing the coverage of test cases, the effectiveness of the results is highly dependent on the composition of test suites. This paper proposes an approach for selecting a ...
Read More

Comments

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Published in
ACM Transactions on Software Engineering and Methodology Volume 32, Issue 3
May 2023
937 pages
ISSN:1049-331X
EISSN:1557-7392
DOI:10.1145/3594533
Editor:
Mauro Pezzè
USI Università della Svizzera italiana and SIT Schaffhausen Institute of Technology, Switzerland
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
- Published: 26 April 2023
- Online AM: 15 December 2022
- Accepted: 26 November 2022
- Revised: 10 October 2022
- Received: 18 February 2022
Published in tosem Volume 32, Issue 3

Permissions
Request permissions about this article.
Request Permissions

Check for updates
Author Tags
Fault localization
trace representation
statement weighting
program dependence
suspiciousness
Qualifiers
- research-article
Conference
Funding Sources
Other Metrics
View Article Metrics

Article Metrics
- 2
  Total Citations
  View Citations
- 430
  Total Downloads
- Downloads (Last 12 months)308
- Downloads (Last 6 weeks)49
Other Metrics
View Author Metrics
Cited By
View all

PDF Format

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

View Full Text

HTML Format

View this article in HTML Format .

View HTML Format

Influential Global and Local Contexts Guided Trace Representation for Fault Localization

ACM Transactions on Software Engineering and Methodology

Abstract

REFERENCES

Cited By

Index Terms

Recommendations

An effective fault aware test case prioritization by incorporating a fault localization technique

Tester Feedback Driven Fault Localization

Test case selection using multi-criteria optimization for effective fault localization