Abstract
Given a bucket of crash reports, it would be helpful for developers to find and fix the corresponding defects if the crash-inducing software changes can be automatically located. Recently, an approach called ChangeLocator was proposed, which used ten change-level features to train a supervised model based on the data from the historical fixed crashes. It was reported that ChangeLocator achieved a good performance in terms of Recall@1, MAP, and MRR, when all the ten features were combined together. However, in ChangeLocator, the redundancy between features are neglected, which may degrade the localization effectiveness. In this paper, we propose an improved approach ChangeRanker with a rank-performance-based feature selection technology (Rfs) to boost the effectiveness of crash-inducing change localization. Our experimental results on NetBeans show that ChangeRanker can achieve an improvement of 35.9%, 17.4%, and 15.3% over ChangeLocator in terms of Recall@1, MRR, and MAP, respectively. Furthermore, compared with three popular feature selection approaches, Rfs is able to select more informative features to boost localization effectiveness. In order to assess the real generalization capability of the proposed extension, we adapt ChangeRanker and ChangeLocator to locate bug-inducing changes on three additional data sets. Again, we observe that, on average, ChangeRanker achieves an improvement of 115.3%, 37.6%, and 41.2% in terms of Recall@1, MRR, and MAP, respectively. This indicates that our proposed rank-performance-based feature selection method has a good generalization capability. In summary, our work provides an easy-to-use approach to boosting the performance of the state-of-the-art crash-inducing change localization approach.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10664-020-09802-1/MediaObjects/10664_2020_9802_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10664-020-09802-1/MediaObjects/10664_2020_9802_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10664-020-09802-1/MediaObjects/10664_2020_9802_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10664-020-09802-1/MediaObjects/10664_2020_9802_Fig4_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10664-020-09802-1/MediaObjects/10664_2020_9802_Fig5_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10664-020-09802-1/MediaObjects/10664_2020_9802_Fig6_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10664-020-09802-1/MediaObjects/10664_2020_9802_Fig7_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10664-020-09802-1/MediaObjects/10664_2020_9802_Fig8_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10664-020-09802-1/MediaObjects/10664_2020_9802_Fig9_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10664-020-09802-1/MediaObjects/10664_2020_9802_Fig10_HTML.png)
Similar content being viewed by others
Notes
CrashPad [online]. Available: https://chromium.googlesource.com/crashpad/crashpad/+/master/README.md
Technical note tn2123: Crashreporter (2015) [online]. Available: developer.apple.com/library/mac/#technotes/tn2004/tn2123.html
NetBeans project [online]. Available: https://netbeans.org/
The data sets and scripts can be download via https://github.com/Naplues/ChangeRanker.
References
P.D. Allison. Logistic regression using the SAS system: theory and application. 1999
An L, Khomh F (2015) An empirical study of crash-inducing commits in Mozilla Firefox. PROMISE:1–10
Benavoli A, Corani G, Mangili F (2016) Should we really use post-hoc tests based on mean-ranks? J Mach Learn Res 17(5):1–10
Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Ser B 57:289–300
Bettenburg N, Premraj R, Zimmermann T (2008) Extracting structural information from bug reports. MSR:27–30
Chawla N, Bowyer K, Hall L, Kegelmeyer P (2002) Smote: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357
Cohen J, Cohen P, West SG (2003) Applied multiple regression/correlation analysis for behavioral sciences. Lawrence Erlbaum, Hillsdale, NJ
Costa DA, Mcintosh S, Shang W (2017) A framework for evaluating the results of the SZZ approach for identifying bug-introducing changes. IEEE Trans Softw Eng 43(7):641–657
Dang Y, Wu R, Zhang H (2012) Rebucket: a method for clustering duplicate crash reports based on call stack similarity. ICSE:1084–1093
Gao K, Khoshgoftaar TM, Seliya N (2012) Predicting high-risk program modules by selecting the right software measurements. Softw Qual J 20(1):3–42
Ghotra B, McIntosh S, Hassan AE (2017) A large-scale study of the impact of feature selection techniques on defect classification models. MSR:146–157
Green SB (1991) How many subjects does it take to do a regression analysis? Multivar Behav Res 26(3):499–510
Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182
Hall M, Holmes G (2003) Benchmarking attribute selection techniques for discrete class data mining. IEEE Trans Knowl Data Eng 15(6):1437–1447
Hoang VD, Oentaryo RJ, Le (2018) TDB Network-clustered multi-modal bug localization. IEEE Transactions on Software Engineering, to appear
Holmes G, Donkin A, Witten IH (2002) WEKA: a machine learning workbench. ANZIIS:357–361
Jacek S, Zimmermann T, Zeller A (2005) When do changes induce fixes? MSR:1–5
Kim S, Zimmermann T, Pan K (2006) Automatic identification of bug-introducing changes. ASE:81–90
Kim S, Zimmermann T, Whitehead EJ (2007) Predicting faults from cached history. ICSE:489–498
Kim S, Whitehead EJ, Zhang Y (2008) Classifying software changes: clean or buggy? IEEE Trans Softw Eng 34(2):181–196
Kim D, Wang X, Kim S (2011) Which crashes should I fix first?: predicting top crashes at an early stage to prioritize debugging efforts. IEEE Trans Softw Eng 37(3):430–447
Kinshumann K, Glerum K, Greenberg S, Aul G, Orgovan V, Nichols G, Grant D, Loihle G, Hunt G (2011) Debugging in the (very) large: ten years of implementation and experience. Commun ACM 54(7):111–116
Kohavi R. R. A study of cross-validation and bootstrap for accuracy estimation and model selection. IJCAI 1995: 1137–1143
Kohavi R, John G (1998) The wrapper approach. Feature Extraction, Construction and Selection. Springer US
Le TDB, Oentaryo RJ, Lo D (2015) Information retrieval and spectrum based bug localization: better together. FSE:579–590
Lewis C, Lin Z, Sadowski C et al (2013) Does bug prediction support human developers? Findings from a google case study. ICSE:372–381
Lin D, Lin F, Lv Y, Cai F, Cao D (2018) Chinese character CAPTCHA recognition and performance estimation via deep neural network. Neurocomputing 288:11–19
Lukins SK, Kraft NA, Etzkorn LH (2008) Source code retrieval for bug localization using latent dirichlet allocation. WCRE:155–164
Manning CD, Raghavan P, Schütze H (2008) Introduction to information retrieval. Cambridge University Press
Moreno L, Bandara W, Haiduc S (2013) On the relationship between the vocabulary of bug reports and source code. ICSM:452–455
Moser R, Pedrycz W, Succi G (2009) A comparative analysis of the efficiency of change metrics and static code attributes for defect prediction. ICSE:181–190
Muthukumaran K, Rallapalli A, Murthy NLB (2015) Impact of feature selection techniques on bug prediction models. ISEC:120–129
Nguyen AT, Nguyen TT, Al-Kofahi J (2011) A topic-based approach for narrowing the search space of buggy files from a bug report. ASE:263–272
Rahman S, Ganguly KK, Sakib K (2016) An improved bug localization using structured information retrieval and version history. ICCIT
Rao S, Kak A (2011) Retrieval from software libraries for bug localization: a comparative study of generic and composite text models. MSR:43–52
Rathore SS, Gupta A (2014) A comparative study of feature-ranking and feature-subset selection techniques for improved fault prediction. ISEC
Rodríguez D, Ruiz R, Cuadrado-Gallego J, AguilarRuiz J (2007) Detecting fault modules applying feature selection to classifiers. IEEE Int Conf Inf Reuse Integr:667–672
Shaffer J (1986) Modified sequentially rejective multiple test procedures. J Am Stat Assoc 81(395):826–831
Song Q, Guo Y, Shepperd M (2018) A comprehensive investigation of the role of imbalanced learning for software defect prediction. IEEE Trans Softw Eng
Tantithamthavorn C, Mclntosh S, Hassan AE, Matsumoto K (2019) The impact of automated parameter optimization on defect prediction models. IEEE Trans Softw Eng 45(7):683–711
Uneno Y, Mizuno O, Choi EH (2016) Using a distributed representation of words in localizing relevant files for bug reports. QRS:183–190
Venkatesh GA (1991) The semantic approach to program slicing. PLDI:107–119
Wang S, Yao X (2009) Diversity analysis on imbalanced data sets by using ensemble models. CIDM:324–331
Wang S, Lo D, Lawall J (2014) Compositional vector space models for improved bug localization. ICSME:171–180
Wen M, Wu R, Cheung SC (2016) Locus: locating bugs from software changes. ASE:262–273
Wilcoxon F (1945) Individual comparisons by ranking methods. Biom Bull 1(6):80–83
Williams C, Spacco J (2008) SZZ revisited: verifying when changes induce fixes. DEFECTS:32–36
Wilson CR, Voorhis V, Morgan BL (2007) Understanding power and rules of thumb for determining sample size. Tutor Quant Methods Psychol 3(2):43–50
Wu R, Zhang H, Cheung SC (2014) Crashlocator: locating crashing faults based on crash stacks. ISSTA:204–214
Wu R, Wen M, Cheung SC (2018) Changelocator: locate crash-inducing changes based on crash reports. Empir Softw Eng 23(5):2866–2900
Xu Z, Liu J, Yang Z, An G, Jia X (2016) The impact of feature selection on defect prediction performance: An empirical comparison. ISSRE:309–320
Youm KC, Ahn J, Kim J (2016) Bug localization based on code change histories and bug reports. APSEC:190–197
Zhang L, Kim M, Khurshid S (2011) Localizing failure-inducing program edits based on spectrum information. ICSM:23–32
Acknowledgements
This work is partially supported by the National Key R&D Program of China (2018YFB1003901) and the National Natural Science Foundation of China (61772259, 61872177, 61832009, 61772263).
Author information
Authors and Affiliations
Corresponding author
Additional information
Guest Editor: Martin Monperrus
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Guo, Z., Li, Y., Ma, W. et al. Boosting crash-inducing change localization with rank-performance-based feature subset selection. Empir Software Eng 25, 1905–1950 (2020). https://doi.org/10.1007/s10664-020-09802-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10664-020-09802-1