Skip to main content
SpringerLink
Log in

Invalid bug reports complicate the software aging situation

  • Published:
Software Quality Journal Aims and scope Submit manuscript

Abstract

Symptoms of software aging include performance degradation and failure occurrence increasing when software systems run for a period of time. Therefore, software aging is closely related to system performance. Understanding and analyzing performance issues in the software system is critical to mastering software aging information. Instead of focusing on normal valid bug reports (VBRs), this paper advocates the usage of invalid bug reports (IBRs) to capture software aging signals. We use performance bugs that are highly related to software aging as an example to construct a binary classification model for bug report classification. We conduct a rigorous evaluation of the constructed models via different performance measures (i.e., recall, precision, F1-score, AUC). Then, the model is used to predict the performance bug reports (PBRs) in IBRs, and a manual analysis of the prediction results is conducted to identify aging-related bug reports (ABRs). The final results show that the ratio of PBRs in IBRs ranges from 4.9 to 42.18% for the two real open-source projects HDFS and HBase when considering five different classifiers. Among these five classifiers, Support Vector Machine (SVM) classifier can achieve the best performance. The ratios of PBRs in IBRs by using this classifier are 11.1% and 15.35% for these two datasets and the performances in terms of F1-score are 85% and 74%. Further analysis of the predicted PBRs of IBRs in the project HDFS is conducted through a manual user case study; some surprising findings revealing the relationship between IBRs, PBRs, and ABRs are presented: (1) Around 50% of the PBRs in IBRs are related to software aging; (2) components that undertake major tasks are more prone to aging problems; (3) more than 50% ARBs lead to timeout, 33% ARBs are caused by improper control of memory or threats, and 29% ARBs are caused by inappropriate management of file operation or disk usage; (4) hard to reproduce is the major reason that ARBs are usually closed as invalid because many aging-related bugs would temporarily disappear by restarting the system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. https://github.com/wuxiaoxue/IBR

References

  • Agrawal, A., & Menzies, T. (2018). Is better data better than better data miners?. In Proceedings of the 40th international conference on software engineering (pp. 1050–1061): IEEE.

  • Apach hadoop project. (2019). http://hadoop.apache.org.

  • Apach hbase project. (2019). http://hadoop.apache.org/hbase.

  • Atlassian JIRA. (2019). https://www.atlassian.com/software/jira.

  • Bao, Y., Sun, X., Trivedi, K.S. (2005). A workload-based analysis of software aging, and rejuvenation. IEEE Transactions on Reliability, 54(3), 541–548.

    Article  Google Scholar 

  • Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.

    Article  Google Scholar 

  • Chang, C.C., & Lin, C.J. (2011). Libsvm: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST), 2(3), 27.

    Google Scholar 

  • Cl and ci. (2019). https://www.surveysystem.com/sscalc.htm.

  • Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297.

    MATH  Google Scholar 

  • Cotroneo, D., Natella, R., Pietrantuono, R. (2011). Is software aging related to software metrics. In IEEE second international workshop on software aging and rejuvenation.

  • Cotroneo, D., Natella, R., Pietrantuono, R. (2013). Predicting aging-related bugs using software complexity metrics. Performance Evaluation, 70(3), 163–178.

    Article  Google Scholar 

  • Cotroneo, D., Pietrantuono, R., Russo, S., Trivedi, K. (2016). How do bugs surface? A comprehensive study on the characteristics of software bugs manifestation. Journal of Systems and Software, 113(C), 27–43.

    Article  Google Scholar 

  • Cucchiara, A. (2012). Applied logistic regression. Technometrics, 34(3), 358–359.

    Article  Google Scholar 

  • Dimensionality reduction. (2019). https://en.wikipedia.org/wiki/Dimensionality_reduction.

  • Domingos, P., & Pazzani, M. (1997). On the optimality of the simple Bayesian classifier under zero-one loss. Machine Learning, 29(2-3), 103–130.

    Article  Google Scholar 

  • Elshishiny, H., Deraz, S., Bahy, O. (2008). Mining software aging patterns by artificial neural networks. In Iapr workshop on artificial neural networks in pattern recognition.

  • Fan, R.E., Chang, K.W., Hsieh, C.J. (2008). Wang: Liblinear: a library for large linear classification. Journal of Machine Learning Research, 9(9), 1871–1874.

    MATH  Google Scholar 

  • Fan, Y., Xia, X., Lo, D., Hassan, A.E. (2018). Chaff from the wheat: characterizing and determining valid bug reports. IEEE Transactions on Software Engineering, PP(99), 1–1.

    Article  Google Scholar 

  • Gardner, M.W., & Dorling, S.R. (1998). Artificial neural networks (the multilayer perceptron): a review of applications in the atmospheric sciences. Atmospheric Environment, 32(14–15), 2627–2636.

    Article  Google Scholar 

  • Ghoneim, S.A., & Fahmy, H.M.A. (2003). Evaluation of the drm and the time for preventive maintenance for aging software. Software Quality Journal, 11(1), 57–75.

    Article  Google Scholar 

  • Goseva-Popstojanova, K., & Tyo, J. (2018). Identification of security related bug reports via text mining using supervised and unsupervised classification. International Conference on Software Quality, Reliability, and Security, QRS 2018 pp. 344–355.

  • Grottke, M., Dong, S.K., Mansharamani, R., Nambiar, M., Natella, R., Trivedi, K.S. (2016). Recovery from software failures caused by mandelbugs. IEEE Transactions on Reliability, 65(1), 70–87.

    Article  Google Scholar 

  • Gunawi, H.S., & Hao, M. (2014). Leesatapornwongsa: what bugs live in the cloud? a study of 3000+ issues in cloud systems. In Proceedings of the ACM symposium on cloud computing.

  • Han, X., Tingting, Y., David, L. (2018). Perflearner: learning from bug reports to understand and generate performance test frames. In Proceedings of the 33rd ACM/IEEE international conference on automated software engineering.

  • Hdfs architecture. (2019). https://hadoop.apache.org/docs/current1/hdfs_design.html.

  • Huang, Y., Kintala, C., Kolettis, N., Fulton, N.D. (1995). Software rejuvenation: analysis, module and applications. In International symposium on fault-tolerant computing.

  • Kotsiantis, S.B., & Kanellopoulos, D. (2006). Data preprocessing for supervised leaning. International Journal of Computer Science, 1(2), 111–117.

    Google Scholar 

  • Kumar, L., & Ashish, S. (2018). Feature selection techniques to counter class imbalance problem for aging related bug prediction: aging related bug prediction. In Proceedings of the 11th innovations in software engineering conference.

  • Lindorfer, M., Neugschwandtner, M., Platzer, C. (2015). Marvin: efficient and comprehensive mobile app classification through static and dynamic analysis. In IEEE computer software and applications conference.

  • Linearsvc. (2019). https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html.

  • Lu, S., Park, S., Seo, E. (2008). Learning from mistakes: a comprehensive study on real world concurrency bug characteristics. In Proceedings of the international conference on architectural support for programming languages and operating systems (ASPLOS).

  • Mani, S., Sankaran, A., Aralikatte, R. (2018). Deeptriage: exploring the effectiveness of deep learning for bug triaging. arXiv:1801.01275.

  • Ng, A.Y. (2004). Feature selection, l 1 vs. l 2 regularization, and rotational invariance. In Proceedings of the twenty-first international conference on machine learning.

  • Peters, F., Tun, T., Yu, Y., Nuseibeh, B. (2017). Text filtering and ranking for security bug report prediction. IEEE Transactions on Software Engineering, PP (99), 1–1.

    Google Scholar 

  • Qiao, Y., Zheng, Z., Yunyu, F. (2018). Two-level rejuvenation for android smartphones and its optimization. IEEE Transactions on Reliability.

  • Qin, F., Zheng, Z., Li, X., Yu, Q., Trivedi, K.S. (2017). An empirical investigation of fault triggers in android operating system. In IEEE Pacific rim international symposium on dependable computing.

  • Qin, F., Zheng, Z., Qiao, Y., Trivedi, K.S. (2018). Studying aging-related bug prediction using cross-project models. IEEE Transactions on Reliability.

  • Romano, J. (2006). Appropriate statistics for ordinal level data: should we really be using t-test and cohen’sd for evaluating group differences on the nsse and other surveys. In Annual meeting of the Florida association of institutional research.

  • Scikit learn. (2019). https://scikit-learn.org/.

  • Selectfrommodel. (2019). https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.SelectFromModel.html.

  • Shu, R., Xia, T., Williams, L., Menzies, T. (2019). Better security bug report classification via hyperparameter optimization. In International conference on automated software engineering, 2019 (pp. 1–12): IEEE/ACM.

  • Storn, R., & Price, K. (1997). Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization, 11(4), 341–359.

    Article  MathSciNet  Google Scholar 

  • Umesh, I.M., & Srinivasan, G.N. (2017). Dynamic software aging detection-based fault tolerant software rejuvenation model for virtualized environment. In Proceedings of the international conference on data engineering and communication technology. Springer, Singapore.

  • Wang, Z., Wu, C., Yuan, X., Wang, Z., Li, J. (2018). Using local clocks to reproduce concurrency bugs. IEEE Transactions on Software Engineering, 44(11), 1112–1128.

    Article  Google Scholar 

  • Yang, X.L., Lo, D., Xia, X., Huang, Q., Sun, J.L. (2017). High-impact bug report identification with imbalanced learning strategies. Journal of Computer Science and Technology, 32, 181–198.

    Article  Google Scholar 

  • Yu, T., Wei, W., Xue, H., Hayes, J. (2018). Conpredictor: concurrency defect prediction in real-world applications. IEEE Transactions on Software Engineering, PP (99), 1–1.

    Article  Google Scholar 

  • Zhang, Y., Lo, D., Xia, X., Sun, J. (2015). An empirical study of classifier combination for cross-project defect prediction. In IEEE computer software and applications conference.

Download references

Author information

Authors and Affiliations

  1. Northwestern Polytechnical University, Xi’an, China

    Xiaoxue Wu, Wei Zheng, Minchao Pu, Jie Chen & Dejun Mu

Authors
  1. Xiaoxue Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Minchao Pu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jie Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dejun Mu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Zheng.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, X., Zheng, W., Pu, M. et al. Invalid bug reports complicate the software aging situation. Software Qual J 28, 195–220 (2020). https://doi.org/10.1007/s11219-019-09481-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11219-019-09481-2

Keywords

Search

Navigation