Abstract
Bug tracking systems receive a large number of bugs on a daily basis. The process of maintaining the integrity of the software and producing high-quality software is challenging. The bug-sorting process is usually a manual task that can lead to human errors and be time-consuming. The purpose of this research is twofold: first, to conduct a literature review on the bug report priority classification approaches, and second, to replicate existing approaches with various classifiers to extract new insights about the priority classification approaches. We used a Systematic Literature Review methodology to identify the most relevant existing approaches related to the bug report priority classification problem. Furthermore, we conducted a replication study on three classifiers: Naive Bayes (NB), Support Vector Machines (SVM), and Convolutional Neural Network (CNN). Two sets of experiments are performed: first, our own NLTK implementation based on NB and CNN, and second, based on Weka implementation for NB, SVM, and CNN. The dataset used consists of several Eclipse projects and one project related to database systems. The obtained results are better for the bug priority P3 for the CNN classifier, and overall the quality relation between the three classifiers is preserved as in the original studies. The replication study confirmed the findings of the original studies, emphasizing the need to further investigate the relationship between the characteristics of the projects used as training and those used as testing.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability and materials
Data will be made available on request.
References
Alenezi, M., Banitaan, S.: Bug reports prioritization Which features and classifier to use? 2013 12th International conference on machine learning and applications 2, 112–116 (2013)
Bugzilla. Bugzilla: Bug tracking systems. https://www.bugzilla.org/. (Accessed March 2023)
Carver, J.C.: Towards reporting guidelines for experimental replications: a proposal. The international workshop on replication in empirical software engineering (pp. 2–5) (2010)
Carver, J.C., Juristo, N., Baldassarre, M.T., Vegas, S.: Replications of software engineering experiments. Emp. Softw. Eng. 19(2), 267–276 (2014). https://doi.org/10.1007/s10664-013-9290-8
Dyba, T., Kitchenham, B.A., Jorgensen, M.: Evidence-based software engineering for practitioners. IEEE Softw. 22(1), 58–65 (2005)
Eclipse-bugs, E.: Eclipse. https://bugs.eclipse.org/bugs/. (Accessed March 2023)
Fagerholm, F., Becker, C., Chatzigeorgiou, A., Betz, S., Duboc, L., Penzenstadler, B.,...Venters, C.C.: Temporal discounting in software engineering: A replication study. 13th acm/ieee international symposium on empirical software engineering and measurement (pp. 1-12). IEEE.(2019, 10 17)
Frank, E., Hall, M.A., Witten, I.H.: Weka - waikato environment for knowledge analysis. (Accessed March 2023). https://www.cs.waikato.ac.nz/ml/weka/
Galbîn-Năsui, A.: Predictia automata a prioritatii unui bug (Unpublished master’s thesis). Bachelor’s thesis. Babes-Bolyai University Cluj-Napoca Faculty of Mathematics and Computer Science.(2020)
Galbîn-Năsui, A.: Bug reports priority classification model (Unpublished master’s thesis). Babes-Bolyai University Cluj-Napoca Faculty of Mathematics and Computer Science. (2022)
Gómez, O.S., Juristo, N., Vegas, S.: Understanding replication of experiments in software engineering: a classification. Inf. Softw. Technol. 56(8), 1033–1048 (2014). https://doi.org/10.1016/j.infsof.2014.04.004
Juristo, N., Vegas, S.: Using differences among replications of software engineering experiments to gain knowledge. In: 2009 3rd international symposium on empirical software engineering and measurement (356–366) (2009)
Kitchenham, B., Charters, S.: Guidelines for performing systematic literature reviews in software engineering. 2 (2007)
Myers, G.: The Art of Software Testing. Wiley, Hoboken (2005)
NLTK. Natural language toolkit. https://www.NLTK.org/. (Accessed March 2023)
Patton, R.: Software testing. SAMS. (2000)
Python.: Python: Python-programming language. https://www.python.org/ (Accessed March 2023).
Ramay, W.Y., Umer, Q., Yin, X.C., Zhu, C., Illahi, I.: Deep neural network based severity prediction of bug reports. IEEE Access 7, 46846–46857 (2019). https://doi.org/10.1109/ACCESS.2019.2909746
SentiWordNet.: Sentiwordnet-lexical resource for opinion mining. https://github.com/aesuli/SentiWordNet. (Accessed March 2023)
Sharma, G., Sharma, S., Gujral, S.: A novel way of assessing software bug severity using dictionary of critical terms. Procedia Comput. Sci. 70, 632–639 (2015). https://doi.org/10.1016/j.procs.2015.10.059
Shepperd, M., Ajienka, N., Counsell, S.: The role and value of replication in empirical software engineering results. Inf. Softw. Technol. 99, 120–132 (2018). https://doi.org/10.1016/j.infsof.2018.01.006
Tian, Y., Lo, D., Sun, C.: Information retrieval based nearest neighbor classification for fine-grained bug severity prediction. In: 2012 19th Working conference on reverse engineering (215–224). (2012)
Tian, Y., Lo, D., Sun, C.: Drone: predicting priority of reported bugs by multifactor analysis. In: 2013 IEEE international conference on software maintenance (200–209). (2013)
Trevor Hastie, J.F., Tibshirani, R.: The Elements of Statistical Learning. Springer, Cham (2016)
Uddin, J., Ghazali, R., Deris, M.M., Naseem, R., Shah, H.: A survey on bug prioritization. Artif. Intell. Rev. 47, 145–180 (2017). https://doi.org/10.1007/s10462-016-9478-6
Umer, Q., Liu, H., Illahi, I.: CNN-based automatic prioritization of bug reports. IEEE Trans. Reliab. 69(4), 1341–1354 (2020). https://doi.org/10.1109/TR.2019.2959624
Umer, Q., Liu, H., Sultan, Y.: Emotion based automated priority prediction for bug reports. IEEE Access 6, 35743–35752 (2018). https://doi.org/10.1109/ACCESS.2018.2850910
Weber, B.G.: Data science in production: Building scalable model pipelines with python. (2020). Independently published (January 1, 2020)
Word2Vec . Word2vec- used to learn word embeddings. https://www.tensorflow.org/tutorials/text/word2vec/. (Accessed March 2023)
Yu, L., Tsai, W.-T., Zhao, W., Wu, F.: Predicting defect priority based on neural networks. In: Cao, L., Zhong, J., Feng, Y. (eds.) Advanced Data Mining and Applications, pp. 356–367. Springer, Berlin Heidelberg (2010)
Acknowledgements
This work was funded by the Ministry of Research, Innovation, and Digitization, CNCS/CCCDI - UEFISCDI, Project number PN-III-P1-1.1-TE2021-0892 within PNCDI III. We also would like to thank professor Alexander Serebrenik, our research collaborator, from the Eindhoven University of Technology for providing us with improvement suggestions for the study and useful insights on how to improve the paper.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
Author Andreea Vescan and Andreea Nasui-Galbin declares that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Galbin-Nasui, A., Vescan, A. Bug reports priority classification models. Replication study. Autom Softw Eng 31, 35 (2024). https://doi.org/10.1007/s10515-024-00432-1
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10515-024-00432-1