Skip to main content
SpringerLink
Log in

Just-in-time software defect prediction using deep temporal convolutional networks

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Software maintenance and evolution can introduce defects in software systems. For this reason, there is a great interest to identify defect prediction and estimation techniques. Recent research proposes just-in-time techniques to predict defective changes just at the commit level allowing the developers to fix the defect when it is introduced. However, the performance of existing just-in-time defect prediction models still requires to be improved. This paper proposes a new approach based on a large feature set containing product and process software metrics extracted from commits of software projects along with their evolution. The approach also introduces a deep temporal convolutional networks variant based on hierarchical attention layers to perform the fault prediction. The proposed approach is evaluated on a large dataset, composed of data gathered from six Java open-source systems. The obtained results show the effectiveness of the proposed approach in timely predicting defect proneness of code components.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. https://github.com/rodhilton/jasome/.

  2. https://keras.io.

  3. https://www.tensorflow.org.

  4. https://keras.io.

References

  1. Ackerman LBA, Lewski F (1989) Software inspections: an effective verification process. IEEE Softw 6:31–36. https://doi.org/10.1109/52.28121

    Article  Google Scholar 

  2. Ahmad J, Farman H, Jan Z (2019) Deep learning methods and applications, pp 31–42. Springer Singapore. https://doi.org/10.1007/978-981-13-3459-7_3

  3. Aniche M (2015) Java code metrics calculator (CK). Available in https://github.com/mauricioaniche/ck/

  4. Ardimento P, Aversano L, Bernardi ML, Cimitile M (2020) Temporal convolutional networks for just-in-time software defect prediction. In M. van Sinderen, H. Fill, L.A. Maciaszek (eds.) Proceedings of the 15th International Conference on Software Technologies, ICSOFT 2020, Lieusaint, Paris, France, July 7-9, 2020, pp 384–393. ScitePress. https://doi.org/10.5220/0009890003840393

  5. Ardimento P, Aversano L, Bernardi ML, Cimitile M, Iammarino M (2021) Temporal convolutional networks for just-in-time design smells prediction using fine-grained software metrics. Neurocomput 463:454–471. 10.1016/j.neucom.2021.08.010. https://www.sciencedirect.com/science/article/pii/S0925231221011942

  6. Ardimento P, Bernardi ML, Cimitile M (2018) A multi-source machine learning approach to predict defect prone components. In Proceedings of the 13th International Conference on Software Technologies, ICSOFT 2018, Porto, Portugal, July 26-28, 2018, pp 306–313. https://doi.org/10.5220/0006857803060313

  7. Aversano L, Bernardi ML, Cimitile M, Iammarino M, Romanyuk K (2020) Investigating on the relationships between design smells removals and refactorings. In M. van Sinderen, H. Fill, L.A. Maciaszek (eds.) Proceedings of the 15th International Conference on Software Technologies, ICSOFT 2020, Lieusaint, Paris, France, July 7-9, 2020, pp 212–219. ScitePress. https://doi.org/10.5220/0009887102120219

  8. Bai S, Kolter JZ, Koltun V (2018) An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. CoRR arXiv:1803.01271

  9. Barnett JG, Gathuru CK, Soldano LS, McIntosh S (2016) The relationship between commit message detail and defect proneness in java projects on github. In Proceedings of the 13th International Conference on Mining Software Repositories, MSR 2016, Austin, TX, USA, May 14-22, 2016, pp 496–499. https://doi.org/10.1145/2901739.2903496

  10. Basili VR, Briand LC, Melo WL (1996) A validation of object-oriented design metrics as quality indicators. IEEE Trans Software Eng 22(10):751–761. https://doi.org/10.1109/32.544352

    Article  Google Scholar 

  11. Bengio Y (2000) Gradient-based optimization of hyperparameters. Neural Comput 12(8):1889–1900. https://doi.org/10.1162/089976600300015187

    Article  MathSciNet  Google Scholar 

  12. Bengio Y, Courville A, Vincent P (2014) Representation learning: a review and new perspectives

  13. Bergstra J, Bardenet R, Bengio Y, Kégl B (2011) Algorithms for hyper-parameter optimization. In Proceedings of the 24th International Conference on Neural Information Processing Systems, NIPS’11, p 2546–2554. Curran Associates Inc., Red Hook, NY, USA

  14. Bergstra J, Yamins D, Cox DD (2013) Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures. In Proceedings of the 30th International Conference on International Conference on Machine Learning - Volume 28, ICML’13, p I–115–I–123. JMLR.org

  15. Bernardi M, Cimitile M, Martinelli F, Mercaldo F (2018) Driver and path detection through time-series classification. J Adv Transp 2018. https://doi.org/10.1155/2018/1758731

    Article  Google Scholar 

  16. Bernardi ML, Cimitile M, Martinelli F, Mercaldo F (2019) Keystroke analysis for user identification using deep neural networks. In 2019 International Joint Conference on Neural Networks (IJCNN), pp 1–8. https://doi.org/10.1109/IJCNN.2019.8852068

  17. Bird C, Nagappan N, Murphy B, Gall H, Devanbu PT (2011) Don’t touch my code!: examining the effects of ownership on software quality. In: SIGSOFT/FSE’11 19th ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE-19) and ESEC’11: 13rd European Software Engineering Conference (ESEC-13), Szeged, Hungary, September 5-9, 2011, pp 4–14. ACM

  18. Boucher A, Badri M (2016) Using software metrics thresholds to predict fault-prone classes in object-oriented software. In 2016 4th Intl Conf on Applied Computing and Information Technology/3rd Intl Conf on Computational Science/Intelligence and Applied Informatics/1st Intl Conf on Big Data, Cloud Computing, Data Science Engineering (ACIT-CSII-BCD), pp 169–176. https://doi.org/10.1109/ACIT-CSII-BCD.2016.042

  19. Brito e Abreu F, Melo W (1996) Evaluating the impact of object-oriented design on software quality. In: Proceedings of the 3rd International Software Metrics Symposium, pp 90–99. https://doi.org/10.1109/METRIC.1996.492446

  20. Cabral GG, Minku LL, Shihab E, Mujahid S (2019) Class imbalance evolution and verification latency in just-in-time software defect prediction. In: Proceedings of the 41st International Conference on Software Engineering, ICSE 2019, Montreal, QC, Canada, May 25-31, 2019, pp 666–676. https://doi.org/10.1109/ICSE.2019.00076

  21. Chawla NV (2009) Data mining for imbalanced datasets: an overview. In Data mining and knowledge discovery handbook, pp 875–886. Springer

  22. Chen X, Zhao Y, Wang Q, Yuan Z (2018) Multi: Multi-objective effort-aware just-in-time software defect prediction. Inf Softw Technol 93:1–13

    Article  Google Scholar 

  23. Chidamber SR, Kemerer CF (1994) A metrics suite for object oriented design. IEEE Trans Software Eng 20(6):476–493. https://doi.org/10.1109/32.295895

    Article  Google Scholar 

  24. Chidamber SR, Kemerer CF (1994) A metrics suite for object oriented design. IEEE Trans Softw Eng 20(6):476–493. https://doi.org/10.1109/32.295895

    Article  Google Scholar 

  25. Dabic O, Aghajani E, Bavota G (2021) Sampling projects in github for MSR studies. In: Proceedings of the 18th International Conference on Mining Software Repositories, MSR’21, p. To appear. arXiv:2103.04682

  26. Dam HK, Tran T, Pham TTM, Ng SW, Grundy J, Ghose A (2018) Automatic feature learning for predicting vulnerable software components. IEEE Trans Softw Eng

  27. Ding Z, Xing L (2020) Improved software defect prediction using pruned histogram-based isolation forest. Reliab Eng Syst Saf 204:107170

    Article  Google Scholar 

  28. DAmbros M, Lanza M, Robbes R (2012) Evaluating defect prediction approaches: a benchmark and an extensive comparison. Empir Softw Eng 17(4–5):531–577

  29. Fischer M, Pinzger M, Gall H (2003) Populating a release history database from version control and bug tracking systems. In 19th International Conference on Software Maintenance (ICSM 2003), The Architecture of Existing Systems, 22-26 September 2003, Amsterdam, The Netherlands, p 23. IEEE Computer Society

  30. Graves TL, Karr AF, Marron JS, Siy H (2000) Predicting fault incidence using software change history. IEEE Trans Softw Eng 26(7):653–661

    Article  Google Scholar 

  31. Greiler, M., Herzig, K., Czerwonka, J. (2015) Code ownership and software quality: a replication study. In 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories, pp 2–12. 10.1109/MSR.2015.8

  32. Hall MA (1999) Correlation-based feature selection for machine learning. Ph.D. thesis, Department of Computer Science, University of Waikato, The address of the publisher

  33. Hassan AE (2009) Predicting faults using the complexity of code changes. In 31st International Conference on Software Engineering, ICSE 2009, May 16-24, 2009, Vancouver, Canada, Proceedings, pp 78–88. https://doi.org/10.1109/ICSE.2009.5070510

  34. Hilton R J: Java Source Metrics (2009 (accessed January 16, 2020)). Available in https://github.com/rodhilton/jasome

  35. Hoang T, Dam HK, Kamei Y, Lo D, Ubayashi N (2019) Deepjit: an end-to-end deep learning framework for just-in-time defect prediction. In 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR), pp 34–45. IEEE

  36. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  37. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32Nd International Conference on International Conference on Machine Learning - Volume 37, ICML’15, pp 448–456. JMLR.org. http://dl.acm.org/citation.cfm?id=3045118.3045167

  38. Jahanshahi H, Jothimani D, Başar A, Cevik M (2019) Does chronology matter in jit defect prediction? a partial replication study. In Proceedings of the Fifteenth International Conference on Predictive Models and Data Analytics in Software Engineering, pp 90–99

  39. Jin C (2021) Cross-project software defect prediction based on domain adaptation learning and optimization. Expert Syst Appl 171(114637):1. 10.1016/j.eswa.2021.114637. https://www.sciencedirect.com/science/article/pii/S0957417421000786

  40. Kamei Y, Fukushima T, McIntosh S, Yamashita K, Ubayashi N, Hassan AE (2016) Studying just-in-time defect prediction using cross-project models. Empir Softw Eng 21(5):2072–2106. https://doi.org/10.1007/s10664-015-9400-x

    Article  Google Scholar 

  41. Kamei Y, Shihab E, Adams B, Hassan AE, Mockus A, Sinha A, Ubayashi N (2013) A large-scale empirical study of just-in-time quality assurance. IEEE Trans Softw Eng 39(6):757–773. https://doi.org/10.1109/TSE.2012.70

    Article  Google Scholar 

  42. Khatri Y, Singh SK (2021) Cross project defect prediction: a comprehensive survey with its swot analysis. Innovations in Systems and Software Engineering pp 1–19

  43. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization

  44. Manjula C, Florence L (2019) Deep neural network based hybrid approach for software defect prediction using software metrics. Clust Comput 22(4):9847–9863. https://doi.org/10.1007/s10586-018-1696-z

    Article  Google Scholar 

  45. Mannor S, Peleg D, Rubinstein R (2005) The cross entropy method for classification. In Proceedings of the 22Nd International Conference on Machine Learning, ICML ’05, pp 561–568. ACM, New York, NY, USA

  46. Misra D M: A self regularized non-monotonic neural activation function (arXiv pre-print, 2019)

  47. Mitchell TM (1997) Machine learning, 1st edn. McGraw-Hill Inc, New York, NY, USA

    MATH  Google Scholar 

  48. Mockus A, Weiss DM (2000) Predicting risk of software changes. Bell Labs Techn J 5(2):169–180

    Article  Google Scholar 

  49. Moser R, Pedrycz W, Succi G (2008) Analysis of the reliability of a subset of change metrics for defect prediction. In Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement, pp 309–311

  50. Moser R, Pedrycz W, Succi G (2008) A comparative analysis of the efficiency of change metrics and static code attributes for defect prediction. In 30th International Conference on Software Engineering (ICSE 2008), Leipzig, Germany, May 10-18, 2008, pp 181–190. https://doi.org/10.1145/1368088.1368114

  51. Myers GJ, Sandler C (2004) The art of software testing. Wiley, Hoboken, NJ, USA

    Google Scholar 

  52. Pascarella L, Palomba F, Bacchelli A (2019) Fine-grained just-in-time defect prediction. J Syst Softw 150:22–36. https://doi.org/10.1016/j.jss.2018.12.001

    Article  Google Scholar 

  53. Peters F, Menzies T, Marcus A (2013) Better cross company defect prediction. In 2013 10th Working Conference on Mining Software Repositories (MSR), pp 409–418. IEEE

  54. Phan AV, Nguyen ML, Bui LT (2018) Convolutional neural networks over control flow graphs for software defect prediction. CoRR arXiv:1802.04986

  55. Pickerill P, Jungen HJ, Ochodek M, Mackowiak M, Staron M (2020) PHANTOM: curating github for engineered software projects using time-series clustering. Empir Softw Eng 25(4):2897–2929. https://doi.org/10.1007/s10664-020-09825-8

    Article  Google Scholar 

  56. Porto FR, Simao A (2016) Feature subset selection and instance filtering for cross-project defect prediction-classification and ranking. CLEI Electron J 19(3):4

    Google Scholar 

  57. Rahman F, Devanbu P (2013) How, and why, process metrics are better. In 2013 35th International Conference on Software Engineering (ICSE), pp 432–441. IEEE

  58. Ramachandran P, Zoph B, Le QV (2017) Searching for activation functions. CoRR arXiv:1710.05941

  59. Schaul T, Antonoglou I, Silver D (2013) Unit tests for stochastic optimization

  60. Spinellis D (2005) Tool writing: a forgotten art? (software tools). IEEE Softw 22(4):9–11. https://doi.org/10.1109/MS.2005.111

    Article  Google Scholar 

  61. Staudemeyer RC, Rothstein Morris E (2019) Understanding LSTM – a tutorial into Long Short-Term Memory Recurrent Neural Networks. arXiv e-prints arXiv:1909.09586

  62. Subramanyam R, Krishnan M (2003) Empirical analysis of ck metrics for object-oriented design complexity: Implications for software defects. IEEE Trans Softw Eng 29:297–310. https://doi.org/10.1109/TSE.2003.1191795

    Article  Google Scholar 

  63. Sutskever I, Martens J, Dahl G, Hinton G (2013) On the importance of initialization and momentum in deep learning. In Proceedings of the 30th International Conference on International Conference on Machine Learning - Volume 28, ICML’13, pp III–1139–III–1147. JMLR.org

  64. Vani S, Rao TVM (2019) An experimental approach towards the performance assessment of various optimizers on convolutional neural network. In 2019 3rd International Conference on Trends in Electronics and Informatics (ICOEI), pp 331–336. https://doi.org/10.1109/ICOEI.2019.8862686

  65. Varma S, Simon R (2006) Bias in error estimation when using cross-validation for model selection. BMC Bioinf 7:91. https://doi.org/10.1186/1471-2105-7-91

    Article  Google Scholar 

  66. Wang T, Zhang Z, Jing X, Zhang L (2016) Multiple kernel ensemble learning for software defect prediction. Autom Softw Eng 23(4):569–590. https://doi.org/10.1007/s10515-015-0179-1

    Article  Google Scholar 

  67. Wang Y, Liu J, Mišić J, Mišić VB, Lv S, Chang X (2019) Assessing optimizer impact on DNN model sensitivity to adversarial examples. IEEE Access 7:152766–152776. https://doi.org/10.1109/ACCESS.2019.2948658

    Article  Google Scholar 

  68. Xu Z, Li S, Xu J, Liu J, Luo X, Zhang Y, Zhang T, Keung J, Tang Y (2019) LDFR: learning deep feature representation for software defect prediction. J Syst Softw. https://doi.org/10.1016/j.jss.2019.110402

    Article  Google Scholar 

  69. Yang X, Lo D, Xia X, Sun J (2017) Tlel: A two-layer ensemble learning approach for just-in-time defect prediction. Inf Softw Technol 87:206–220

    Article  Google Scholar 

  70. Yang X, Lo D, Xia X, Zhang Y, Sun J (2015) Deep learning for just-in-time defect prediction. In 2015 IEEE International Conference on Software Quality, Reliability and Security, QRS 2015, Vancouver, BC, Canada, August 3-5, 2015, pp 17–26. https://doi.org/10.1109/QRS.2015.14

  71. Yang Z, Yang D, Dyer C, He X, Smola A, Hovy E (2016) Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp 1480–1489. Association for Computational Linguistics, San Diego, California. https://doi.org/10.18653/v1/N16-1174. https://www.aclweb.org/anthology/N16-1174

  72. Young S, Abdou T, Bener A (2018) A replication study: just-in-time defect prediction with ensemble learning. In Proceedings of the 6th International Workshop on Realizing Artificial Intelligence Synergies in Software Engineering, pp 42–47

  73. Zimmermann T, Nagappan N, Gall H, Giger E, Murphy B (2009) Cross-project defect prediction: a large scale experiment on data vs. domain vs. process. In Proceedings of the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering, pp 91–100

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Bari Aldo Moro, Bari, Italy

    Pasquale Ardimento

  2. Department of Engineering, University of Sannio, Benevento, Italy

    Lerina Aversano, Mario Luca Bernardi & Martina Iammarino

  3. Department of Law and Economics, Unitelma Sapienza of Rome, Rome, Italy

    Marta Cimitile

Authors
  1. Pasquale Ardimento
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lerina Aversano
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mario Luca Bernardi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marta Cimitile
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Martina Iammarino
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pasquale Ardimento.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ardimento, P., Aversano, L., Bernardi, M.L. et al. Just-in-time software defect prediction using deep temporal convolutional networks. Neural Comput & Applic 34, 3981–4001 (2022). https://doi.org/10.1007/s00521-021-06659-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06659-3

Keywords

Search

Navigation