Skip to main content
SpringerLink
Log in

Defect prediction using deep learning with Network Portrait Divergence for software evolution

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Understanding software evolution is essential for software development tasks, including debugging, maintenance, and testing. As a software system evolves, it grows in size and becomes more complex, hindering its comprehension. Researchers proposed several approaches for software quality analysis based on software metrics. One of the primary practices is predicting defects across software components in the codebase to improve agile product quality. While several software metrics exist, graph-based metrics have rarely been utilized in software quality. In this paper, we explore recent network comparison advancements to characterize software evolution and focus on aiding software metrics analysis and defect prediction. We support our approach with an automated tool named GraphEvoDef. Particularly, GraphEvoDef provides three major contributions: (1) detecting and visualizing significant events in software evolution using call graphs, (2) extracting metrics that are suitable for software comprehension, and (3) detecting and estimating the number of defects in a given code entity (e.g., class). One of our major findings is the usefulness of the Network Portrait Divergence metric, borrowed from the information theory domain, to aid the understanding of software evolution. To validate our approach, we examined 29 different open-source Java projects from GitHub and then demonstrated the proposed approach using 9 use cases with defect data from the the PROMISE dataset. We also trained and evaluated defect prediction models for both classification and regression tasks. Our proposed technique has an 18% reduction in the mean square error and a 48% increase in squared correlation coefficient over the state-of-the-art approaches in the defect prediction domain.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Akiyama F (1971) An example of software system debugging. In: IFIP congress (1), vol 71, pp 353–359

  • Akoglu L, Tong H, Koutra D (2015) Graph based anomaly detection and description: a survey. Data Min Knowl Discov 29(3):626–688

    Article  MathSciNet  Google Scholar 

  • Alanazi R, Gharibi G, Lee Y (2021) Facilitating program comprehension with call graph multilevel hierarchical abstractions. J Syst Softw 176:110945. https://doi.org/10.1016/j.jss.2021.110945. https://www.sciencedirect.com/science/article/pii/S016412122100042X

    Article  Google Scholar 

  • Allamanis M, Barr ET, Devanbu P, Sutton C (2018) A survey of machine learning for big code and naturalness. ACM Comput Surv (CSUR) 51(4):1–37

    Article  Google Scholar 

  • Alon U, Zilberstein M, Levy O, Yahav E (2019) code2vec: learning distributed representations of code. Proc ACM Program Lang 3(POPL):1–29

    Article  Google Scholar 

  • Alsaeedi A, Khan MZ (2019) Software defect prediction using supervised machine learning and ensemble techniques: a comparative study. J Softw Eng Appl 12(5):85–100

    Article  Google Scholar 

  • Alsolai H, Roper M (2020) A systematic literature review of machine learning techniques for software maintainability prediction. Inf Softw Technol 119:106214. https://doi.org/10.1016/j.infsof.2019.106214. https://www.sciencedirect.com/science/article/pii/S0950584919302228

    Article  Google Scholar 

  • Arshad A, Riaz S, Jiao L (2019) Semi-supervised deep fuzzy c-mean clustering for imbalanced multi-class classification. IEEE Access 7:28100–28112

    Article  Google Scholar 

  • Bagrow JP, Bollt EM (2019) An information-theoretic, all-scales approach to comparing networks. Appl Netw Sci 4(1):1–15

    Article  Google Scholar 

  • Bell RM, Ostrand TJ, Weyuker EJ (2011) Does measuring code change improve fault prediction?. In: Proceedings of the 7th international conference on predictive models in software engineering, association for computing machinery Promise ’11. New York. https://doi.org/10.1145/2020390.2020392

  • Bhattacharya P, Iliofotou M, Neamtiu I, Faloutsos M (2012) Graph-based analysis and prediction for software evolution. In: Proceedings of the 34th international conference on software engineering, ICSE ’12. http://dl.acm.org/citation.cfm?id=2337223.2337273. IEEE Press, Piscataway, pp 419–429

  • BigQuery G (2018) https://cloud.google.com/bigquery/

  • Camel A (2019) https://camel.apache.org/

  • Cao Y, Ding Z, Xue F, Rong X (2018) An improved twin support vector machine based on multi-objective cuckoo search for software defect prediction. Int J Bio-Inspired Comput 11(4):282–291

    Article  Google Scholar 

  • Cassandra A (2019) https://junit.org/junit4/index.html

  • Chen M, Ma Y (2015) An empirical study on predicting defect numbers. In: SEKE, pp 397–402

  • CISQ HK (2018) The cost of poor quality software in the us: a 2018 report. https://www.it-cisq.org/the-cost-of-poor-quality-software-in-the-us-a-2018-report/The-Cost-of-Poor-Quality-Software-in-the-US-2018-Report.pdf

  • Concas G, Marchesi M, Pinna S, Serra N (2007) Power-laws in a large object-oriented software system. IEEE Trans Softw Eng 33(10):687–708

    Article  Google Scholar 

  • Ellson J, Gansner E, Koutsofios L, North SC, Woodhull G (2001) Graphviz—open source graph drawing tools. In: International symposium on graph drawing. Springer, pp 483–484

  • Ferenc R, Tóth Z, Ladányi G, Siket I, Gyimóthy T (2018) A public unified bug dataset for java. In: Proceedings of the 14th international conference on predictive models and data analytics in software engineering, PROMISE’18. https://doi.org/10.1145/3273934.3273936. Association for Computing Machinery, New York, pp 12–21

  • Fernández-Sáez AM, Chaudron MR, Genero M (2018) An industrial case study on the use of uml in software maintenance and its perceived benefits and hurdles. Empir Softw Eng 1–65

  • Gansner ER, North SC (2000) An open graph visualization system and its applications to software engineering. Softw: Pract Exp 30(11):1203–1233

    MATH  Google Scholar 

  • Gao X, Xiao B, Tao D, Li X (2010) A survey of graph edit distance. Pattern Anal Appl 13(1):113–129

    Article  MathSciNet  MATH  Google Scholar 

  • Gharibi G, Alanazi R, Lee Y (2018a) Automatic hierarchical clustering of static call graphs for program comprehension. In: 2018 IEEE international conference on big data (big data). IEEE, pp 4016–4025

  • Gharibi G, Tripathi R, Lee Y (2018b) Code2graph: automatic generation of static call graphs for python source code. In: Proceedings of the 33rd ACM/IEEE international conference on automated software engineering, ASE 2018. https://doi.org/10.1145/3238147.3240484. Association for Computing Machinery, New York, pp 880–883

  • Gharibi G, Tripathi R, Lee Y (2018c) Code2graph: automatic generation of static call graphs for python source code. In: Proceedings of the 33rd ACM/IEEE international conference on automated software engineering, pp 880–883

  • Gharibi G, Walunj V, Rella S, Lee Y (2019) Modelkb: towards automated management of the modeling lifecycle in deep learning. In: 2019 IEEE/ACM 7th international workshop on realizing artificial intelligence synergies in software engineering (RAISE). IEEE, pp 28–34

  • Gharibi G, Walunj V, Nekadi R, Marri R, Lee Y (2021) Automated end-to-end management of the modeling lifecycle in deep learning. Empir Softw Eng 26(2):1–33

    Article  Google Scholar 

  • Ghezzi C, Jazayeri M, Mandrioli D (2002) Fundamentals of software engineering. Prentice Hall PTR

  • Grove D, DeFouw G, Dean J, Chambers C (1997) Call graph construction in object-oriented languages. ACM SIGPLAN Not 32(10):108–124

    Article  Google Scholar 

  • Gu X, Zhang H, Kim S (2018) Deep code search. In: 2018 IEEE/ACM 40th international conference on software engineering (ICSE), pp 933–944

  • Guo L, Lei Y, Xing S, Yan T, Li N (2019) Deep convolutional transfer learning network: a new method for intelligent fault diagnosis of machines with unlabeled data. IEEE Trans Ind Electron 66(9):7316–7325

    Article  Google Scholar 

  • Hall T, Beecham S, Bowes D, Gray D, Counsell S (2011) A systematic literature review on fault prediction performance in software engineering. IEEE Trans Softw Eng 38(6):1276–1304

    Article  Google Scholar 

  • Hamming RW (1950) Error detecting and error correcting codes. Bell Syst Tech J 29(2):147–160

    Article  MathSciNet  MATH  Google Scholar 

  • Hammouri A, Hammad M, Alnabhan M, Alsarayrah F (2018) Software bug prediction using machine learning approach. Int J Adv Comput Sci Appl 9(2):78–83

    Google Scholar 

  • Hartle H, Klein B, McCabe S, Daniels A, St-Onge G, Murphy C, Hébert-Dufresne L (2020) Network comparison and the within-ensemble graph distance. Proc R Soc A 476(2243):20190744

    Article  MathSciNet  MATH  Google Scholar 

  • Huda S, Alyahya S, Ali MM, Ahmad S, Abawajy J, Al-Dossari H, Yearwood J (2017) A framework for software defect prediction and metric selection. IEEE Access 6:2844–2858

    Article  Google Scholar 

  • Ikehara K, Clauset A (2017) Characterizing the structural diversity of complex networks across domains. arXiv:171011304

  • Jaccard P (1901) Etude de la distribution florale dans une portion des alpes et du jura. Bull Soc Vaudoise Sci Nat 37:547–579. https://doi.org/10.5169/seals-266450

    Google Scholar 

  • JUnit (2019) https://junit.org/junit4/index.html

  • Kamei Y, Shihab E (2016) Defect prediction: accomplishments and future challenges. In: 2016 IEEE 23rd international conference on software analysis, evolution, and reengineering (SANER), vol 5, pp 33–45

  • Koutra D, Shah N, Vogelstein JT, Gallagher B, Faloutsos C (2016) Deltacon: principled massive-graph similarity function with attribution. ACM Trans Knowl Discov Data (TKDD) 10(3):1–43

    Article  Google Scholar 

  • Krishnan S, Strasburg C, Lutz R R, Goševa-Popstojanova K (2011) Are change metrics good predictors for an evolving software product line?. In: Proceedings of the 7th international conference on predictive models in software engineering, Promise ’11. https://doi.org/10.1145/2020390.2020397. Association for Computing Machinery, New York

  • Krüger J (2019) Tackling knowledge needs during software evolution. In: Proceedings of the 2019 27th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, pp 1244–1246

  • Kuchaiev O, Pržulj N (2011) Integrative network alignment reveals large regions of global network similarity in yeast and human. Bioinformatics 27(10):1390–1396

    Article  Google Scholar 

  • Kuchaiev O, Milenković T, Memišević V, Hayes W, Pržulj N (2010) Topological network alignment uncovers biological function and phylogeny. J R Soc Interface 7(50):1341–1354

    Article  Google Scholar 

  • Lance GN, Williams WT (1966) Computer programs for hierarchical polythetic classification (“similarity analyses”). Comput J 9(1):60–64

    Article  MATH  Google Scholar 

  • Le Novere N (2015) Quantitative and logic modelling of molecular and gene networks. Nat Rev Genet 16(3):146

    Article  Google Scholar 

  • Li Z, Jing XY, Zhu X (2018) Progress on approaches to software defect prediction. IET Softw 12(3):161–175

    Article  Google Scholar 

  • Lyu MR et al (1996) Handbook of software reliability engineering, vol 222. IEEE Computer Society Press, California

    Google Scholar 

  • Manjula C, Florence L (2019) Deep neural network based hybrid approach for software defect prediction using software metrics. Clust Comput 22(4):9847–9863

    Article  Google Scholar 

  • Matsumoto S, Kamei Y, Monden A, Matsumoto K, Nakamura M (2010) An analysis of developer metrics for fault prediction. In: Proceedings of the 6th international conference on predictive models in software engineering, PROMISE ’10. https://doi.org/10.1145/1868328.1868356. Association for Computing Machinery, New York

  • McCabe S, Torres L, LaRock T, Haque SA, Yang CH, Hartle H, Klein B (2020) netrd: a library for network reconstruction and graph distances. arXiv:201016019

  • Menzies T, Milton Z, Turhan B, Cukic B, Jiang Y, Bener A (2010) Defect prediction from static code features: current results, limitations, new approaches. Autom Softw Eng 17(4):375–407

    Article  Google Scholar 

  • Milo R, Shen-Orr S, Itzkovitz S, Kashtan N, Chklovskii D, Alon U (2002) Network motifs: simple building blocks of complex networks. Science 298 (5594):824–827

    Article  Google Scholar 

  • Moser R, Pedrycz W, Succi G (2008) A comparative analysis of the efficiency of change metrics and static code attributes for defect prediction. In: 2008 ACM/IEEE 30th international conference on software engineering, pp 181–190

  • Myers C R (2003) Software systems as complex networks: Structure, function, and evolvability of software collaboration graphs. Phys Rev E 68(4):046116

    Article  Google Scholar 

  • Nagappan N, Zeller A, Zimmermann T, Herzig K, Murphy B (2010) Change bursts as defect predictors. In: 2010 IEEE 21st international symposium on software reliability engineering, pp 309–318

  • Premraj R, Herzig K (2011) Network versus code metrics to predict defects: a replication study. In: 2011 International symposium on empirical software engineering and measurement, pp 215–224

  • Qiao L, Li X, Umer Q, Guo P (2020) Deep learning based software defect prediction. Neurocomputing 385:100–110

    Article  Google Scholar 

  • Rahman F, Devanbu P (2013) How, and why, process metrics are better. In: 2013 35th International conference on software engineering (ICSE). IEEE, pp 432–441

  • Rahman M, Palani D, Rigby PC (2019) Natural software revisited. In: 2019 IEEE/ACM 41st international conference on software engineering (ICSE). IEEE, pp 37–48

  • Rathore SS, Kumar S (2016) A decision tree regression based approach for the number of software faults prediction. ACM SIGSOFT Softw Eng Notes 41(1):1–6

    Article  Google Scholar 

  • Rathore S S, Kumar S (2017) An empirical study of some software fault prediction techniques for the number of faults prediction. Soft Comput 21(24):7417–7434

    Article  Google Scholar 

  • Rathore SS, Kumar S (2019) A study on software fault prediction techniques. Artif Intell Rev 51(2):255– 327

    Article  Google Scholar 

  • Russo B (2018) Profiling call changes via motif mining. In: Proceedings of the 15th international conference on mining software repositories, MSR ’18. https://doi.org/10.1145/3196398.3196426. Association for Computing Machinery, New York, pp 203–214

  • Sarajlić A, Malod-Dognin N, Yaveroğlu ÖN, Pržulj N (2016) Graphlet-based characterization of directed networks. Sci Rep 6(1):1–14

    Article  Google Scholar 

  • Savić M, Ivanović M, Radovanović M (2017) Analysis of high structural class coupling in object-oriented software systems. Computing 99(11):1055–1079

    Article  MathSciNet  MATH  Google Scholar 

  • Savić M, Ivanović M, Jain LC (2019) Analysis of software networks. In: Complex networks in software, knowledge, and social systems. Springer, pp 59–141

  • Scott DW (2015) Multivariate density estimation: theory, practice, and visualization. Wiley, New York

    MATH  Google Scholar 

  • Shirabad JS, Menzies TJ (2005) The promise repository of software engineering databases. School of Information Technology and Engineering, University of Ottawa, Canada 24

  • Silverman BW (2018) Density estimation for statistics and data analysis. Routledge

  • Song Q, Jia Z, Shepperd M, Ying S, Liu J (2011) A general software defect-proneness prediction framework. IEEE Trans Softw Eng 37(3):356–370

    Article  Google Scholar 

  • Spinellis D JM (2018) http://gromit.iiar.pwr.wroc.pl/pinf/ckjm/metric.html

  • Stone L, Simberloff D, Artzy-Randrup Y (2019) Network motifs and their origins. PLoS Comput Biol 15(4):e1006749

    Article  Google Scholar 

  • Subramanyam R, Krishnan MS (2003) Empirical analysis of ck metrics for object-oriented design complexity: implications for software defects. IEEE Trans Softw Eng 29(4):297–310

    Article  Google Scholar 

  • Tantardini M, Ieva F, Tajoli L, Piccardi C (2019) Comparing methods for comparing networks. Sci Rep 9(1):1–19

    Article  Google Scholar 

  • Valverde S, Solé RV (2003) Hierarchical small worlds in software architecture. arXiv:cond-mat/0307278

  • Vapnik V, Golowich SE, Smola A, et al (1997) Support vector method for function approximation, regression estimation, and signal processing. Advances in neural information processing systems, pp 281– 287

  • Vasa R, Schneider JG, Nierstrasz O (2007) The inevitable stability of software change. In: IEEE international conference on software maintenance, 2007. ICSM 2007. IEEE, pp 4–13

  • Walunj V, Gharibi G, Ho DH, Lee Y (2019) Graphevo: characterizing and understanding software evolution using call graphs. In: 2019 IEEE international conference on big data (big data), pp 4799–4807

  • Wang L, Wang Z, Yang C, Zhang L, Ye Q (2009) Linux kernels as complex networks: a novel method to study evolution. In: IEEE international conference on software maintenance, 2009. ICSM 2009. IEEE, pp 41–50

  • Wang Y, Wen S, Xiang Y, Zhou W (2014) Modeling the propagation of worms in networks: a survey. IEEE Commun Surv Tutor 16(2):942–960

    Article  Google Scholar 

  • White M, Tufano M, Vendome C, Poshyvanyk D (2016) Deep learning code fragments for code clone detection. In: Proceedings of the 31st IEEE/ACM international conference on automated software engineering, ASE 2016. https://doi.org/10.1145/2970276.2970326. Association for Computing Machinery, New York, pp 87–98

  • Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A (2012) Experimentation in software engineering. Springer Science & Business Media

  • Xia X, Bao L, Lo D, Xing Z, Hassan AE, Li S (2017) Measuring program comprehension: a large-scale field study with professionals. IEEE Trans Softw Eng 44(10):951–976

    Article  Google Scholar 

  • Yao Y, Liu Y, Huang S, Chen H, Liu J, Yang F (2020) Cross-project dynamic defect prediction model for crowdsourced test. In: 2020 IEEE 20th international conference on software quality, reliability and security (QRS). https://doi.org/10.1109/QRS51102.2020.00040, pp 223–230

  • Zhou Y., Leung H (2006) Empirical analysis of object-oriented design metrics for predicting high and low severity faults. IEEE Trans Softw Eng 32 (10):771–789

    Article  Google Scholar 

  • Zhang H (2009) An investigation of the relationships between lines of code and defects. In: 2009 IEEE international conference on software maintenance, pp 274–283

  • Zhang W, Du Y, Yoshida T, Wang Q, Li X (2018) Samen-svr: using sample entropy and support vector regression for bug number prediction. IET Softw 12(3):183–189

    Article  Google Scholar 

  • Zhao G, Huang J (2018) Deepsim: deep learning code functional similarity. In: Proceedings of the 2018 26th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, ESEC/FSE 2018. https://doi.org/10.1145/3236024.3236068. Association for Computing Machinery, New York, pp 141–151

  • Zimmermann T, Nagappan N (2008) Predicting defects using network analysis on dependency graphs. In: Proceedings of the 30th international conference on software engineering, ICSE ’08. https://doi.org/10.1145/1368088.1368161. Association for Computing Machinery, New York, pp 531–540

  • Zimmermann T, Nagappan N (2009) Predicting defects with program dependencies. In: ESEM ’09. https://doi.org/10.1109/ESEM.2009.5316024. IEEE Computer Society, pp 435–438

  • Zimmermann T, Premraj R, Zeller A (2007) Predicting defects for eclipse. In: Third international workshop on predictor models in software engineering (PROMISE’07: ICSE workshops 2007), pp 9–9

  • Zookeeper A (2019) https://zookeeper.apache.org/

Download references

Acknowledgements

We would like to thank Duy Ho for his help in some implementation parts in the early versions of GraphEvo. We also thank the anonymous reviewers for their time and effort reviewing this work. The first author thanks Mattia Tantardini for sharing the network comparisons study code-base. The co-author, Yugyung Lee, would like to acknowledge the partial support of the NSF, USA Grant No. 1747751, 1935076, and 1951971.

Author information

Authors and Affiliations

  1. School of Computing and Engineering, University of Missouri-Kansas City, Kansas City, MO, USA

    Vijay Walunj, Gharib Gharibi, Rakan Alanazi & Yugyung Lee

  2. Faculty of Computing and Information Technology, Northern Border University, Rafha, Saudi Arabia

    Rakan Alanazi

Authors
  1. Vijay Walunj
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gharib Gharibi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rakan Alanazi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yugyung Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vijay Walunj.

Ethics declarations

Conflict of Interest

The authors whose names are listed immediately below certify that they have NO affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational grants; participation in speakers? bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements), or non-financial interest (such as personal or professional relationships, affiliations, knowledge or beliefs) in the subject matter or materials discussed in this manuscript.

Additional information

Communicated by: Foutse Khomh, Gemma Catolino and Pasquale Salza

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Machine Learning Techniques for Software Quality Evaluation (MaLTeSQuE)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Walunj, V., Gharibi, G., Alanazi, R. et al. Defect prediction using deep learning with Network Portrait Divergence for software evolution. Empir Software Eng 27, 118 (2022). https://doi.org/10.1007/s10664-022-10147-0

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10664-022-10147-0

Keywords

Search

Navigation