Skip to main content
SpringerLink
Log in

An Empirical Study on Data Sampling for Just-in-Time Defect Prediction

  • Conference paper
  • First Online:
Artificial Intelligence and Security (ICAIS 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12737))

Included in the following conference series:

Abstract

In this paper, the impact of Data Sampling on Just-in-Time defect prediction is explored. We find that there is a significant negative relationship between the class imbalance ratio of the dataset and the performance of the instant software defect prediction model. Secondly although most software defect data are not as unbalanced as expected, a moderate degree of imbalance is sufficient to affect the performance of traditional learning. This means that if the training data for immediate software defects show moderate or more severe imbalances, one need not expect good defect prediction performance and the data sampling approach to balancing the training data can improve the performance of the model. Finally, the empirical approach shows that although the under-sampling method slightly improves model performance, the different sampling methods do not have a substantial impact on the evaluation of immediate software defect prediction models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Atluri, S.N., Shen, S.: Global weak forms, weighted residuals, finite elements, boundary elements & local weak forms. In: The Meshless Local Petrov-Galerkin (MLPG) Method, 1st edn, vol. 1, pp. 15–64. Tech Science Press, Henderson (2004)

    Google Scholar 

  2. Kamei, Y., Shihab, E.: Defect prediction: accomplishments and future challenges. In: Proceedings of the 23rd International Conference on Software Analysis, Evolution, and Reengineering. IEEE, Washington (2016)

    Google Scholar 

  3. Kamei, Y., et al.: A large-scale empirical study of just-in-time quality assurance. IEEE Trans. Software Eng. 39(6), 757–773 (2013)

    Article  Google Scholar 

  4. Mockus, A., Weiss, D.M.: Predicting risk of software changes. Bell Labs Tech. J. 5(2), 169–180 (2000)

    Article  Google Scholar 

  5. Kim, S., Whitehead, E.J., Zhang, Y.: Classifying software changes: clean or buggy? IEEE Trans. Software Eng. 34(2), 181–196 (2008)

    Article  Google Scholar 

  6. Yang, Y., et al.: Effort-aware just-in-time defect prediction: simple unsupervised models could be better than supervised models. In: Proceedings of the 24th International Symposium on Foundations of Software Engineering. ACM Press, New York (2016)

    Google Scholar 

  7. Li, H., Zhou, C., Haitao, X., Lv, X., Han, Z.: Joint optimization strategy of computation offloading and resource allocation in multi-access edge computing environment. IEEE Trans. Veh. Technol. 69(9), 10214–10226 (2020)

    Article  Google Scholar 

  8. Jiang, T., Tan, L., Kim, S.: Personalized defect prediction. In: Proceedings of the 28th International Conference on Automated Software Engineering. IEEE, Washington (2013)

    Google Scholar 

  9. Shivaji, S., Whitehead, E.J., Akella, R., Kim, S.: Reducing features to improve code change-based bug prediction. IEEE Trans. Software Eng. 9(4), 552–569 (2013)

    Article  Google Scholar 

  10. Cabral, G.G., Minku, L.L., Shihab, E., Mujahid, S.: Class imbalance evolution and verification latency in just-in-time software defect prediction. In: 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE), Montreal, QC, Canada (2019)

    Google Scholar 

  11. Catolino, G., Di Nucci, D., Ferrucci, F.: Cross-project just-in-time bug prediction for mobile apps: an empirical assessment. In: 2019 IEEE/ACM 6th International Conference on Mobile Software Engineering and Systems (MOBILESoft), Montreal, QC, Canada (2019)

    Google Scholar 

  12. Fukushima, T., Kamei, Y., McIntosh, S., Yamashita, K., Ubayashi, N.: An empirical study of just-in-time defect prediction using cross-project models. In: Proceedings of the 11th Working Conference on Mining Software Repositories, New York (2014)

    Google Scholar 

  13. Tan, M., Tan, L., Dara, S., Mayeux, C.: Online defect prediction for imbalanced data. In: Proceedings of the 37th International Conference on Software Engineering. IEEE, Washington (2015)

    Google Scholar 

  14. Kamei, Y., Fukushima, T., McIntosh, S., Yamashita, K., Ubayashi, N., Hassan, A.E.: Studying just-in-time defect prediction using cross project models. Empir. Softw. Eng. 21(5), 2072–2106 (2016)

    Article  Google Scholar 

  15. Yang, X., Lo, D., Xia, X., Zhang, Y., Sun, J.: Deep learning for just-in-time defect prediction. In: Proceedings of the 15th International Conference on Software Quality, Reliability and Security. IEEE, Washington (2015)

    Google Scholar 

  16. Huang, Q., Xia, X., Lo, D.: Supervised vs unsupervised models: a holistic look at effort-aware just-in-time defect prediction. In: Proceedings of the 33rd International Conference on Software Maintenance and Evolution. IEEE, Washington (2017)

    Google Scholar 

  17. Fu, W., Menzies, T.: Revisiting unsupervised learning for defect prediction. In: Proceedings of the 25th International Symposium on Foundations of Software Engineering. ACM Press, New York (2017)

    Google Scholar 

  18. McIntosh, S., Kamei, Y.: Are fix-inducing changes a moving target? A longitudinal case study of just-in-time defect prediction. IEEE Trans. Software Eng. 44(5), 412–428 (2018)

    Article  Google Scholar 

  19. Wilcoxon, F.: Individual comparisons by ranking methods. Biometrics Bull. 1, 80–83 (1945)

    Article  Google Scholar 

  20. Cliff, N.: Ordinal Methods for Behavioral Data Analysis. Psychology Press, New York (1996)

    Google Scholar 

  21. Tantithamthavorn, C., Hassan, A.E., Matsumoto, K.: The impact of class rebalancing techniques on the performance and interpretation of defect prediction models. IEEE Trans. Software Eng. 46(11), 1200–1219 (2018)

    Article  Google Scholar 

  22. Galar, M., Fernandez, A., Tartas, E.B., Sola, H.B., Herrera, F.: A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Trans. Syst. Man Cybern. Part C 42(4), 463–484 (2012)

    Article  Google Scholar 

  23. Batista, G., Prati, R., Monard, M.: A study of the behavior of several methods for balancing machine learning training data. ACM SIGKDD Explor. Newsl. 6, 20–29 (2004)

    Article  Google Scholar 

  24. Chawla, N., Bowyer, K., Hall, L., Kegelmeyer, P.: Smote: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)

    Article  Google Scholar 

  25. Song, Q., Guo, Y., Shepperd, M.: A comprehensive investigation of the role of imbalanced learning for software defect prediction. IEEE Trans. Software Eng. 45, 1253–1269 (2019)

    Article  Google Scholar 

Download references

Acknowledgement

This work is supported by the National Key R&D Program of China (No. 2018YFB1003905) and the National Natural Science Foundation of China under Grant (No. 61971032), Fundamental Research Funds for the Central Universities (No. FRF-TP-18-008A3).

Author information

Authors and Affiliations

  1. University of Science and Technology Beijing, Beijing, 100083, China

    Haitao Xu, Ruifeng Duan & Shengsong Yang

  2. Systems Engineering Institute AMS, Beijing, 100071, China

    Lei Guo

Authors
  1. Haitao Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruifeng Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shengsong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lei Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Nanjing University of Information Science and Technology, Nanjing, China

    Xingming Sun

  2. Nanjing University of Information Science and Technology, Nanjing, China

    Xiaorui Zhang

  3. Jinan University, Guangzhou, China

    Zhihua Xia

  4. Purdue University, West Lafayette, IN, USA

    Elisa Bertino

Ethics declarations

The authors declare that they have no conflicts of interest to report regarding the present study.The authors declare that they have no conflicts of interest to report regarding the present study.

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xu, H., Duan, R., Yang, S., Guo, L. (2021). An Empirical Study on Data Sampling for Just-in-Time Defect Prediction. In: Sun, X., Zhang, X., Xia, Z., Bertino, E. (eds) Artificial Intelligence and Security. ICAIS 2021. Lecture Notes in Computer Science(), vol 12737. Springer, Cham. https://doi.org/10.1007/978-3-030-78612-0_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-78612-0_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-78611-3

  • Online ISBN: 978-3-030-78612-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation