Skip to main content
Springer Nature Link
Log in

An Empirical Analysis on Just-In-Time Defect Prediction Models for Self-driving Software Systems

  • Conference paper
  • First Online:
Current Trends in Web Engineering (ICWE 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1668))

Included in the following conference series:

  • 510 Accesses

Abstract

Just-in-time (JIT) defect prediction has been used to predict whether a code change is defective or not. Existing JIT prediction has been applied to different kind of open-source software platform for cloud computing, but JIT defect prediction has never been applied in self-driving software. Unlike other software systems, self-driving system is an AI-enabled system and is a representative system to which edge cloud service is applied. Therefore, we aim to identify whether the existing JIT defect prediction models for traditional software systems also work well for self-driving software. To this end, we collect and label the dataset of open-source self-driving software project using SZZ (Śliwerski, Zimmermann and Zeller) algorithm. And we select four traditional machine learning methods and state-of-the-art research (i.e., JIT-Line) as our baselines and compare their prediction performance. Our experimental results show that JITLine and logistic regression produce superior performance, however, there exists a room to be improved. Through XAI (Explainable AI) analysis it turned out that the prediction performance is mainly affected by experience and history-related features among change-level metrics. Our study is expected to provide important insight for practitioners and subsequent researchers performing defect prediction in AI-enabled system.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Jiang, L., Jiang, S., Gong, L., Dong, Y., Yu, Q.: Which process metrics are significantly important to change of defects in evolving projects: an empirical study. IEEE Access 8, 93705–93722 (2020)

    Article  Google Scholar 

  2. Wang, S., Liu, T., Nam, J., Tan, L.: Deep semantic feature learning for software defect prediction. IEEE Trans. Softw. Eng. 46(12), 1267–1293 (2018)

    Article  Google Scholar 

  3. Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. Adv. Neural Inf. Process. Syst. 30, 4765–4774 (2017)

    Google Scholar 

  4. Ni, C., Xia, X., Lo, D., Yang, X., Hassan, A.E.: just-in-time defect prediction on JavaScript projects: a replication study (2020)

    Google Scholar 

  5. Garcia, J., Feng, Y., Shen, J., Almanee, S., Xia, Y., Chen, A.Q.A.: A comprehensive study of autonomous vehicle bugs. In: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, pp. 385–396 (2020)

    Google Scholar 

  6. Hoang, T., Dam, H.K., Kamei, Y., Lo, D., Ubayashi, N.: DeepJIT: an end-to-end deep learning framework for just-in-time defect prediction. In 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR) (pp. 34–45). IEEE (2019)

    Google Scholar 

  7. Hoang, T., Kang, H. J., Lo, D., Lawall, J.: CC2vec: distributed representations of code changes. In: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, pp. 518–529 (2020)

    Google Scholar 

  8. Pornprasit, C., Tantithamthavorn, C.K.: JITLine: a simpler, better, faster, finer-grained just-in-time defect prediction. In: 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR), pp. 369–379. IEEE (2021)

    Google Scholar 

  9. Matsumoto, S., Kamei, Y., Monden, A., Matsumoto, K.I., Nakamura, M.: An analysis of developer metrics for fault prediction. In: Proceedings of the 6th International Conference on Predictive Models in Software Engineering, pp. 1–9 (2010)

    Google Scholar 

  10. Altinger, H., Siegl, S., Dajsuren, Y., Wotawa, F.: A novel industry grade dataset for fault prediction based on model-driven developed automotive embedded software. In: 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories, pp. 494–497. IEEE (2015)

    Google Scholar 

  11. Altinger, H., Herbold, S., Schneemann, F., Grabowski, J., Wotawa, F.: Performance tuning for automotive software fault prediction. In: 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER), pp. 526–530. IEEE (2017)

    Google Scholar 

  12. Da Costa, D.A., McIntosh, S., Shang, W., Kulesza, U., Coelho, R., Hassan, A.E.: A framework for evaluating the results of the SZZ approach for identifying bug-introducing changes. IEEE Trans. Softw. Eng. 43(7), 641–657 (2016)

    Article  Google Scholar 

  13. Peng, Z., Yang, J., Chen, T.H., Ma, L.: A first look at the integration of machine learning models in complex autonomous driving systems: a case study on apollo. In: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 1240–1250 (2020)

    Google Scholar 

  14. Lou, G.: Testing of autonomous driving systems: where are we and where should we go? (2021)

    Google Scholar 

  15. Moser, R., Pedrycz, W., & Succi, G.: A comparative analysis of the efficiency of change metrics and static code attributes for defect prediction. In Proceedings of the 30th International Conference on Software Engineering, pp. 181–190 (2008)

    Google Scholar 

  16. Rosa, G., et al.: Evaluating SZZ implementations through a developer-informed oracle. In: 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), pp. 436–447. IEEE (2021)

    Google Scholar 

  17. Fan, Y., Xia, X., da Costa, D.A., Lo, D., Hassan, A.E., Li, S.: The impact of mislabeled changes by SZZ on just-in-time defect prediction. IEEE Trans. Softw. Eng. 47(8), 1559–1586 (2019)

    Article  Google Scholar 

  18. Kamei, Y., et al.: A large-scale empirical study of just-in-time quality assurance. IEEE Trans. Softw. Eng. 39(6), 757–773 (2012)

    Article  Google Scholar 

  19. Kim, S., Zimmermann, T., Pan, K., James Jr., E.: Automatic identification of bug-introducing changes. In: 21st IEEE/ACM international conference on automated software engineering (ASE 2006), pp. 81–90. IEEE (2006)

    Google Scholar 

  20. Sawilowsky, S.S.: New effect size rules of thumb. J. Mod. Appl. Stat. Methods 8(2), 26 (2009)

    Article  Google Scholar 

  21. Zeng, Z., Zhang, Y., Zhang, H., Zhang, L.: Deep just-in-time defect prediction: how far are we? In: Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 427–438 (2021)

    Google Scholar 

  22. Sasaki, Y., et al.: An edge-cloud computing model for autonomous vehicles. In: 11th IROS Workshop on Planning, Perception, Navigation for Intelligent Vehicle (2019)

    Google Scholar 

Download references

Acknowledgement

This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2022R1I1A3069233) and the Nuclear Safety Research Program through the Korea Foundation Of Nuclear Safety (KoFONS) using the financial resource granted by the Nuclear Safety and Security Commission(NSSC) of the Republic of Korea (No. 2105030) and the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2022-2020-0-01795) supervised by the IITP (Institute of Information & Communications Technology Planning & Evaluation) and NRF grant funded by the Korea government (MSIT) (No. NRF-2020R1F1A1071888).

Author information

Authors and Affiliations

  1. Department of Software Engineering, Jeonbuk National University, Jeonju, Korea

    Jiwon Choi, Saranya Manikandan & Duksan Ryu

  2. School of Computing, Korea Advanced Institute of Science and Technology, Daejeon, Korea

    Jongmoon Baik

Authors
  1. Jiwon Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Saranya Manikandan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Duksan Ryu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jongmoon Baik
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Duksan Ryu .

Editor information

Editors and Affiliations

  1. Magna Graecia University, Catanzaro, Italy

    Giuseppe Agapito

  2. Politecnico di Milano, Milano, Italy

    Anna Bernasconi

  3. Politecnico di Milano, Milano, Italy

    Cinzia Cappiello

  4. National University of Sciences and Technology, Islamabad, Pakistan

    Hasan Ali Khattak

  5. Korea Institute of Science and Technology, Daejeon, Korea (Republic of)

    InYoung Ko

  6. LUM University, Casamassima, Italy

    Giuseppe Loseto

  7. University of Primorska, koper, Slovenia

    Michael Mrissa

  8. University of Lausanne, Lausanne, Switzerland

    Luca Nanni

  9. Politecnico di Milano, Milano, Italy

    Pietro Pinoli

  10. University of Bari Aldo Moro, Bari, Italy

    Azzurra Ragone

  11. Polytechnic University of Bari, Bari, Italy

    Michele Ruta

  12. Polytechnic University of Bari, Bari, Italy

    Floriano Scioscia

  13. Indian Institute of Technology Indore, Indore, India

    Abhishek Srivastava

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Choi, J., Manikandan, S., Ryu, D., Baik, J. (2023). An Empirical Analysis on Just-In-Time Defect Prediction Models for Self-driving Software Systems. In: Agapito, G., et al. Current Trends in Web Engineering. ICWE 2022. Communications in Computer and Information Science, vol 1668. Springer, Cham. https://doi.org/10.1007/978-3-031-25380-5_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-25380-5_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-25379-9

  • Online ISBN: 978-3-031-25380-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics