Skip to main content

Advertisement

SpringerLink
Log in

Improving the prediction of continuous integration build failures using deep learning

  • Published:
Automated Software Engineering Aims and scope Submit manuscript

Abstract

Continuous Integration (CI) aims at supporting developers in integrating code changes constantly and quickly through an automated build process. However, the build process is typically time and resource-consuming as running failed builds can take hours until discovering the breakage; which may cause disruptions in the development process and delays in the product release dates. Hence, preemptively detecting when a software state is most likely to trigger a failure during the build is of crucial importance for developers. Accurate build failures prediction techniques can cut the expenses of CI build cost by early predicting its potential failures. However, developing accurate prediction models is a challenging task as it requires learning long- and short-term dependencies in the historical CI build data as well as extensive feature engineering to derive informative features to learn from. In this paper, we introduce DL-CIBuild a novel approach that uses Long Short-Term Memory (LSTM)-based Recurrent Neural Networks (RNN) to construct prediction models for CI build outcome prediction. The problem is comprised of a single series of CI build outcomes and a model is required to learn from the series of past observations to predict the next CI build outcome in the sequence. In addition, we tailor Genetic Algorithm (GA) to tune the hyper-parameters for our LSTM model. We evaluate our approach and investigate the performance of both cross-project and online prediction scenarios on a benchmark of 91,330 CI builds from 10 large and long-lived software projects that use the Travis CI build system. The statistical analysis of the obtained results shows that the LSTM-based model outperforms traditional Machine Learning (ML) models with both online and cross-project validations. DL-CIBuild has shown also a less sensitivity to the training set size and an effective robustness to the concept drift. Additionally, by considering several Hyper-Parameter Optimization (HPO) methods as baseline for GA, we demonstrate that the latter performs the best

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. https://github.com/rails/rails.

  2. https://github.com/ruby/ruby.

  3. https://github.com/rapid7/metasploit-framework.

  4. https://github.com/apache/jackrabbit-oak.

  5. https://github.com/opf/openproject.

  6. https://github.com/CloudifySource/cloudify.

  7. https://github.com/Graylog2/graylog2-server.

  8. https://github.com/hashicorp/vagrant.

  9. https://github.com/SonarSource/sonarqube.

  10. https://github.com/openSUSE/open-build-service.

References

  • Abdalkareem, R., Mujahid, S., Shihab, E.: A machine learning approach to improve the detection of ci skip commits. IEEE Trans. Softw. Eng., (2020)

  • Agrawal, A., Menzies, T.: Is" better data" better than" better data miners"? In: 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE). IEEE, 1050–1061 (2018)

  • Arcuri, A., Briand, L.: A practical guide for using statistical tests to assess randomized algorithms in software engineering. In: 2011 33rd International Conference on Software Engineering (ICSE). IEEE, 1–10 (2011)

  • Atchison, Abigail, Berardi, Christina, Best, Natalie, Stevens, Elizabeth, Linstead, Erik: A time series analysis of travistorrent builds: to everything there is a season. In IEEE/ACM 14th International Conference on Mining Software Repositories (MSR), 463–466. IEEE (2017)

  • Athiwaratkun, Ben, Stokes, Jack W.: Malware classification with lstm and gru language models and a character-level cnn. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2482–2486 (2017)

  • Beller, M., Gousios, G., Zaidman, A.: Oops, my tests broke the build: An explorative analysis of travis ci with github. In: IEEE/ACM International Conference on Mining Software Repositories 356–367, (2017)

  • Beller, M., Gousios, G., Zaidman, A.: Travistorrent: Synthesizing travis ci and github for full-stack research on continuous integration. In 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR), 447–450 (2017)

  • Bergstra, James, Bengio, Yoshua: Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13(Feb), 281–305 (2012)

    MathSciNet  MATH  Google Scholar 

  • Bergstra, J., Bardenet, R., Bengio, Y., Kégl, B.: Algorithms for hyper-parameter optimization. In: 25th annual conference on neural information processing systems (NIPS 2011), volume 24. Neural Information Processing Systems Foundation, (2011)

  • Bhowan, Urvesh, Johnston, Mark, Zhang, Mengjie, Yao, Xin: Reusing genetic programming for ensemble selection in classification of unbalanced data. IEEE Trans. Evolut. Comput. 18(6), 893–908 (2013)

    Article  Google Scholar 

  • Bhowan, Urvesh, Johnston, Mark, Zhang, Mengjie: Evolving ensembles in multi-objective genetic programming for classification with unbalanced data. In Annual conference on Genetic and evolutionary computation (GECCO), 1331–1338, (2011)

  • Bhowan, Urvesh, Zhang, Mengjie, Johnston, Mark: Genetic programming for classification with unbalanced data. In European Conference on Genetic Programming. Springer, 1–13 (2010)

  • Bhowan, U., Johnston, M., Zhang, M.: Differentiating between individual class performance in genetic programming fitness for classification with unbalanced data. In: 2009 IEEE Congress on Evolutionary Computation. IEEE, 2802–2809 (2009)

  • Bishop, C.M.: Pattern recognition and machine learning (information science and statistics). Springer-Verlag New York, inc., Secaucus, NJ, USA, (2006))

  • Bouktif, Salah, Fiaz, Ali, Ouni, Ali, Adel Serhani, Mohamed: Multi-sequence lstm-rnn deep learning and metaheuristics for electric load forecasting. Energies 13(2), 391 (2020)

    Article  Google Scholar 

  • Bouktif, Salah, Fiaz, Ali, Ouni, Ali, Serhani, Mohamed Adel: Optimal deep learning lstm model for electric load forecasting using feature selection and genetic algorithm: Comparison with machine learning approaches. Energies 11(7), 1636 (2018)

    Article  Google Scholar 

  • Breiman, Leo: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  MATH  Google Scholar 

  • Buda, Mateusz, Maki, Atsuto, Mazurowski, Maciej A.: A systematic study of the class imbalance problem in convolutional neural networks. Neural Netw. 106, 249–259 (2018)

    Article  Google Scholar 

  • Cervantes, J., Li, X., Yu, W.: Using genetic algorithm to improve classification accuracy on imbalanced data. In: 2013 IEEE International Conference on Systems, Man, and Cybernetics. IEEE, 2659–2664 (2013)

  • Chawla, Nitesh V., Bowyer, Kevin W., Hall, Lawrence O., Kegelmeyer, W Philip: Smote: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)

    Article  MATH  Google Scholar 

  • Choetkiertikul, Morakot, Dam, Hoa Khanh, Tran, Truyen, Pham, Trang, Ghose, Aditya, Menzies, Tim: A deep learning model for estimating story points. IEEE Trans. Softw. Eng. 45(7), 637–656 (2018)

    Article  Google Scholar 

  • Cliff, Norman: Dominance statistics: Ordinal analyses to answer ordinal questions. Psychol. Bull. 114(3), 494 (1993)

    Article  Google Scholar 

  • colah’sblog. Understanding lstm networks. http://colah.github.io/posts/2015-08-Understanding-LSTMs/,. Accessed: 2020-03-01

  • Collell, Guillem, Prelec, Drazen, Patil, Kaustubh R.: A simple plug-in bagging ensemble based on threshold-moving for classifying binary and multiclass imbalanced data. Neurocomputing 275, 330–340 (2018)

    Article  Google Scholar 

  • Cui, Yiming,Wang, Shijin, Li, Jianfeng: Lstm neural reordering feature for statistical machine translation. arXiv preprint arXiv:1512.00177, (2015)

  • Dey, Rahul., Salemt, F.M.: Gate-variants of gated recurrent unit (gru) neural networks. In: 2017 IEEE 60th international midwest symposium on circuits and systems (MWSCAS). IEEE 1597–1600 (2017)

  • Di Martino, S., Ferrucci, F., Gravino, C., Sarro, F.: A genetic algorithm to configure support vector machines for predicting fault-prone components. In: International Conference on Product Focused Software Process Improvement. Springer, Berlin, Heidelberg 247–261 (2011)

  • Duvall, P.M., Matyas, S., Glover, A.: Continuous integration: improving software quality and reducing risk. Pearson Education, (2007)

  • Ebo Bennin, Kwabena, Keung, Jacky, Phannachitta, Passakorn, Monden, Akito, Mensah, Solomon: Mahakil: Diversity based oversampling approach to alleviate the class imbalance issue in software defect prediction. IEEE Trans. Softw. Eng. 44(6), 534–550 (2017)

    Article  Google Scholar 

  • Ekanayake, J., Tappolet, J., Gall, H.C., Bernstein, A.: Tracking concept drift of software projects using defect prediction quality. In: 2009 6th IEEE International Working Conference on Mining Software Repositories. IEEE, 51–60 (2009)

  • Falkner, S., Klein, A., Hutter, F.: BOHB: Robust and efficient hyperparameter optimization at scale. In International Conference on Machine Learning. PMLR, 1437–1446 (2018)

  • Fowler, Martin: Continuous integration. https://www.martinfowler.com/articles/continuousIntegration.html, (2006). Accessed: 2020-01-01

  • Gallaba, K., Macho, C., Pinzger, M., McIntosh, S.: Noise and heterogeneity in historical build data: an empirical study of travis ci. In: Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, 87–97, (2018)

  • Ghaleb, T.A., Da Costa, D.A., Zou, Y.: An empirical study of the long duration of continuous integration builds. Empir. Softw. Eng. 24(4), 1–38 (2019)

    Article  Google Scholar 

  • Ghaleb, T.A., da Costa, D.A., Zou, Y., Hassan, A.E.: Studying the impact of noises in build breakage data. IEEE Trans. Softw. Eng., (2019b)

  • Ghotra, B., McIntosh, S., Hassan, A.E.: Revisiting the impact of classification techniques on the performance of defect prediction models. In: 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering 1, 789–800 (2015)

  • Goldberg, David E.: Genetic algorithms in search. Optimization, and Machine Learning, (1989)

  • Graves, Alex, Schmidhuber, Jürgen.: Framewise phoneme classification with bidirectional lstm and other neural network architectures. Neural Netw. 18(5–6), 602–610 (2005)

    Article  Google Scholar 

  • Graves, Alex, Jaitly, Navdeep, Mohamed, Abdel-rahman: Hybrid speech recognition with deep bidirectional lstm. In: 2013 IEEE workshop on automatic speech recognition and understanding. IEEE 273–278 (2013)

  • Guo, Baosu, Jingwen, Hu., Wenwen, Wu., Peng, Qingjin, Fenghe, Wu.: The tabu_genetic algorithm: a novel method for hyper-parameter optimization of learning algorithms. Electronics 8(5), 579 (2019)

    Article  Google Scholar 

  • Haris, Muhammad, Hasan, Muhammad Noman, Qin, Shiyin: Early and robust remaining useful life prediction of supercapacitors using bohb optimized deep belief network. Appl. Energy 286, 116541 (2021)

    Article  Google Scholar 

  • Harman, Mark, Mansouri, S Afshin, Zhang, Yuanyuan: Search-based software engineering: Trends, techniques and applications. ACM Comput. Surv. (CSUR) 45(1), 11 (2012)

    Article  Google Scholar 

  • Harman, Mark, McMinn, Phil, De Souza, Jerffeson Teixeira, Yoo, Shin: Search based software engineering: Techniques, taxonomy, tutorial. In: Empirical software engineering and verification. Springer, Heidelberg 1–59 (2010)

  • Hassan, F., Wang, X.: Change-aware build prediction model for stall avoidance in continuous integration. In: ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, 157–162 (2017)

  • Hastie, Trevor, Tibshirani, Robert, Friedman, Jerome: The elements of statistical learning: data mining, inference, and prediction. Springer Science & Business Media, Berlin, Heidelberg (2009)

    Book  MATH  Google Scholar 

  • Hilton, M., Nelson, N., Tunnell, T., Marinov, D., Dig, D.:Trade-offs in continuous integration: assurance, security, and flexibility. In: 11th Joint Meeting on Foundations of Software Engineering, 197–207. ACM. (2017)

  • Hilton, M., Tunnell, T., Huang, K., Marinov, D., Dig, D.: Usage, costs, and benefits of continuous integration in open-source projects. In: 31st IEEE/ACM International Conference on Automated Software Engineering, ASE 2016, 426–437, (2016)

  • Hochreiter, Sepp, Schmidhuber, Jürgen.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  • Hochreiter, Sepp, Schmidhuber, Jürgen: Lstm can solve hard long time lag problems. In: Advances in neural information processing systems, 473–479 (1997b)

  • Hpbandster. https://automl.github.io/HpBandSter/build/html/index.html,. Accessed: 2021-12-16

  • Hsu, Chih-Wei, Chang, Chih-Chung, Lin, Chih-Jen, et al.: A practical guide to support vector classification, (2003)

  • Hyperopt framework. http://hyperopt.github.io/hyperopt/,. Accessed: 2021-12-16

  • jruby project. https://github.com/jruby/jruby,. Accessed: 2021-12-16

  • Javier Ordóñez, Francisco, Roggen, Daniel: Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors 16(1), 115 (2016)

    Article  Google Scholar 

  • Jebnoun, Hadhemi, Ben Braiek, Houssem, Masudur Rahman, Mohammad, Khomh, Foutse: The scent of deep learning code: An empirical study. (2020)

  • Jelihovschi, Enio G., Faria, José Cláudio., Allaman, Ivan Bezerra: Scottknott: a package for performing the scott-knott clustering algorithm in R. TEMA (São Carlos) 15(1), 3–17 (2014)

    Article  MathSciNet  Google Scholar 

  • Karpathy, Andrej, Johnson, Justin, Fei-Fei, Li: Visualizing and understanding recurrent networks. arXiv preprint arXiv:1506.02078, (2015)

  • Krawczyk, Bartosz, Woźniak, Michał: Cost-sensitive neural network with roc-based moving threshold for imbalanced classification. In: International Conference on Intelligent Data Engineering and Automated Learning. Springer, Heidelberg 45–52 (2015)

  • Krishna, R., Menzies, T., Fu, W.: Too much automation? the bellwether effect and its implications for transfer learning. In: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering. ACM 122–131 (2016)

  • Li, Hang, Gong, Xiu-Jun., Hua, Yu., Zhou, Chang: Deep neural network based predictions of protein interactions using primary sequences. Molecules 23(8), 1923 (2018)

    Article  Google Scholar 

  • Li, Lisha, Jamieson, Kevin, DeSalvo, Giulia, Rostamizadeh, Afshin, Talwalkar, Ameet: Hyperband: A novel bandit-based approach to hyperparameter optimization. J. Mach. Learn. Res. 18(1), 6765–6816 (2017)

    MathSciNet  MATH  Google Scholar 

  • Li, Yanxi, Dong, Minjing, Wang, Yunhe, Xu, Chang: Neural architecture search in a proxy validation loss landscape. In: International Conference on Machine Learning. PMLR, 5853–5862 (2020)

  • Lorenzo, P.R., Nalepa, J., Kawulok, M., Ramos, L.S., Pastor, J.R.: Particle swarm optimization for hyper-parameter selection in deep neural networks. In: Proceedings of the genetic and evolutionary computation conference, 481–488, (2017)

  • Luo, Y., Zhao, Y., Ma, W., Chen, L.: What are the factors impacting build breakage?. In: 2017 14th Web Information Systems and Applications Conference (WISA). IEEE, 139–142 (2017)

  • Längkvist, Martin, Karlsson, Lars, Loutfi, Amy: A review of unsupervised feature learning and deep learning for time-series modeling. Patt. Recog. Lett. 42, 11–24 (2014)

    Article  Google Scholar 

  • Malhotra, Ruchika, Khanna, Megha: An exploratory study for software change prediction in object-oriented systems using hybridized techniques. Auto. Softw. Eng. 24(3), 673–717 (2017)

    Article  Google Scholar 

  • Mkaouer, Wiem, Kessentini, Marouane, Shaout, Adnan, Koligheu, Patrice, Bechikh, Slim, Deb, Kalyanmoy, Ouni, Ali: Many-objective software remodularization using nsga-iii. ACM Trans. Softw. Eng. Methodol. (TOSEM) 24(3), 17 (2015)

    Article  Google Scholar 

  • Ni, A., Li, M.: Cost-effective build outcome prediction using cascaded classifiers. In: International Conference on Mining Software Repositories (MSR), 455–458 (2017)

  • Ni, A., Li, M.: Poster: Acona: Active online model adaptation for predicting continuous integration build failures. In: 2018 IEEE/ACM 40th International Conference on Software Engineering: Companion (ICSE-Companion). IEEE, 366–367 (2018)

  • Optunity framework. https://optunity.readthedocs.io/en/latest/,. Accessed: 2021-12-16

  • Ouni, Ali, Kessentini, Marouane, Sahraoui, Houari, Inoue, Katsuro, Deb, Kalyanmoy: Multi-criteria code refactoring using search-based software engineering: An industrial case study. ACM Trans. Softw. Eng. Methodol. (TOSEM) 25(3), 23 (2016)

    Article  Google Scholar 

  • Pascanu, Razvan, Mikolov, Tomas, Bengio, Yoshua: On the difficulty of training recurrent neural networks. In: International conference on machine learning, 1310–1318, (2013)

  • Provost, F.: Machine learning from imbalanced data sets 101. In: Proceedings of the AAAI’2000 workshop on imbalanced data sets, Vol. 68, No. 2000, pp. 1-3. AAAI Press (2000)

  • Rausch, T., Hummer, W., Leitner, P., Schulte, S.: An empirical analysis of build failures in the continuous integration workflows of java-based open-source software. In: Proceedings of the 14th international conference on mining software repositories. IEEE Press, 345–355 (2017)

  • Replication package for ci build prediction. Available at : https://github.com/stilab-ets/DL-CIBuild, (2020)

  • Romano, J., Kromrey, J.D., Coraggio, J., Skowronek, J.: Appropriate statistics for ordinal level data: Should we really be using t-test and cohen’sd for evaluating group differences on the nsse and other surveys. In: annual meeting of the Florida Association of Institutional Research, 1–33, (2006)

  • Ross Quinlan, J.: C4. 5: A programs for machine learning. Elsevier, Amsterdam (2014)

    Google Scholar 

  • Saidani, Islem, Ouni, Ali, Mkaouer, Mohamed Wiem, Palomba, Fabio: On the impact of continuous integration on refactoring practice: An exploratory study on travistorrent. Inf. Softw. Technol. 138, 106618 (2021)

    Article  Google Scholar 

  • Saidani, I., Ouni, A., Mkaouer, W.: Detecting skipped commits in continuous integration using multi-objective evolutionary search. IEEE Trans. Softw. Eng. (2022)

  • Saidani, I., Ouni, A., Chouchen, M., Mkaouer, M.W.: Predicting continuous integration build failures using evolutionary search. J. Inf. Soft. Technol. 128, (2020)

  • Saidani, I., Ouni, A., Chouchen, M., Mkaouer, M.W.: Bf-detector: an automated tool for ci build failure detection. In: 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 1530–1534 (2021b)

  • Santolucito, M., Zhang, J., Zhai, E., Piskac, R.: Statically verifying continuous integration configurations. Technical Report, (2018)

  • Schapire, Robert E.: Explaining adaboost. In: Empirical inference. Springer, Heidelberg 37–52 (2013)

  • Shan, Y., Hoens, T.R., Jiao, J., Wang, H., Yu, D., Mao, J.C.: Deep crossing: Web-scale modeling without manually crafted combinatorial features. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 255–262 (2016)

  • Shi, Y., Eberhart, R.C.: Parameter selection in particle swarm optimization. In: International conference on evolutionary programming. Springer, Berlin 591–600 (1998)

  • Shrikanth, NC., Majumder, S., Menzies, T.: Early life cycle software defect prediction. why? how? arXiv preprint arXiv:2011.13071, (2020)

  • Singh, A., Walenstein, A., Lakhotia, A.: Tracking concept drift in malware families. In: Proceedings of the 5th ACM workshop on Security and artificial intelligence. 81–92, (2012)

  • Snoek, J., Larochelle, H., Adams, R.P.: Practical bayesian optimization of machine learning algorithms. arXiv preprint arXiv:1206.2944, (2012)

  • Sundermeyer, Martin, Schlüter, Ralf, Ney, Hermann: Lstm neural networks for language modeling. In: Thirteenth annual conference of the international speech communication association, (2012)

  • Sundsøy, Pål , Bjelland, Johannes, Reme, Bjørn-Atle, Iqbal, Asif M., Jahani, Eaman: Deep learning applied to mobile phone data for individual income classification. In Proceedings of the 2016 International Conference on Artificial Intelligence: Technologies and Applications, Bangkok, Thailand, 24–25 (2016)

  • Tantithamthavorn, C., McIntosh, S., Hassan, A.E., Matsumoto, K.: An empirical comparison of model validation techniques for defect prediction models. IEEE Trans. Softw. Eng. 43(1), 1–18 (2017)

    Article  Google Scholar 

  • Tantithamthavorn, Chakkrit, McIntosh, Shane, Hassan, Ahmed E., Matsumoto, Kenichi: The impact of automated parameter optimization on defect prediction models. IEEE Trans. Softw. Eng. 45(7), 683–711 (2018)

    Article  Google Scholar 

  • Tantithamthavorn, C., McIntosh, S., Hassan, A.E., Matsumoto, K.: The impact of automated parameter optimization for defect prediction models. IEEE Trans. Softw. Eng. 45(7), 683–711 (2018)

    Article  Google Scholar 

  • Tharwat, Alaa, Hassanien, Aboul Ella: Quantum-behaved particle swarm optimization for parameter optimization of support vector machine. J. Classif. 36(3), 576–598 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  • Travis ci. https://travis-ci.org/. Accessed: 2021-12-16

  • Tsymbal, Alexey: The problem of concept drift: definitions and related work. Comput. Sci. Depart., Trinity Coll. Dublin 106(2), 58 (2004)

    Google Scholar 

  • Vasilescu, B., Yu, Y., Wang, H., Devanbu, P., Filkov, V.: Quality and productivity outcomes relating to continuous integration in github. In: 10th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2015, 805–816, (2015)

  • Wang, Yequan, Huang, Minlie, Zhu, Xiaoyan, Zhao, Li: Attention-based lstm for aspect-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, 606–615, (2016)

  • Wicaksono, Ananto Setyo, Supianto, Ahmad Afif: Hyper parameter optimization using genetic algorithm on machine learning methods for online news popularity prediction. Int. J. Adv. Comput. Sci. Appl 9(12), 263–267 (2018)

    Google Scholar 

  • Widmer, Gerhard, Kubat, Miroslav: Learning in the presence of concept drift and hidden contexts. Mach. Learn. 23(1), 69–101 (1996)

    Google Scholar 

  • Wilcoxon, Frank, Katti, S.K., Wilcox, Roberta A.: Critical values and probability levels for the wilcoxon rank sum test and the wilcoxon signed rank T. Select. Tab. Math. Stat. 1, 171–259 (1970)

    MATH  Google Scholar 

  • Xia, Yufei, Liu, Chuanzhe, Li, YuYing, Liu, Nana: A boosted decision tree approach using bayesian hyper-parameter optimization for credit scoring. Expert Syst. Appl. 78, 225–241 (2017)

    Article  Google Scholar 

  • Xia, Jing, Li, Yanhui: Could we predict the result of a continuous integration build? an empirical study. In: IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C), 311–315, (2017)

  • Xia, Jing, Li, Yanhui, Wang, Chuanqi: An empirical study on the cross-project predictability of continuous integration outcomes. In: 14th Web Information Systems and Applications Conference (WISA), 234–239. IEEE, (2017a)

  • Xie, Zheng, Li, Ming: Cutting the software building efforts in continuous integration by semi-supervised online auc optimization. In IJCAI, 2875–2881 (2018)

  • Yang, Li., Shami, Abdallah: On hyperparameter optimization of machine learning algorithms: Theory and practice. Neurocomputing 415, 295–316 (2020)

    Article  Google Scholar 

  • Zenisek, Jan, Holzinger, Florian, Affenzeller, Michael: Machine learning based concept drift detection for predictive maintenance. Comput. Indus. Eng. 137, 106031 (2019)

    Article  Google Scholar 

  • Zhang, F., Zheng, Q., Zou, Y., Hassan, A.E.: Cross-project defect prediction using a connectivity-based unsupervised classifier. In: 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE). IEEE, 309–320 (2016)

  • Zhao, Y., Serebrenik, A., Zhou, Y., Filkov, V., Vasilescu, B.: The impact of continuous integration on other software development practices: A large-scale empirical study. In: 32nd IEEE/ACM International Conference on Automated Software Engineering, 60–71 (2017)

  • Zheng, Jun: Cost-sensitive boosting neural networks for software defect prediction. Expert Syst. Appl. 37(6), 4537–4543 (2010)

    Article  Google Scholar 

  • Zhou, Zhi-Hua., Liu, Xu-Ying.: Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Trans. Knowl. Data Eng. 18(1), 63–77 (2005)

    Article  Google Scholar 

Download references

Acknowledgements

This work has been supported by the Natural Sciences and Engineering Research Council of Canada (NSERC), Discovery Grant RGPIN-2018-05960.

Author information

Authors and Affiliations

  1. ETS Montreal, University of Quebec, Montreal, Canada

    Islem Saidani & Ali Ouni

  2. Rochester Institute of Technology, Rochester, USA

    Mohamed Wiem Mkaouer

Authors
  1. Islem Saidani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ali Ouni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohamed Wiem Mkaouer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ali Ouni.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

A optimal parameters for online validation (RQ1 and RQ2)

See Table 14.

Table 14 Optimal Parameters for RQ1 and RQ2
Full size table

B optimal parameters for cross-project validation (RQ3)

See Table 15.

Table 15 Optimal Parameters for cross-project validation (Jruby is the training project)
Full size table

C optimal parameters for RQ4

See Table 16.

Table 16 Optimal Parameters for RQ4
Full size table

Optimal parameters for RQ5

See Tables 17 and 18.

Table 17 Optimal Parameters for RQ5 (old data)
Full size table
Table 18 Optimal Parameters for RQ5 (recent data)
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Saidani, I., Ouni, A. & Mkaouer, M.W. Improving the prediction of continuous integration build failures using deep learning. Autom Softw Eng 29, 21 (2022). https://doi.org/10.1007/s10515-021-00319-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10515-021-00319-5

Keywords

Search

Navigation