research-article

BF-detector: an automated tool for CI build failure detection

Authors:

Moataz Chouchen,

Mohamed Wiem MkaouerAuthors Info & Claims

ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Pages 1530 - 1534

https://doi.org/10.1145/3468264.3473115

Published: 18 August 2021 Publication History

Abstract

Continuous Integration (CI) aims at supporting developers in inte-grating code changes quickly through automated building. How-ever, there is a consensus that CI build failure is a major barrierthat developers face, which prevents them from proceeding furtherwith development. In this paper, we introduceBF-Detector, anautomated tool to detect CI build failure. Based on the adaptationof Non-dominated Sorting Genetic Algorithm (NSGA-II), our toolaims at finding the best prediction rules based on two conflictingobjective functions to deal with both minority and majority classes.We evaluated the effectiveness of our tool on a benchmark of 56,019CI builds. The results reveal that our technique outperforms state-of-the-art approaches by providing a better balance between bothfailed and passed builds.BF-Detectortool is publicly available,with a demo video, at: https://github.com/stilab-ets/BF-Detector.

References

[1]

Rabe Abdalkareem, Suhaib Mujahid, and Emad Shihab. 2020. A Machine Learning Approach to Improve the Detection of CI Skip Commits. IEEE Transactions on Software Engineering ( 2020 ).

[2]

Rabe Abdalkareem, Suhaib Mujahid, Emad Shihab, and Juergen Rilling. 2019. Which Commits Can Be CI Skipped? IEEE Transactions on Software Engineering ( 2019 ).

[3]

Nuri Almarimi, Ali Ouni, Moataz Chouchen, Islem Saidani, and Mohamed Wiem Mkaouer. 2020. On the detection of community smells using genetic programming-based ensemble classifier chain. In 15th International Conference on Global Software Engineering (ICGSE). 43-54.

Digital Library

[4]

M. Beller, G. Gousios, and A. Zaidman. 2017. TravisTorrent: Synthesizing Travis CI and GitHub for Full-Stack Research on Continuous Integration. In IEEE/ACM 14th International Conference on Mining Software Repositories (MSR). 447-450.

[5]

Urvesh Bhowan, Mark Johnston, and Mengjie Zhang. 2011. Evolving ensembles in multi-objective genetic programming for classification with unbalanced data. In Annual conference on Genetic and evolutionary computation (GECCO). 1331-1338.

Digital Library

[6]

Urvesh Bhowan, Mark Johnston, Mengjie Zhang, and Xin Yao. 2012. Evolving diverse ensembles using genetic programming for classification with unbalanced data. IEEE Transactions on Evolutionary Computation 17, 3 ( 2012 ), 368-386.

[7]

Urvesh Bhowan, Mark Johnston, Mengjie Zhang, and Xin Yao. 2013. Reusing genetic programming for ensemble selection in classification of unbalanced data. IEEE Transactions on Evolutionary Computation 18, 6 ( 2013 ), 893-908.

[8]

Travis CI. [n.d.]. Travis CI. https://travis-ci.org/, note = Accessed: 2021-04-01.

[9]

Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and TAMT Meyarivan. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE transactions on evolutionary computation 6, 2, 182-197.

[10]

Paul M Duvall, Steve Matyas, and Andrew Glover. 2007. Continuous integration: improving software quality and reducing risk. Pearson Education.

[11]

Eclipse. [n.d.]. Eclipse Java Git library. https://www.eclipse.org/jgit//, note = Accessed: 2021-04-01.

[12]

Martin Fowler. 2006. Continuous Integration. https://www.martinfowler.com/ articles/continuousIntegration.html,. Accessed: 2020-01-01.

[13]

David Hadka. [n.d.]. MOEA Framework. http://moeaframework.org/,. Accessed: 2020-01-01.

[14]

David Hadka. 2014. MOEA framework user guide. ( 2014 ).

[15]

Foyzul Hassan and Xiaoyin Wang. 2017. Change-aware build prediction model for stall avoidance in continuous integration. In ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. 157-162.

Digital Library

[16]

Michael Hilton, Nicholas Nelson, Timothy Tunnell, Darko Marinov, and Danny Dig. 2017. Trade-ofs in continuous integration: assurance, security, and flexibility. In 11th Joint Meeting on Foundations of Software Engineering. 197-207.

[17]

Michael Hilton, Timothy Tunnell, Kai Huang, Darko Marinov, and Danny Dig. 2016. Usage, Costs, and Benefits of Continuous Integration in Open-source Projects. In 31st IEEE/ACM International Conference on Automated Software Engineering (Singapore, Singapore) (ASE 2016 ). 426-437.

Digital Library

[18]

Jsoup. [n.d.]. Jsoup. https://jsoup.org/, note = Accessed: 2021-04-01.

[19]

Marouane Kessentini and Ali Ouni. 2017. Detecting android smells using multiobjective genetic programming. In Proceedings of the 4th International Conference on Mobile Software Engineering and Systems. 122-132.

Digital Library

[20]

Yang Luo, Yangyang Zhao, Wanwangying Ma, and Lin Chen. 2017. What are the Factors Impacting Build Breakage?. In 14th Web Information Systems and Applications Conference (WISA). 139-142.

[21]

Ruchika Malhotra. 2015. A systematic review of machine learning techniques for software fault prediction. Applied Soft Computing 27 ( 2015 ), 504-518.

[22]

Ali Ouni. 2020. Search-based Software Engineering: Challenges, Opportunities and Recent Applications. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO). 1114-1146.

Digital Library

[23]

Ali Ouni, Raula Gaikovina Kula, Marouane Kessentini, and Katsuro Inoue. 2015. Web service antipatterns detection using genetic programming. In Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation. 1351-1358.

Digital Library

[24]

Ali Ouni, Marouane Kessentini, Katsuro Inoue, and Mel O Cinnéide. 2017. Searchbased web service antipatterns detection. IEEE Transactions on Services Computing 10, 4 ( 2017 ), 603-617.

[25]

Ali Ouni, Marouane Kessentini, Houari Sahraoui, and Mounir Boukadoum. 2013. Maintainability defects detection and correction: a multi-objective approach. Automated Software Engineering 20, 1 ( 2013 ), 47-79.

[26]

Ali Ouni, Marouane Kessentini, Houari Sahraoui, Katsuro Inoue, and Kalyanmoy Deb. 2016. Multi-criteria code refactoring using search-based software engineering: An industrial case study. ACM Transactions on Software Engineering and Methodology (TOSEM) 25, 3 ( 2016 ), 23.

Digital Library

[27]

Thomas Rausch, Waldemar Hummer, Philipp Leitner, and Stefan Schulte. 2017. An empirical analysis of build failures in the continuous integration workflows of Java-based open-source software. In 14th international conference on mining software repositories. 345-355.

Digital Library

[28]

Islem Saidani, Ali Ouni, Moataz Chouchen, and Mohamed Wiem Mkaouer. 2020. On the prediction of continuous integration build failures using search-based software engineering. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion. 313-314.

Digital Library

[29]

Islem Saidani, Ali Ouni, Moataz Chouchen, and Mohamed Wiem Mkaouer. 2020. Predicting continuous integration build failures using evolutionary search. Information and Software Technology 128 ( 2020 ), 106392.

[30]

Selenium. [n.d.]. Selenium. https://www.selenium.dev/, note = Accessed: 2021-04-01.

[31]

TravisTorrent. [n.d.]. TravisTorrent Dataset. https://travistorrent.testroots.org/, note = Accessed: 2021-04-01.

[32]

Bogdan Vasilescu, Yue Yu, Huaimin Wang, Premkumar Devanbu, and Vladimir Filkov. 2015. Quality and Productivity Outcomes Relating to Continuous Integration in GitHub. In 10th Joint Meeting on Foundations of Software Engineering (Bergamo, Italy) (ESEC/FSE 2015 ). 805-816.

Digital Library

[33]

Jing Xia and Yanhui Li. 2017. Could we predict the result of a continuous integration build? An empirical study. In IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C). 311-315.

[34]

Jing Xia, Yanhui Li, and Chuanqi Wang. 2017. An Empirical Study on the CrossProject Predictability of Continuous Integration Outcomes. In 14th Web Information Systems and Applications Conference (WISA). 234-239.

[35]

Huimin Zhao. 2007. A multi-objective genetic programming approach to developing Pareto optimal decision trees. Decision Support Systems 43, 3 ( 2007 ), 809-826.

Cited By

Freeda R ARajendran P S(2024)Test suite optimization under multi-objective constraints for software fault detection and localization: Hybrid optimization based modelWeb Intelligence10.3233/WEB-22013122:2(151-166)Online publication date: 22-Apr-2024
https://doi.org/10.3233/WEB-220131
Yin MKashiwa YGallaba KAlfadel MKamei YMcIntosh SFilkov VRay BZhou M(2024)Developer-Applied Accelerations in Continuous Integration: A Detection Approach and Catalog of PatternsProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695533(1655-1666)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695533
Sun GHabchi SMcIntosh S(2024)RavenBuild: Context, Relevance, and Dependency Aware Build Outcome PredictionProceedings of the ACM on Software Engineering10.1145/36437711:FSE(996-1018)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3643771
Show More Cited By

Index Terms

BF-detector: an automated tool for CI build failure detection
1. Software and its engineering
  1. Software notations and tools
    1. Software maintenance tools

Recommendations

On the prediction of continuous integration build failures using search-based software engineering
GECCO '20: Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion

Continuous Integration (CI) aims at supporting developers in integrating code changes quickly through automated building. However, in such context, the build process is typically time and resource-consuming. As a response, the use of machine learning (...
Predicting continuous integration build failures using evolutionary search
Abstract
Context: Continuous Integration (CI) is a common practice in modern software development and it is increasingly adopted in the open-source as well as the software industry markets. CI aims at supporting developers in ...
Improving the prediction of continuous integration build failures using deep learning
Abstract
Continuous Integration (CI) aims at supporting developers in integrating code changes constantly and quickly through an automated build process. However, the build process is typically time and resource-consuming as running failed builds can take ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

August 2021

1690 pages

ISBN:9781450385626

DOI:10.1145/3468264

General Chairs:
Diomidis Spinellis
Athens University of Economics and Business, Greece
,
Georgios Gousios
Facebook, Netherlands / Delft University of Technology, Netherlands
,
Program Chairs:
Marsha Chechik
University of Toronto, Canada
,
Massimiliano Di Penta
University of Sannio, Italy

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 August 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Natural Sciences and Engineering Research Council of Canada

Conference

ESEC/FSE '21

Sponsor:

SIGSOFT

ESEC/FSE '21: 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

August 23 - 28, 2021

Athens, Greece

Acceptance Rates

Overall Acceptance Rate 112 of 543 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
278
Total Downloads

Downloads (Last 12 months)66
Downloads (Last 6 weeks)2

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Freeda R ARajendran P S(2024)Test suite optimization under multi-objective constraints for software fault detection and localization: Hybrid optimization based modelWeb Intelligence10.3233/WEB-22013122:2(151-166)Online publication date: 22-Apr-2024
https://doi.org/10.3233/WEB-220131
Yin MKashiwa YGallaba KAlfadel MKamei YMcIntosh SFilkov VRay BZhou M(2024)Developer-Applied Accelerations in Continuous Integration: A Detection Approach and Catalog of PatternsProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695533(1655-1666)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695533
Sun GHabchi SMcIntosh S(2024)RavenBuild: Context, Relevance, and Dependency Aware Build Outcome PredictionProceedings of the ACM on Software Engineering10.1145/36437711:FSE(996-1018)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3643771
Saidani IOuni AMkaouer M(2022)Detecting Continuous Integration Skip Commits Using Multi-Objective Evolutionary SearchIEEE Transactions on Software Engineering10.1109/TSE.2021.312916548:12(4873-4891)Online publication date: 1-Dec-2022
https://doi.org/10.1109/TSE.2021.3129165
Saidani IOuni AMkaouer M(2022)Improving the prediction of continuous integration build failures using deep learningAutomated Software Engineering10.1007/s10515-021-00319-529:1Online publication date: 1-May-2022
https://dl.acm.org/doi/10.1007/s10515-021-00319-5
Saidani IOuni A(2021)Toward a Smell-aware Prediction Model for CI Build Failures2021 36th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW)10.1109/ASEW52652.2021.00017(18-25)Online publication date: Nov-2021
https://doi.org/10.1109/ASEW52652.2021.00017

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten