Empirical evidence on technical challenges when adopting continuous practices
Pages 11 - 20
Abstract
Context: Continuous Software Engineering offers several benefits like fast iterations, increased reliability, and shorter lead times to react to business needs through practices like Continuous Integration, Delivery, and Deployment, but these practices come with many adoption challenges, which may appear as social (hard to identify and overcome) or technical challenges (TCs). Since anticipating even a small amount of possible challenges can be overwhelming, there is value in detecting them as early as possible to make it cheaper, faster, and more feasible to adopt this set of practices. Objective: We summarize evidence about identifying TCs to adopt continuous deployment and related practices. Method: We performed a Rapid Review that analyzed 14 studies and performed a narrative summary to provide an evidence briefing to practitioners. Results: We identified 121 TCs and organized them into nine categories: architecture, building, code, deployment, development practices, operations, pipeline, security, and testing. TCs related to code, pipeline, and testing are the most prevalent in the reviewed literature, while operations and deployment are the least. Static analysis strategies are the most used for detecting TCs, accounting for 94 detection strategies out of 100. However, most of these static analyses resort to some parsing technique. Specifically, Abstract Syntax Tree is the most used, detecting 44 TCs. Conclusion: Most studies have mixed evidence about identifying TCs to adopt continuous practices - some focus on evaluating the efficiency of detection strategies. In contrast, others are more focused on their characterization. We also identified varying levels of efficiency when detecting TCs. Finally, we advise attention to interpreting the results based on the overall quality score.
References
[1]
2020. Guidelines for creating a Narrative Summary. Technical Report. Vanderbilt-Meharry Community Engaged Research Core. https://victr.vumc.org/wp-content/uploads/2020/01/Narrative-Summary-Guidelines_1.23.2020.pdf
[2]
2023. Replication Package. Zenodo. https://doi.org/10.5281/zenodo.7942306
[3]
Amine Barrak, Ellis E Eghan, Bram Adams, and Foutse Khomh. 2021. Why do builds fail?—A conceptual replication study. J. Syst. Softw. 177, 110939 (July 2021), 110939. https://doi.org/10.1016/j.jss.2021.110939
[4]
Bruno Cartaxo, Gustavo Pinto, and Sergio Soares. 2020. Rapid Reviews in Software Engineering. In Contemporary Empirical Methods in Software Engineering. Springer International Publishing, Cham, 357–384. https://doi.org/10.1007/978-3-030-32489-6_13
[5]
Lianping Chen. 2015. Continuous delivery: Huge benefits, but challenges too. IEEE software 32, 2 (2015), 50–54.
[6]
Brian Fitzgerald and Klaas-Jan Stol. 2017. Continuous software engineering: A roadmap and agenda. Journal of Systems and Software 123 (2017), 176–189.
[7]
Keheliya Gallaba and Shane McIntosh. 2020. Use and misuse of continuous integration features: An empirical study of projects that (mis)use Travis CI. IEEE Trans. Softw. Eng. 46, 1 (Jan. 2020), 33–50. https://doi.org/10.1109/TSE.2018.2838131
[8]
Jordan Henkel, Christian Bird, Shuvendu K Lahiri, and Thomas Reps. 2020. Learning from, understanding, and supporting DevOps artifacts for docker. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering (Seoul South Korea). ACM, New York, NY, USA, 38–49. https://doi.org/10.1145/3377811.3380406
[9]
Adha Hrusto, Per Runeson, and Emelie Engström. 2021. Closing the feedback loop in DevOps through autonomous monitors in operations. SN Comput. Sci. 2, 6 (Nov. 2021), 447–. https://doi.org/10.1007/s42979-021-00826-y
[10]
Jez Humble and David Farley. 2011. Continuous delivery reliable software releases through build, test, and Deployment Automation. Addison-Wesley. https://www.oreilly.com/library/view/continuous-delivery-reliable/9780321670250/
[11]
Eero Laukkanen, Juha Itkonen, and Casper Lassenius. 2017. Problems, causes and solutions when adopting continuous delivery—A systematic literature review. Information and Software Technology 82 (2017), 55–79. https://doi.org/10.1016/j.infsof.2016.10.001
[12]
Evangelos Ntentos, Uwe Zdun, Konstantinos Plakidas, and Sebastian Geiger. 2021. Evaluating and improving microservice architecture conformance to architectural design decisions. In Service-Oriented Computing. Springer International Publishing, Cham, 188–203. https://doi.org/10.1007/978-3-030-91431-8_12
[13]
Akond Rahman, Effat Farhana, and Laurie Williams. 2020. The ‘as code’ activities: development anti-patterns for infrastructure as code. Empir. Softw. Eng. 25, 5 (Sept. 2020), 3430–3467. https://doi.org/10.1007/s10664-020-09841-8
[14]
Akond Rahman, Chris Parnin, and Laurie Williams. 2019. The seven sins: Security smells in infrastructure as code scripts. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) (Montreal, QC, Canada). IEEE. https://doi.org/10.1109/ICSE.2019.00033
[15]
Akond Rahman, Md Rayhanur Rahman, Chris Parnin, and Laurie Williams. 2021. Security smells in Ansible and Chef scripts: A Replication Study. ACM Trans. Softw. Eng. Methodol. 30, 1 (Jan. 2021), 1–31. https://doi.org/10.1145/3408897
[16]
Zhongshan Ren, Wei Wang, Guoquan Wu, Chushu Gao, Wei Chen, Jun Wei, and Tao Huang. 2018. Migrating web applications from monolithic structure to microservices architecture. In Proceedings of the Tenth Asia-Pacific Symposium on Internetware (Beijing China). ACM, New York, NY, USA. https://doi.org/10.1145/3275219.3275230
[17]
Victor Vidigal Ribeiro, Daniela Soares Cruzes, and Guilherme Horta Travassos. 2021. Replication package - Moderator Factors of Software Security and Performance Verification. https://doi.org/10.5281/zenodo.5093800
[18]
Pilar Rodríguez, Alireza Haghighatkhah, Lucy Ellen Lwakatare, Susanna Teppola, Tanja Suomalainen, Juho Eskeli, Teemu Karvonen, Pasi Kuvaja, June M. Verner, and Markku Oivo. 2017. Continuous deployment of software intensive products and services: A systematic mapping study. Journal of Systems and Software 123 (1 2017), 263–291. https://doi.org/10.1016/j.jss.2015.12.015
[19]
Islem Saidani and Ali Ouni. 2021. Toward a smell-aware prediction model for CI build failures. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW) (Melbourne, Australia). IEEE. https://doi.org/10.1109/ASEW52652.2021.00017
[20]
Julian Schwarz, Andreas Steffens, and Horst Lichter. 2018. Code Smells in Infrastructure as Code. In 2018 11th International Conference on the Quality of Information and Communications Technology (QUATIC) (Coimbra). IEEE. https://doi.org/10.1109/QUATIC.2018.00040
[21]
Mojtaba Shahin, Muhammad Ali Babar, and Liming Zhu. 2017. Continuous integration, delivery and deployment: A systematic review on approaches, tools, challenges and practices. IEEE Access 5 (2017), 3909–3943. https://doi.org/10.1109/ACCESS.2017.2685629
[22]
Carmine Vassallo, Sebastian Proksch, Harald C Gall, and Massimiliano Di Penta. 2019. Automated reporting of anti-patterns and decay in continuous integration. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) (Montreal, QC, Canada). IEEE. https://doi.org/10.1109/ICSE.2019.00028
[23]
Carmine Vassallo, Sebastian Proksch, Anna Jancso, Harald C Gall, and Massimiliano Di Penta. 2020. Configuration smells in continuous delivery pipelines: a linter and a six-month study on GitLab. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Virtual Event USA). ACM, New York, NY, USA. https://doi.org/10.1145/3368089.3409709
[24]
Fiorella Zampetti, Carmine Vassallo, Sebastiano Panichella, Gerardo Canfora, Harald Gall, and Massimiliano Di Penta. 2020. An empirical characterization of bad practices in continuous integration. Empir. Softw. Eng. 25, 2 (March 2020), 1095–1135. https://doi.org/10.1007/s10664-019-09785-8
Index Terms
- Empirical evidence on technical challenges when adopting continuous practices
Recommendations
Synthesizing Continuous Deployment Practices Used in Software Development
AGILE '15: Proceedings of the 2015 Agile ConferenceContinuous deployment speeds up the process of existing agile methods, such as Scrum, and Extreme Programming (XP) through the automatic deployment of software changes to end-users upon passing of automated tests. Continuous deployment has become an ...
Continuous deployment in software-intensive system-of-systems
Abstract Context:While continuous deployment is popular among web-based software development organizations, adopting continuous deployment in software-intensive system-of-systems is more challenging. On top of the challenges arising ...
Highlights- New software field testing and validation activities become continuous.
- ...
Managing Quality Assurance Challenges of DevOps through Analytics
ICSCA '19: Proceedings of the 2019 8th International Conference on Software and Computer ApplicationsDevOps is an intermarriage between developmental practices and operational modalities. The methodology employs the practices of continuous integration and delivery and places the deployment pipeline as the main requirement to automate, deliver and ...
Comments
Information & Contributors
Information
Published In
September 2023
570 pages
ISBN:9798400707872
DOI:10.1145/3613372
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 25 September 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
SBES 2023
SBES 2023: XXXVII Brazilian Symposium on Software Engineering
September 25 - 29, 2023
Campo Grande, Brazil
Acceptance Rates
Overall Acceptance Rate 147 of 427 submissions, 34%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 80Total Downloads
- Downloads (Last 12 months)36
- Downloads (Last 6 weeks)5
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format