Cinders: The continuous integration and delivery architecture framework

https://doi.org/10.1016/j.infsof.2016.11.006Get rights and content

Abstract

Context: The popular agile practices of continuous integration and delivery have become an essential part of the software development process in many companies, yet effective methods and tools to support design, description and communication of continuous integration and delivery systems are lacking.

Objective: The work reported on in this paper addresses that lack by presenting Cinders — an architecture framework designed specifically to meet the needs of such systems, influenced both by prominent enterprise and software architecture frameworks as well as experiences from continuous integration and delivery modeling in industry.

Method: The state of the art for systematic design and description of continuous integration and delivery systems is established through review of literature, whereupon a proposal for an architecture framework addressing requirements derived from continuous integration and delivery modeling experiences is proposed. This framework is subsequently evaluated through interviews and workshops with engineers in varying roles in three independent companies.

Results: Cinders, an architecture framework designed specifically for the purpose of describing continuous integration and delivery systems is proposed and confirmed to constitute an improvement over previous methods. This work presents software professionals with a demonstrably effective method for describing their continuous integration and delivery systems from multiple points of view and supporting multiple use-cases, including system design, communication and documentation.

Conclusion: It is concluded that an architecture framework for the continuous integration and delivery domain has value; at the same time potential for further improvement is identified, particularly in the area of tool support for data collection as well as for manual modeling.

Introduction

This paper addresses the problem of constructing systems for rapidly and frequently transforming source code changes into verified and deliverable software product revisions with known content, known functionality and known quality — through software integration and test — in a controlled and methodical way.

Historically, the problem of transforming lines of source code into functioning, verified products running in their target environment could be regarded as a question of enterprise architecture: of organizational responsibilities and manual processes. With the advent and growth of continuous integration [2], [8] and delivery [9], [14], however, and the automation this brings, this is increasingly becoming a domain of software engineering: we see ever more sophisticated software systems being constructed, with the purpose of compiling, integrating, testing, delivering and deploying other software.

While such systems are generally perceived as adding value and increasing the efficiency of the development project, we have found in previous work that the exact nature of these benefits is highly uncertain and varies from case to case [35]. Furthermore, even though there are numerous popular tools that do much of the heavy lifting in these integration systems, they only address isolated parts of a very large problem domain. In all our industry case studies [35], [36], [38], [39] we have never found a complete off-the-shelf solution for continuous integration. Rather, the integration systems we find often use similar tools, but configured differently, put to different purposes and integrated with one another in varying constellations. Not surprisingly, a review of literature reveals that reported continuous integration systems display a high degree of variance [37]. In other words, as a rule, continuous integration and delivery systems are highly customized and purpose-built software products in their own right.

Similarly to the variance in system design, there is little consensus on the exact definition of continuous integration and delivery, particularly as opposed to related terms such as continuous testing, continuous release or continuous deployment. For the purposes of this paper, we use the term continuous integration and delivery system to mean any system of automated activities performed in order to transform source code into working and potentially shippable and deployable products with known quality, content and functionality, i.e. including compilation, linking, packaging, testing, profiling, documentation generation and much more, serving to ensure that “the software can be released to production at any time” [9]. In this paper we propose and evaluate a structured and systematic approach to the design and description of such systems, in order to help software professionals become more effective and efficient in a critical domain where significant resources are currently being invested.

The key contribution of this paper is that it provides an architectural framework for the design and description of continuous integration and delivery systems, thereby addressing the lack of systematic engineering approaches to solving a critical problem in the software development industry.

The rest of the paper is structured as follows. The problem statement is presented in the following section, discussing in greater depth the relevancy of systematic approaches to continuous integration delivery system design and description. Following this, Section 3 provides a background to the work reported from in this paper by discussing related tools and methods as well as previous work on continuous integration modeling. Section 4 presents the employed research method. Then the results are presented in Section 5 and evaluated in Section 6. Threats to validity are discussed in Section 7 and the paper is then concluded in Section 8.

Section snippets

Problem statement

It is easy to find introductions and tutorials to continuous integration — apart from a number of books, a web search for “continuous integration tutorial” returns well over a thousand hits [10]. While a small development project may require little work in order to set up a simplistic solution, designing a reliable continuous integration and delivery system for a medium to large scale project — particularly one satisfying the high levels of traceability required in large segments of industry [3]

Background

This section provides context by discussing the background of the work reported from in this paper.

Research method

This section describes the research method applied in the work we report from in this article.

Results

This section presents the results of the work conducted in the research project.

Evaluation

This section presents evaluation of the designed architecture framework by investigating each of the two suppositions phrased in Section 4.1.

Threats to validity

This section discusses any threats to the validity of the presented work.

Conclusion

This paper addresses the question of In what way can the paradigm of architecture frameworks favorably be applied to facilitate the design and description of continuous integration and delivery systems? We establish that this is an area of considerable investment in the industry, yet lacking in supporting tools and methods, coinciding with a tendency by studied industry cases to not address its challenges using as rigorous an approach as in regular product development. Based on thematic

Acknowledgments

We wish to extend our sincere gratitude to all of the participant companies and their employees for their time, patience and insights.

Daniel Ståhl is continuous integration subject matter expert and architect at Ericsson AB. He has a background of nine years of software development, integration and architecting in the telecom industry, where his work primarily revolves the application of continuous integration and delivery practices to multinational enterprise scale organizations. He received a MSc degree from Linköping University, Sweden, in 2007.

References (45)

  • S.T. March et al.

    Design and natural science research on information technology

    Decis. Support Syst.

    (1995)
  • D. Ståhl et al.

    Modeling continuous integration practice differences in industry software development

    Journal of Systems and Software

    (2014)
  • R. Wieringa et al.

    Six strategies for generalizing software engineering theories

    Sci. Comput. Program

    (2015)
  • O. Beaumont et al.

    Mixed data-parallel scheduling for distributed continuous integration

    26th International Parallel and Distributed Processing Symposium Workshops & PhD Forum (IPDPSW)

    (2012)
  • K. Beck

    Extreme Programming Explained: Embrace Change

    (2000)
  • BEL-v, bfs, CSN, ISTec, ONR, SSM, STUK, 2013,. Licensing of safety critical software for nuclear...
  • S. Brown, Software architecture for developers, 2013, Coding the...
  • T.D. Cook et al.

    Quasi-experimentation: Design & Analysis Issues for Field Settings

    (1979)
  • D.S. Cruzes et al.

    Recommended steps for thematic synthesis in software engineering

    International Symposium on Empirical Software Engineering and Measurement

    (2011)
  • European cooperation for space standardization, 2015,...
  • M. Fowler, Continuous integration, 2006, http://www.martinfowler.com/articles/continuousIntegration.html, [Online;...
  • M. Fowler, Continuous Delivery, 2013, http://martinfowler.com/bliki/ContinuousDelivery.html, [Online; accessed...
  • Google, Google search for continuous integration tutorials, 2016,...
  • A. Hevner et al.

    Design science in information systems research

    MIS quarterly

    (2004)
  • B. Hoffman et al.

    Software process for rapid development of hpc software using cmake

    DoD High Performance Computing Modernization Program Users Group Conference (HPCMP-UGC)

    (2009)
  • C. Hofmeister et al.

    Applied Software Architecture

    (2000)
  • J. Humble et al.

    Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation

    (2010)
  • ISO/IEC/IEEE 42010, 2015a, ISO/IEC/IEEE 42010. http://www.iso-architecture.org/42010, [Online; accessed...
  • ISO/IEC/IEEE 42010, 2015b, ISO/IEC/IEEE 42010 frequently asked questions....
  • ISO/IEC/IEEE 42010, 2015c, ISO/IEC/IEEE 42010 survey of architecture frameworks....
  • S.H. Kan

    Metrics and Models in Software Quality Engineering

    (2002)
  • E.H. Kim et al.

    Implementing an effective test automation framework

    33rd Annual IEEE International Computer Software and Applications Conference (COMPSAC’09)

    (2009)
  • Cited by (0)

    Daniel Ståhl is continuous integration subject matter expert and architect at Ericsson AB. He has a background of nine years of software development, integration and architecting in the telecom industry, where his work primarily revolves the application of continuous integration and delivery practices to multinational enterprise scale organizations. He received a MSc degree from Linköping University, Sweden, in 2007.

    Jan Bosch is professor of software engineering and director of the software research center at Chalmers University Technology in Gothenburg, Sweden. Earlier, he has worked as Vice President Engineering Process at Intuit Inc and as head of the Software and Application Technologies Laboratory at Nokia Research Center, Finland. Before joining Nokia, he headed the software engineering research group at the University of Groningen, The Netherlands, where he holds a professorship in software engineering. He received a MSc degree from the University of Twente, The Netherlands, and a PhD degree from Lund University, Sweden.

    View full text