Progressive Outcomes: A framework for maturing in agile software development

https://doi.org/10.1016/j.jss.2014.12.032Get rights and content

Highlights

  • Agile software development maturing process has people play the central role.

  • Ambidexterity is a key ability to agile maturity.

  • Improvement is guided by outcomes agile teams pursue.

  • Pursued outcomes in practices, team, deliveries, requirements, product and customer.

Abstract

Maturity models are used to guide improvements in the software engineering field and a number of maturity models for agile methods have been proposed in the last years. These models differ in their underlying structure prescribing different possible paths to maturity in agile software development, neglecting the fact that agile teams struggle to follow prescribed processes and practices. Our objective, therefore, was to empirically investigate how agile teams evolve to maturity, as a means to conceive a theory for agile software development evolvement that considers agile teams nature. The complex adaptive systems theory was used as a lens for analysis and four case studies were conducted to collect qualitative and quantitative data. As a result, we propose the Progressive Outcomes framework to describe the agile software development maturing process. It is a framework in which people have the central role, ambidexterity is a key ability to maturity, and improvement is guided by outcomes agile teams pursue, instead of prescribed practices.

Introduction

Maturity models are tools to describe how an element evolves. This element, which may be a person, an object, or a social system, may use the description provided by maturity models to assess its own situation and find guidance to improve in a specific focus area (Kohlegger et al., 2009). In the software engineering field, process improvement is based mainly on the guidelines given by the Capability Maturity Model Integration for Development –CMMI-DEV (CMMI Product Team, 2010) and the international standard ISO/IEC 15504 (ISO/IEC, 2004). Both CMMI-DEV and ISO/IEC 15504 share the underlying assumption that organizational capabilities must be codified in processes that are previously planned and designed (Maier et al., 2012). The improvement for the organization comes from the definition, institutionalization and quantitative management of these processes (CMMI Product Team, 2010).

A number of agile software development teams have been implementing these process improvement initiatives (Al-Tarawneh et al., 2011, Anderson, 2005, Baker, 2006, Caffery et al., 2008, Cohan and Glazer, 2009, Jakobsen and Johnson, 2008, Lina and Dan, 2012, Lukasiewicz and Miler, 2012, Spoelstra et al., 2011, Sutherland et al., 2007, Tuan and Thang, 2013). The benefits of such initiatives have been recognized as a “magic potion”, as they provide a powerful combination of adaptability and predictability (Sutherland et al., 2007). However, if teams are meant to keep agile in the highest maturity levels, the improvement path cannot be based on current established maturity models. The increasing processes definition hinders sustaining agility (Lukasiewicz and Miler, 2012, Paulk, 2001).

If agile methods place people and interaction over processes and tools (Beck, 2001, Conboy et al., 2011), the improvement road map for these methods should not be based on processes definition (Fontana et al., 2014). There are, for this reason, a number of agile maturity models proposed in the literature (Leppänen, 2013, Ozcan-Top and Demirörs, 2013, Schweigert et al., 2012). They are built over agile values and the improvement paths they suggest consider sustaining agility in the highest maturity levels. Two issues linger, though: the first is that they prescribe the practices the team should implement, even agile teams, which consider their work as a highly context-specific job to be prescribed (Fontana et al., 2014, Kettunen, 2012, Schweigert et al., 2012, Sidky et al., 2007); and the second is that the models still differ in their proposals, which indicates that the path to maturing in agile software development has not been uncovered yet.

These two issues thus, motivated this study. Our objective was to identify how agile software development teams evolve to maturity. We conducted an empirical research, through a multiple-case study approach, that identified how real agile teams evolve their practices and mature over time. The findings interest researchers, as they innovate in the underlying theory for a maturity model; and practitioners, as they provide practical guidelines for improving agile methods.

This paper is organized as follows: Section 2 outlines related work; Section 3 discusses our theoretical foundation and the theoretical framework we used for data analysis; and Section 4 presents the research structure. The results of the multiple-case study are presented in Section 5 and, finally, the findings are discussed and concluded in Sections 6 and 7, respectively.

Section snippets

Related work

Improvement paths in software engineering are currently defined by the guidelines given by CMMI-DEV and the international standard ISO/IEC 15504. The standard defines that software process improvement is accomplished through implementation of processes that address software acquisition, supply, engineering, operation, management, improvement, resources and infrastructure, reuse, configuration control, quality assurance and product quality (ISO/IEC, 2004).

Besides the incremental implementation

Theoretical foundation

Agile software development teams are complex adaptive systems (Power, 2014, Vidgen and Wang, 2009). They consist of a number of connected agents that “interact with each other according to sets of rules that require them to examine and to respond to each other's behavior in order to improve their behavior and thus the behavior of the system they comprise” (Stacey, 1996, p. 10). These interactions are not visible and not immediately understood (Perrow, 1981) and complex behavior emerges from

Research approach

The objective of this study was to identify the mechanisms teams apply to mature in agile software development. Our research question was thus “how do teams mature in agile software development?” To answer this question, we chose a multiple-case study approach. Case study is a research strategy to understand the dynamics present within single settings (Eisenhardt, 1989) and appropriate to answer the “how” questions in research (Yin, 2005).

The unit of analysis was the agile software development

Data analysis

This section first presents the analysis of data collected in each team and, later, the verification of the propositions considering the cross-case results. The within-case data analysis section comprises the description of each team's: business context, analysis of the evolvement of agile practices, ambidexterity data and project success perception. The cross-case data analysis section shows how the cross-case results support – or not – our propositions.

Discussion

This study aimed to identify how agile software development teams evolve to maturity. We identified it is a discontinuous process of experimentation, based on Progressive Outcomes, which the team pursues through the implementation of practices that cannot be prescribed. The team implements practices to chase outcomes in work practice, in its own behavior, in the deliveries, in the requirements, in the final product and in the relationship with the customer. It is a framework that reflects an

Conclusion

This study presented a framework for maturing in agile software development. We built this framework based on the analysis of qualitative and quantitative data in four Brazilian agile teams. Our theoretical foundation, based on complex adaptive systems theory, led us to build a framework for agile software evolvement that considers people as agents who play the key role in the maturing process, sees ambidexterity as a key ability to maturity, and does not prescribe practices, but describes

Acknowledgments

We would like to thank the team leaders and the practitioners for telling us their stories and answering our questionnaires.

Rafaela Mantovani Fontana has a Bachelor's degree in Computer Science and a Master's Degree in Systems and Production Engineering. She is a doctoral student at the Pontifical Catholic University of Paraná and a professor at the Federal University of Paraná. Her research interests include agile software development methods, project management, software process improvement, software quality and complexity theory applied to management.

References (90)

  • QumerA. et al.

    A framework to support the evaluation, adoption and improvement of agile methods in practice

    J. Syst. Softw.

    (2008)
  • SheffieldJ. et al.

    Factor associated with the software development agility of successful projects

    Int. J. Proj. Manag.

    (2013)
  • AndersinJ.

    TPI – A Model for Test Process Improvement

  • AndersonD.J.

    Stretching agile to fit CMMI level 3 – the story of creating MSF for CMMI process improvement at microsoft corporation

  • ArmbrustO. et al.

    The right process for each context: objective evidence needed

  • AugustineS. et al.

    Agile project management: steering from the edges

    Commun. ACM

    (2005)
  • BakerS.W.

    Formalizing agility, Part 2: how an agile organization embraced the CMMI

  • Bardin, L., 2011. Análise de Conteúdo. Edições...
  • Beck, K. et al., 2001. Agile Manifesto. Available in http://agilemanifesto.org/. Accessed in 2014,...
  • BenefieldR.

    Seven dimensions of agile maturity in the global enterprise: a case study

  • BoehmB. et al.

    Balancing agility and discipline: evaluating and integrating agile and plan-driven methods

  • BuglioneL.

    Light maturity models (LMM): an agile application

  • BustardD. et al.

    The maturation of agile software development principles and practice: Observations on successive industrial studies in 2010 and 2012

  • CafferyF.M. et al.

    AHAA – agile, hybrid assessment method for automotive, safety critical SMEs

  • Campbell-HuntC.

    Complexity in practice

    Hum. Relat.

    (2007)
  • ClarkeP. et al.

    An approach to evaluating software process adaptation

  • CMMI Product Team, 2010. CMMI for Development, Version 1.3. Software Engineering Institute, Carnegie Mellon University,...
  • CockburnA.

    Agile Software Development: The Cooperative Game

    (2007)
  • CohanS. et al.

    An agile development team's quest for CMMI maturity level 5

  • ConboyK. et al.

    People over process: key challenges in agile development

    IEEE Softw.

    (2011)
  • EijnattenF.M.

    Chaordic systems thinking: chaos and complexity to explain human performance management

  • EisenhardtK.

    Building theories from case study research

    Acad. Manage. Rev.

    (1989)
  • EisenhardtK.M. et al.

    Dynamic capabilities: what are they?

    Strateg. Manage. J.

    (2000)
  • FontanaR.M. et al.

    Maturing in agile: what is it about?

  • GibsonC. et al.

    The antecedents, consequences, and mediating role of organizational ambidexterity

    Acad. Manage. J.

    (2004)
  • HidalgoC.

    The value in between: organizations as adapting and evolving networks

  • HodaR. et al.

    Self-organizing roles on agile software development teams

    IEEE Trans. Softw. Eng.

    (2012)
  • HumphreyW.

    A Discipline for Software Engineering

    (1995)
  • Humphrey, W., Chick, T., Nichols, W., Pomeroy-Huff, M. 2010. Team Software Process (TSP) Body of Knowledge (BOK),...
  • ISO/IEC: 15504-1

    Information Technology – Process Assessment – Part 1: Concepts and Vocabulary

    (2004)
  • ISO/IEC: 33003. 2014. Accessed in 2014, October. Information technology – Process assessment – Requirements for process...
  • Jakobsen, C.R., Johnson, K.A., 2008. Mature agile with a twist of CMMI. Agile Conference 2008, 4–8 August, pp. 212-217....
  • KettunenP.

    Realizing agile software enterprise transformations by team performance development

  • KettunenP.

    Systematizing software development agility: towards an enterprise capability improvement framework

    J. Enterp. Transform.

    (2012)
  • KohleggerM. et al.

    Understanding maturity models results of a structured content analysis

  • Cited by (67)

    • Comparative effects of knowledge-based antecedents in different realms of CMMI-based software process improvement success

      2022, Computer Standards and Interfaces
      Citation Excerpt :

      Exploitation and exploration are also recognized as an organization's two knowledge-based enablers of software development. For example, Vidgen and Wang [18] and Fontana et al. [15] discovered how an agile development team continually leverages members’ knowledge and capabilities (exploitation) and searches for new techniques and methods (exploration) to achieve higher performance in dynamic development. Temizkan and Kumar [17] reported that in open-source development, exploitation broadens the effect of organizations’ existing knowledge with “patch development (e.g., APIs)”, while exploration enables “future requests” that help add new features to software.

    • Issues and challenges impacting the successful management of agile-hybrid projects: A grounded theory approach

      2021, International Journal of Project Management
      Citation Excerpt :

      15 studies were identified, selected, and analysed. These studies were based on the areas of: (1) strategic PMgmt framework (Shenhar, 1999), (2) framework for the classification of information systems (IS) methodologies (Iivari et al., 2000), (3) PMO framework (Kaufman & Korrapati, 2007), (4) framework on understanding organisational PMgmt through a PMO (Aubry et al., 2007), (5) agile software process assessment framework (Zhu, 2010), (6) agile PMgmt framework (Molhanec, 2010), (7) framework for agile transition and adoption (Gandomani & Nafchi, 2015), (8) hybrid framework for agile and stage gate model (Conforto & Amaral, 2015), (9) agile and stage-gate hybrid model (Sommer et al., 2015), (10) agile software development maturity framework (Fontana et al., 2015a), (11) software outsourcing evaluation framework (Ali & Khan, 2016), (12) agile distributed software development framework (Vallon et al., 2016), (13) PMO evaluation framework (Szalay, 2017), (14) maturity model for scaled agile framework (Turetken, 2017), and (15) adaptation of the scrum framework (Hidalgo, 2019). The title of the identified studies, along with the research gap, in terms of the findings and contributions of this study, are described in Table 10.

    View all citing articles on Scopus

    Rafaela Mantovani Fontana has a Bachelor's degree in Computer Science and a Master's Degree in Systems and Production Engineering. She is a doctoral student at the Pontifical Catholic University of Paraná and a professor at the Federal University of Paraná. Her research interests include agile software development methods, project management, software process improvement, software quality and complexity theory applied to management.

    Victor Meyer Jr. is Professor of Strategic Management at the Post Graduate Program in Business Administration, Pontifícia Universidade Católica do Paraná, Brazil. He received his Master and Doctorate degrees from the University of Houston, USA and conducted postdoctoral studies at the University of Michigan, USA. He has been a visiting professor at the School of Public Services, DePaul University, Chicago and the author of books and academic journal articles published in Brazil and abroad. He is a member of the Editorial Board of the Universidade em Debate, a Brazilian journal devoted to the main issues in the field at both national and international contexts. His main research focuses on strategic management in complex organizations, with special interest in strategic practices in higher education institutions and hospitals.

    Sheila Reinehr has a Bachelor's Degree in Mechanical Engineering, a Master's Degree in Informatics and a Doctorate in Engineering. She is a professor at the Pontifical Catholic University of Paraná and an experienced researcher in the following areas of computer science: software engineering, software process improvement, software quality, project management, software product lines and metrics.

    Andreia Malucelli has a Bachelor's Degree in Informatics, a Master's Degree in Electrical Engineering and a Doctorate in Electrical and Computing Engineering. She is a professor at the Pontifical Catholic University of Paraná and an experienced researcher in the following areas of computer science: software engineering, artificial intelligence, organizational learning, ontologies, multiagent systems and healthcare information systems.

    View full text