Reconciling perspectives: A grounded theory of how people manage the process of software development

https://doi.org/10.1016/j.jss.2012.01.059Get rights and content

Abstract

Social factors are significant cost drivers for the process of software development. In this field study we generate a grounded theory of how people manage the process of software development. The main concern of engineers involved in the process of software development is getting the job done. To get the job done, people engage in a four-stage process of Reconciling Perspectives. Reconciling Perspectives represents an attempt to converge individuals’ points of view or perspectives about a software project. The process emphasizes the importance of individuals’ abilities to both reach out and engage in negotiations and create shelter from environmental noise to bring a software project to fruition.

Highlights

► A grounded theory of how people manage the process of software development. ► Perspectives mismatches between individuals and groups impede the process. ► Reconciling Perspectives is a four-stage process for removing impediments. ► Early stages depends on the ability of individuals to engage and interact with others. ► Later stages require moderation of interaction to focus on getting the job done.

Introduction

Software development is a risky and expensive proposition. Talking with software developers quickly illustrates the quagmire that software development can become. Late delivery, defective products, cost overruns, and frustrated staff and stake holders are some of the debris resulting from a failed or less than successful project. This is expensive debris considering that global software development is a 1.6 trillion US dollar industry (Bartels et al., 2006). Improving the productivity of software development teams and the quality of delivered software will result in significant economic returns.

Numerous studies have demonstrated that individual abilities and team social factors are significant cost drivers for software engineering projects, often swamping all other factors (Boehm, 1984, Cockburn, 2002, Cockburn and Highsmith, 2001, Curtis et al., 1987, Jones, 2000, Lister and DeMarco, 1987, Sawyer and Guinan, 1998). Caper Jones (2000) data demonstrate that high levels of management and staff experience contribute 120% to productivity while effective methods and processes contribute only 35%. Cost drivers associated with personal factors from the COCOMO model “reflect the strong influence of personal capability on software productivity” (Boehm et al., 1995, p. 86). Earlier, Boehm concluded:

“Personnel attributes and human relations activities provide by far the largest source of opportunity for improving software productivity” (Boehm, 1984)

If social factors are the biggest cost drivers, and explain variance in the productivity of software development teams, research studies that help us identify and understand social processes in software engineering should yield significant benefit to the industry. In the past, most software engineering research has focused on tools and production methods, which have limited ability to account for and manage the variance in software projects (Glass et al., 2002, Sawyer and Guinan, 1998, Shaw, 2003, Sjoeberg et al., 2005, Zannier et al., 2006). Shaw's (2003) analysis of papers accepted by the International Conference on Software Engineering (ICSE) shows the majority of accepted papers to be

“…reports [of] an improved method or means of developing software that is, of designing, implementing, evolving, maintaining, or otherwise operating on the software system itself. Papers addressing these questions dominate both the submitted and the accepted papers” (Shaw, 2003, p. 727).

In the past, procedures or techniques and tools and notation accounted for the majority of the accepted papers (69%), while less than 10% of the accepted papers described empirical or qualitative models. Another survey of the software engineering research literature (Glass et al., 2002) reported a very small percentage (in many cases less than 1%) of research papers explored organizational issues or employed research methods useful in understanding social behavior.

The software engineering research agenda is changing, with more researchers investigating the influence that personal attributes and human relationships have on software projects (Dittrich et al., 2007). The Agile Manifesto and the Agile software development movement placed a spotlight on the importance of social interactions with the first article of the Manifesto for Agile Software Development:

“Individuals and interactions over processes and tools”

The emphasis on individuals and interactions motivated research into social interaction (Chong, 2005, Hoda et al., 2010, Moe et al., 2008, Robinson and Sharp, 2005, Whitworth and Biddle, 2007).

However, we believe the state of software engineering research can still be summed up in Curtis et al.’s (1987) retelling of the 13th century story of a man who, after losing his keys, crosses the street to search under a lamppost because the light is much better there. This should not come as a surprise to us because we are engineers and not sociologists. Engineers—especially at the research level—are engineers because they enjoy solving technical problems. Developing tools and methods is what we are educated to do, and what we enjoy doing; however, if we are to better understand the factors that have the greatest influence on software development performance, our studies must also examine social processes.

This paper is an extended revision of our Agile 2011 conference paper (Adolph and Kruchten, 2011). For this research project, we had the opportunity to study a number of Agile software development teams in order to understand the variability in the productivity of software development by creating a substantive theory of how people actually do, or “manage” as we have referred to it, the process of software development. Our study makes two contributions: the first is our findings, and the second is our experience using grounded theory (Glaser and Strauss, 1967). In this paper, the conceptual elements of the theory are highlighted with an underlined italicized font; for example, the conceptual element Bunkering is highlighted as Bunkering. In Section 2 we present an outline of our study and a description of our use of the grounded theory method. In Section 3 we present our theory, and in Section 4 we provide a discussion of our results within the context of existing theories. Section 5 summarizes our results and recommendations.

Section snippets

Motivation

The study we conducted was aimed at understanding the social processes that influence software team performance and the effects of software methods on those processes. Methodologists argue the benefits of following a software development methodology, and there are studies that demonstrate a positive correlation between software development method adoption and team effectiveness in terms of product quality and productivity (Diaz and Sligo, 1997, Harter et al., 2000). On the other hand, industry

Results

Software development takes place in the context of a diverse organizational Eco-system where an Acquirer requests a Supplier to perform a Job for them. To perform a Job, the Supplier and Acquirer must agree on their expectations of the Job, including the Work Products that will be delivered as part of the Job and the schedule for their delivery. The Supplier creates the Work Products for the Job and delivers them to the Acquirer for acceptance. The diversity of the organizational Eco-system,

Discussion

This discussion of Reconciling Perspectives compares it to other relevant grounded theories and extant theory to both situate Reconciling Perspectives within the existing body of knowledge and to generate policy recommendations. The Anticipation-eXecution-Expectation (AxE) model (Hsieh et al., 2006) and Organizing Self Organizing Teams (Hoda et al., 2010) are two grounded theories that illuminate some properties of Reconciling Perspectives. The AxE model emerged out of a study of coordination

Summary

Our study explains how people manage the process of developing software using the process of Reconciling Perspectives to remove impediments created by Perspective Mismatches that prevent Getting the Job Done. The process breaks down when:

  • Individuals and teams go dark (McCarthy and Gilbert, 1995) and the process simply does not start because the number of Communications is so low or intermittent that individuals are unable to reflect on the Feedback they are hearing in the communications and do

Acknowledgements

The authors wish to express their thanks to the AgileAlliance, the Scrum Alliance, and the Eclipse Foundation for their support of this research. Our further thanks go to our reviewers, and especially the anonymous reviewer who introduced us to Boland's and Tenkasi's paper.

Steve Adolph is a PhD candidate in Electrical and Computer Engineering at the University of British Columbia and an agile coach with Rally Software. His research interests include software process engineering, requirements management, and software architecture.

References (60)

  • B.W. Boehm et al.

    Cost models for future software life cycle processes: COCOMO 2.0

    Annals of Software Engineering

    (1995)
  • R.J. Boland et al.

    Perspective making and perspective taking in communities of knowing

    Organization Science

    (1995)
  • T. Burns et al.

    The Management of Innovation

    (1961)
  • J.A. Cannon-Bowers et al.

    Shared mental models in expert team decision making

  • Chong, J., 2005. Social behaviors on XP and non-XP teams: a comparative study. Paper presented at the Proceedings of...
  • A. Cockburn

    Agile software development joins the would-be crowd

    Cutter IT Journal

    (2002)
  • A. Cockburn

    People and Methodologies in Software Development

    (2003)
  • A. Cockburn et al.

    Agile software development, the people factor

    Computer

    (2001)
  • M. Cronin et al.

    Representational gaps, information processing, and conflict in functionally diverse teams

    Academy of Management Review

    (2007)
  • Curtis, W., Krasner, H., Shen, V., Iscoe, N., 1987. On building software process models under the lamppost. Paper...
  • M. Diaz et al.

    How software process improvement helped Motorola

    IEEE Software

    (1997)
  • Dyba, T., Moe, N.B., Arisholm, E., 2005. Measuring software methodology usage: challenges of conceptualization and...
  • R. Emerson et al.

    Writing Ethnographic Fieldnotes

    (1995)
  • S. Firore et al.

    Group dynamics and shared mental model development

  • B. Glaser et al.

    Remodeling grounded theory

    Forum Qualitative Sozialforschung/Forum: Qualitative Social Research in Nursing & Health

    (2004)
  • B.G. Glaser

    Theoretical Sensitivity

    (1978)
  • B.G. Glaser

    Basics of Grounded Theory Analysis

    (1992)
  • B.G. Glaser

    Doing Grounded Theory: Issues and Discussions

    (1998)
  • B.G. Glaser et al.

    The Discovery of Grounded Theory: Strategies for Qualitative Research

    (1967)
  • D.E. Harter et al.

    Effects of process maturity on quality, cycle time, and effort in software product development

    Management Science

    (2000)
  • Cited by (77)

    • Adopting DevOps in the real world: A theory, a model, and a case study

      2019, Journal of Systems and Software
      Citation Excerpt :

      Open coding lasted until there was no doubt about the core category of the study. Similar to that described by Adolph et al. (2012), we started considering a core category candidate and changed later. The first core category candidate was automation, but we realized that this category did not explain most of the behaviors or events in data.

    • Age stereotypes in distributed software development: The impact of culture on age-related performance expectations

      2018, Information and Software Technology
      Citation Excerpt :

      Although valuing diversity has become a catchphrase, field research on the simultaneous impact of diversity in age and national culture on stereotypes is scarce. In general, information systems (IS) research primarily focuses on processes and technology-related aspects of software development (SD: [1]), despite the fact that performance in software development teams is strongly influenced by human and social factors [4,73,91]. In fact, team work processes and resistance from groups or individuals are two of the main factors for human-related failures in modern software development [23].

    • Professional Empowerment in the Software Industry through Experience-Driven Shared Tacit Knowledge: A Case Study from China

      2023, Professional Empowerment in the Software Industry through Experience-Driven Shared Tacit Knowledge: A Case Study from China
    View all citing articles on Scopus

    Steve Adolph is a PhD candidate in Electrical and Computer Engineering at the University of British Columbia and an agile coach with Rally Software. His research interests include software process engineering, requirements management, and software architecture.

    Philippe Kruchten is a professor of software engineering at the University of British Columbia, and NSERC Chair in Design Engineering. He led the development of the Rational Unified Process.

    Wendy Hall is a nursing professor in the School of Nursing at the University of British Columbia. Her research interests include research methods and parent child development.

    View full text