Reconciling perspectives: A grounded theory of how people manage the process of software development
Highlights
► A grounded theory of how people manage the process of software development. ► Perspectives mismatches between individuals and groups impede the process. ► Reconciling Perspectives is a four-stage process for removing impediments. ► Early stages depends on the ability of individuals to engage and interact with others. ► Later stages require moderation of interaction to focus on getting the job done.
Introduction
Software development is a risky and expensive proposition. Talking with software developers quickly illustrates the quagmire that software development can become. Late delivery, defective products, cost overruns, and frustrated staff and stake holders are some of the debris resulting from a failed or less than successful project. This is expensive debris considering that global software development is a 1.6 trillion US dollar industry (Bartels et al., 2006). Improving the productivity of software development teams and the quality of delivered software will result in significant economic returns.
Numerous studies have demonstrated that individual abilities and team social factors are significant cost drivers for software engineering projects, often swamping all other factors (Boehm, 1984, Cockburn, 2002, Cockburn and Highsmith, 2001, Curtis et al., 1987, Jones, 2000, Lister and DeMarco, 1987, Sawyer and Guinan, 1998). Caper Jones (2000) data demonstrate that high levels of management and staff experience contribute 120% to productivity while effective methods and processes contribute only 35%. Cost drivers associated with personal factors from the COCOMO model “reflect the strong influence of personal capability on software productivity” (Boehm et al., 1995, p. 86). Earlier, Boehm concluded:
“Personnel attributes and human relations activities provide by far the largest source of opportunity for improving software productivity” (Boehm, 1984)
If social factors are the biggest cost drivers, and explain variance in the productivity of software development teams, research studies that help us identify and understand social processes in software engineering should yield significant benefit to the industry. In the past, most software engineering research has focused on tools and production methods, which have limited ability to account for and manage the variance in software projects (Glass et al., 2002, Sawyer and Guinan, 1998, Shaw, 2003, Sjoeberg et al., 2005, Zannier et al., 2006). Shaw's (2003) analysis of papers accepted by the International Conference on Software Engineering (ICSE) shows the majority of accepted papers to be
“…reports [of] an improved method or means of developing software that is, of designing, implementing, evolving, maintaining, or otherwise operating on the software system itself. Papers addressing these questions dominate both the submitted and the accepted papers” (Shaw, 2003, p. 727).
In the past, procedures or techniques and tools and notation accounted for the majority of the accepted papers (69%), while less than 10% of the accepted papers described empirical or qualitative models. Another survey of the software engineering research literature (Glass et al., 2002) reported a very small percentage (in many cases less than 1%) of research papers explored organizational issues or employed research methods useful in understanding social behavior.
The software engineering research agenda is changing, with more researchers investigating the influence that personal attributes and human relationships have on software projects (Dittrich et al., 2007). The Agile Manifesto and the Agile software development movement placed a spotlight on the importance of social interactions with the first article of the Manifesto for Agile Software Development:
“Individuals and interactions over processes and tools”
The emphasis on individuals and interactions motivated research into social interaction (Chong, 2005, Hoda et al., 2010, Moe et al., 2008, Robinson and Sharp, 2005, Whitworth and Biddle, 2007).
However, we believe the state of software engineering research can still be summed up in Curtis et al.’s (1987) retelling of the 13th century story of a man who, after losing his keys, crosses the street to search under a lamppost because the light is much better there. This should not come as a surprise to us because we are engineers and not sociologists. Engineers—especially at the research level—are engineers because they enjoy solving technical problems. Developing tools and methods is what we are educated to do, and what we enjoy doing; however, if we are to better understand the factors that have the greatest influence on software development performance, our studies must also examine social processes.
This paper is an extended revision of our Agile 2011 conference paper (Adolph and Kruchten, 2011). For this research project, we had the opportunity to study a number of Agile software development teams in order to understand the variability in the productivity of software development by creating a substantive theory of how people actually do, or “manage” as we have referred to it, the process of software development. Our study makes two contributions: the first is our findings, and the second is our experience using grounded theory (Glaser and Strauss, 1967). In this paper, the conceptual elements of the theory are highlighted with an underlined italicized font; for example, the conceptual element Bunkering is highlighted as Bunkering. In Section 2 we present an outline of our study and a description of our use of the grounded theory method. In Section 3 we present our theory, and in Section 4 we provide a discussion of our results within the context of existing theories. Section 5 summarizes our results and recommendations.
Section snippets
Motivation
The study we conducted was aimed at understanding the social processes that influence software team performance and the effects of software methods on those processes. Methodologists argue the benefits of following a software development methodology, and there are studies that demonstrate a positive correlation between software development method adoption and team effectiveness in terms of product quality and productivity (Diaz and Sligo, 1997, Harter et al., 2000). On the other hand, industry
Results
Software development takes place in the context of a diverse organizational Eco-system where an Acquirer requests a Supplier to perform a Job for them. To perform a Job, the Supplier and Acquirer must agree on their expectations of the Job, including the Work Products that will be delivered as part of the Job and the schedule for their delivery. The Supplier creates the Work Products for the Job and delivers them to the Acquirer for acceptance. The diversity of the organizational Eco-system,
Discussion
This discussion of Reconciling Perspectives compares it to other relevant grounded theories and extant theory to both situate Reconciling Perspectives within the existing body of knowledge and to generate policy recommendations. The Anticipation-eXecution-Expectation (AxE) model (Hsieh et al., 2006) and Organizing Self Organizing Teams (Hoda et al., 2010) are two grounded theories that illuminate some properties of Reconciling Perspectives. The AxE model emerged out of a study of coordination
Summary
Our study explains how people manage the process of developing software using the process of Reconciling Perspectives to remove impediments created by Perspective Mismatches that prevent Getting the Job Done. The process breaks down when:
- •
Individuals and teams go dark (McCarthy and Gilbert, 1995) and the process simply does not start because the number of Communications is so low or intermittent that individuals are unable to reflect on the Feedback they are hearing in the communications and do
Acknowledgements
The authors wish to express their thanks to the AgileAlliance, the Scrum Alliance, and the Eclipse Foundation for their support of this research. Our further thanks go to our reviewers, and especially the anonymous reviewer who introduced us to Boland's and Tenkasi's paper.
Steve Adolph is a PhD candidate in Electrical and Computer Engineering at the University of British Columbia and an agile coach with Rally Software. His research interests include software process engineering, requirements management, and software architecture.
References (60)
- et al.
For the special issue on qualitative software engineering research
Information and Software Technology
(2007) An empirical investigation into the adoption of systems development methodologies
Information & Management
(1998)- et al.
Research in software engineering: an analysis of the literature
Information and Software Technology
(2002) - et al.
Using grounded theory to study the experience of software development
Empirical Software Engineering
(2011) - Adolph, S., Kruchten, P., 2011. Reconciling perspectives: how people manage the process of software development. Paper...
Manifesto for Agile Software Development
(2001)Principles of Agile Software Development
(2001)- et al.
US Slowdown in 2007 will Dampen the $1.6 Trillion Global IT Market
(2006) Common pitfalls in published grounded theory research
Qualitative Health Research
(1993)Software engineering economics
IEEE Transaction on Software Engineering
(1984)
Cost models for future software life cycle processes: COCOMO 2.0
Annals of Software Engineering
Perspective making and perspective taking in communities of knowing
Organization Science
The Management of Innovation
Shared mental models in expert team decision making
Agile software development joins the would-be crowd
Cutter IT Journal
People and Methodologies in Software Development
Agile software development, the people factor
Computer
Representational gaps, information processing, and conflict in functionally diverse teams
Academy of Management Review
How software process improvement helped Motorola
IEEE Software
Writing Ethnographic Fieldnotes
Group dynamics and shared mental model development
Remodeling grounded theory
Forum Qualitative Sozialforschung/Forum: Qualitative Social Research in Nursing & Health
Theoretical Sensitivity
Basics of Grounded Theory Analysis
Doing Grounded Theory: Issues and Discussions
The Discovery of Grounded Theory: Strategies for Qualitative Research
Effects of process maturity on quality, cycle time, and effort in software product development
Management Science
Cited by (77)
A study of human resource digital transformation (HRDT): A phenomenon of innovation capability led by digital and individual factors
2023, Journal of Business ResearchAdopting DevOps in the real world: A theory, a model, and a case study
2019, Journal of Systems and SoftwareCitation Excerpt :Open coding lasted until there was no doubt about the core category of the study. Similar to that described by Adolph et al. (2012), we started considering a core category candidate and changed later. The first core category candidate was automation, but we realized that this category did not explain most of the behaviors or events in data.
Age stereotypes in distributed software development: The impact of culture on age-related performance expectations
2018, Information and Software TechnologyCitation Excerpt :Although valuing diversity has become a catchphrase, field research on the simultaneous impact of diversity in age and national culture on stereotypes is scarce. In general, information systems (IS) research primarily focuses on processes and technology-related aspects of software development (SD: [1]), despite the fact that performance in software development teams is strongly influenced by human and social factors [4,73,91]. In fact, team work processes and resistance from groups or individuals are two of the main factors for human-related failures in modern software development [23].
A New Way of Cataloging Research through Grounded Theory
2023, Applied Sciences (Switzerland)Narrowing the communication gap in internationally distributed teams: the case of software-development teams in Sri Lanka and Japan
2023, Asian Business and ManagementProfessional Empowerment in the Software Industry through Experience-Driven Shared Tacit Knowledge: A Case Study from China
2023, Professional Empowerment in the Software Industry through Experience-Driven Shared Tacit Knowledge: A Case Study from China
Steve Adolph is a PhD candidate in Electrical and Computer Engineering at the University of British Columbia and an agile coach with Rally Software. His research interests include software process engineering, requirements management, and software architecture.
Philippe Kruchten is a professor of software engineering at the University of British Columbia, and NSERC Chair in Design Engineering. He led the development of the Rational Unified Process.
Wendy Hall is a nursing professor in the School of Nursing at the University of British Columbia. Her research interests include research methods and parent child development.