Skip to main content
SpringerLink
Log in

Understanding cost drivers of software evolution: a quantitative and qualitative investigation of change effort in two evolving software systems

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Making changes to software systems can prove costly and it remains a challenge to understand the factors that affect the costs of software evolution. This study sought to identify such factors by investigating the effort expended by developers to perform 336 change tasks in two different software organizations. We quantitatively analyzed data from version control systems and change trackers to identify factors that correlated with change effort. In-depth interviews with the developers about a subset of the change tasks further refined the analysis. Two central quantitative results found that dispersion of changed code and volatility of the requirements for the change task correlated with change effort. The analysis of the qualitative interviews pointed to two important, underlying cost drivers: Difficulties in comprehending dispersed code and difficulties in anticipating side effects of changes. This study demonstrates a novel method for combining qualitative and quantitative analysis to assess cost drivers of software evolution. Given our findings, we propose improvements to practices and development tools to manage and reduce the costs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Arisholm E (2006) Empirical assessment of the impact of structural properties on the changeability of object-oriented software. Inf Softw Technol 48(11):1046–1055

    Article  Google Scholar 

  • Arisholm E, Sjøberg DIK (2004) Evaluating the effect of a delegated versus centralized control style on the maintainability of object-oriented software. IEEE Trans Softw Eng 30(8):521–534

    Article  Google Scholar 

  • Arisholm E, Briand LC, Føyen A (2004) Dynamic coupling measurement for object-oriented software. IEEE Trans Softw Eng 30(8):491–506

    Article  Google Scholar 

  • Atkins DL, Ball T, Graves TL, Mockus A (2002) Using version control data to evaluate the impact of software tools: A case study of the version editor. IEEE Trans Softw Eng 28(7):625–637

    Article  Google Scholar 

  • Atlassian, Jira bug and issue tracker. Available at: http://www.atlassian.com/software/jira/. Accessed Sep 26 2008

  • Banker RD, Datar SM, Kemerer CF, Zweig D (1993) Software complexity and maintenance costs. Commun ACM 36(11):81–94

    Article  Google Scholar 

  • Beck K (1999) Embracing change with extreme programming. Computer 32(10):70–77

    Article  Google Scholar 

  • Belady LA, Lehman MM (1976) A model of large program development. IBM Syst J 15(3):225–252

    Article  MATH  Google Scholar 

  • Benestad HC (2008) Technical report 12-2008: assessing the reliability of developers’ classification of change tasks: A field experiment, Simula Research Laboratory

  • Benestad HC, Arisholm E, Sjøberg D (2005) How to recruit professionals as subjects in software engineering experiments. Information Systems Research in Scandinavia (IRIS), Kristiansand, Norway

  • Benestad HC, Anda BC, Arisholm E (2008) Technical report 10-2008: A systematic review of empirical software engineering studies that analyze individual changes, Simula Research Laboratory

  • Benestad HC, Anda B, Arisholm E (2009) Technical report 02-2009: an investigation of change effort in two evolving software systems, Simula Research Laboratory

  • Bhatt P, Shroff G, Anantaram C, Misra AK (2006) An influence model for factors in outsourced software maintenance. J Softw Maint Evol Res Pract 18(6):385–423

    Article  Google Scholar 

  • Birk A, Dingsøyr T, Stålhane T (2002) Postmortem: Never leave a project without it. IEEE Softw 19(3):43–45

    Article  Google Scholar 

  • Briand LC, Basili VR (1992) A classification procedure for the effective management of changes during the maintenance process. In: Proceedings of 1992 Conference on Software Maintenance. IEEE Computer Society: Los Alamitos CA, pp 328–336

  • Briand LC, Wüst J (2001a) The impact of design properties on development cost in object-oriented systems. IEEE Trans Softw Eng 27(11):963–986

    Article  Google Scholar 

  • Briand LC, Wüst J (2001b) Integrating scenario-based and measurement-based software product assessment. J Syst Softw 59(1):3–22

    Article  Google Scholar 

  • Briand LC, Wüst J (2002) Empirical studies of quality models in object-oriented systems. Adv Comput 59(1):97–166

    Google Scholar 

  • Briand LC, Labiche Y, Miao Y (2003) Towards the reverse engineering of UML sequence diagrams. In: Proceedings of 10th Working Conference on Reverse Engineering, WCRE 2003, pp 57–66

  • Chillarege R, Bhandari IS, Chaar JK, Halliday MJ, Moebus DS, Ray BK, Wong MY (1992) Orthogonal defect classification-a concept for in-process measurements. Softw Eng IEEE Trans 18(11):943–956

    Article  Google Scholar 

  • Christensen R (1996) Principal component regression. In: Analysis of variance, design and regression. 446–451

  • Cohn M (2006) Agile estimating and planning. Pearson Education, Inc., Boston, MA

    Google Scholar 

  • Conte SD, Dunsmore HE, Shen VY (1986) Software engineering metrics and models. Benjamin-Cummings Publishing Co., Inc., Redwood City, CA, USA

    Google Scholar 

  • Curtis B, Sheppard SB, Kruesi-Bailey E, Bailey J, Boehm-Davis DA (1989) Experimental evaluation of software documentation formats. J Syst Softw 9(2):167–207

    Article  Google Scholar 

  • DeMarco T, Lister T (1985) Programmer performance and the effects of the workplace. In: Proceedings of Proceedings of the 8th international conference on Software engineering, pp 268–272

  • Derby E, Larsen D (2006) Agile retrospectives: Making good teams great. Pragmatic Bookshelf, Raleigh, NC

    Google Scholar 

  • Détienne F, Bott F (2002a) Influence of the task. In: Software design—cognitive aspects. Springer-Verlag, London.pp 105–110

  • Détienne F, Bott F (2002b) Software design—cognitive aspects. Springer-Verlag, London

    MATH  Google Scholar 

  • Dzidek WJ, Arisholm E, Briand LC (2008) A realistic empirical evaluation of the costs and benefits of UML in software maintenance. IEEE Trans Softw Eng 34(3):407–432

    Article  Google Scholar 

  • Eick SG, Graves TL, Karr AF, Marron JS, Mockus A (2001) Does code decay? Assessing the evidence from change management data. IEEE Trans Softw Eng 27(1):1–12

    Article  Google Scholar 

  • Etzkorn L, Bansiya J, Davis C (1999) Design and code complexity metrics for OO classes. J Object-Oriented Program 12(1):35–40

    Google Scholar 

  • Evanco WM (1999) Analyzing change effort in software during development. In: Proceedings of 6th International Symposium on Software Metrics (METRICS99), pp 179–188

  • Evanco WM (2001) Prediction models for software fault correction effort. In: Proceedings of 5th European Conference on Software Maintenance and Reengineering, IEEE Computer Society Press: Los Alamitos CA, pp 114–120

  • Fenton N (1994) Software measurement: A necessary scientific basis. IEEE Trans Softw Eng 20(3):199–205

    Article  Google Scholar 

  • Fluri B, Gall HC (2006) Classifying change types for qualifying change couplings. In: Proceedings of 14th International Conference on Program Comprehension (ICPC), Athens, Greece, pp 35–45

  • Gamma E, Helm R, Johnson R, Vlissides J (1995) Design patterns: Elements of reusable object-oriented software. Addison-Wesley

  • Geppert B, Mockus A, Rößler F (2005) Refactoring for changeability: A way to go? In: Proceedings of 11th International Symposium on Software Metrics, IEEE Computer Society Press: Los Alamitos CA

  • GNU, Concurrent Versions System. Available at: http://www.nongnu.org/cvs/. Accessed Sep 26 2008

  • Graves TL, Mockus A (1998) Inferring change effort from configuration management databases. In: Proceedings of 5th International Symposium on Software Metrics, pp 267–273

  • Graves TL, Karr AF, Marron JS, Siy H (2000) Predicting fault incidence using software change history. IEEE Trans Softw Eng 26(7):653–661

    Article  Google Scholar 

  • Hayes JH, Patel SC&Zhao L (2004) A metrics-based software maintenance effort model. In: Proceedings of 8th European Conference on Software Maintenance and Reengineering, IEEE Computer Society Press: Los Alamitos CA, pp 254–258

  • Hunt JW, McIlroy MD (1975) An algorithm for differential file comparison. Computing Science Technical Report 41, Bell Laboratories

  • IBM, Rational ClearCase LT. Available at: http://www-306.ibm.com/software/awdtools/clearcase/cclt/. Accessed Sep 26 2008

  • Jolliffe IT (2002) Principal component analysis. Springer-Verlag, New York

    MATH  Google Scholar 

  • Jørgensen M (1995a) An empirical study of software maintenance tasks. J Softw Maint Res Pract 7(1):27–48

    Article  Google Scholar 

  • Jørgensen M (1995b) Experience with the accuracy of software maintenance task effort prediction models. IEEE Trans Softw Eng 21(8):674–681

    Article  Google Scholar 

  • Kemerer C (1995) Software complexity and software maintenance: A survey of empirical research. Ann Softw Eng 1(1):1–22

    Article  Google Scholar 

  • Kitchenham BA, Pleeger SL, Pickard LM, Jones PW, Hoaglin DC, El Emam K, Rosenberg J (2002) Preliminary guidelines for empirical research in software engineering. IEEE Trans Softw Eng 12(4):1106–1125

    Google Scholar 

  • Kitchenham BA, Dybå T, Jørgensen M (2004) Evidence-based software engineering. In: Proceedings of 26th International Conference on Software Engineering (ICSE), Edinburgh, Scotland, IEEE Computer Society, pp 273–281

  • Koenemann J, Robertson SP (1991) Expert problem solving strategies for program comprehension. In: Proceedings of SIGCHI conference on Human factors in computing systems: Reaching through technology, pp 125–130

  • Krishnan MS, Kriebel CH, Kekre S, Mukhopadhyay T (2000) An empirical analysis of productivity and quality in software products. Manag Sci 46(6):745–759

    Article  Google Scholar 

  • Lientz BP (1983) Issues in software maintenance. ACM Comput Surv 15(3):271–278

    Article  Google Scholar 

  • Miller A (2002) Generating all subsets. In: Subset selection in regression. 48–52

  • Mockus A (2000) Missing data in software engineering. In: Guide to advanced empirical software engineering. 185–200

  • Mockus A, Weiss DM (2000) Predicting risk of software changes. Bell Labs Tech J 5(2):169–180

    Article  Google Scholar 

  • Moløkken-Østvold K, Haugen NC, Benestad HC (2008) Using planning poker for combining expert estimates in software projects. Accepted for publication in Journal of Systems and Software

  • Munson JC, Elbaum SG (1998) Code churn: A measure for estimating the impact of code change. In: Proceedings of 14th International Conference on Software Maintenance, IEEE Computer Society Press: Los Alamitos CA, pp 24–31

  • Myers RH, Montgomery DC, Vining GG (2001) The generalized linear model. In: Generalized linear models with applications in engineering and the sciences. Wiley Series in Probability and Statistics, 4–6

  • Niessink F, van Vliet H (1997) Predicting maintenance effort with function points. In: Proceedings of 1997 International Conference on Software Maintenance, IEEE Computer Society Press: Los Alamitos CA, pp 32–39

  • Niessink F, van Vliet H (1998) Two case studies in measuring software maintenance effort. In: Proceedings of 14th International Conference on Software Maintenance, IEEE Computer Society Press: Los Alamitos CA, pp 76–85

  • NSB, Norwegian State Railways. Available at: http://www.nsb.no/about_nsb/. Accessed Sep 26 2008

  • Ott RL, Longnecker M (2001) Inferences in multiple regression. In: Statistical methods and data analysis. Duxbury, 646–657

  • Perry DE, Porter AA, Votta LG (2000) Empirical studies of software engineering: A roadmap. In: Proceedings of Conference on The Future of Software Engineering, pp 345–355

  • Pinches GE, Mingo KA (1973) A multivariate analysis of industrial bond ratings. J Finance 28(1):1–18

    Article  Google Scholar 

  • Polo M, Piattini M, Ruiz F (2001) Using code metrics to predict maintenance of legacy programs: A case study. In: Proceedings of 2001 International Conference on Software Maintenance, IEEE Computer Society Press: Los Alamitos CA, pp 202–208

  • Purushothaman R, Perry DE (2005) Toward understanding the rhetoric of small source code changes. IEEE Trans Softw Eng 31(6):511–526

    Article  Google Scholar 

  • Rajaraman C, Lyu MR (1992) Reliability and maintainability related software coupling metrics in C++ programs. In: Proceedings of Third International Symposium on Software Reliability Engineering, pp 303–311

  • RCN, Research Council of Norway, My RCN Web. Available at: https://www.forskningsradet.no/mittNettstedWeb/common/security/login.jsp?setLocale=en. Accessed Sep 26 2008

  • Reformat M, Wu V (2003) Analysis of software maintenance data using multi-technique approach. In: Proceedings of 15th International Conference on Tools with Artificial Intelligence, IEEE Computer Society Press: Los Alamitos CA, pp 53–59

  • Sackman H, Erikson WJ, Grant EE (1968) Exploratory experimental studies comparing online and offline programming performance. Commun ACM 11(1):3–11

    Article  Google Scholar 

  • Schneidewind NF (2001) Investigation of the risk to software reliability and maintainability of requirements changes. In: Proceedings of 2001 International Conference on Software Maintenance, IEEE Computer Society Press: Los Alamitos CA, pp 127–136

  • Seaman CB (1999) Qualitative methods in empirical studies of software engineering. IEEE Trans Softw Eng 25(4):557–572

    Article  Google Scholar 

  • Shin M, Goel AL (2000) Empirical data modeling in software engineering using radial basis functions. IEEE Trans Softw Eng 26(6):567–576

    Article  Google Scholar 

  • Soloway E, Pinto J, Letovsky S (1988) Designing documentation to compensate for delocalized plans. Commun ACM 31(11):1259–1267

    Article  Google Scholar 

  • Stone M (1974) Cross-validatory choice and assessment of statistical predictions. J Roy Stat Soc 36(2):111–133

    MATH  Google Scholar 

  • Swanson EB (1976) The dimensions of maintenance. In: Proceedings of 2nd International Conference on Software Engineering, San Francisco, California, United States, IEEE Computer Society Press: Los Alamitos CA, pp 492–497

  • TPTP, Eclipse Test&Performance Tools Platform Project. Available at: http://www.eclipse.org/tptp/home/documents/tutorials/profilingtool/profilingexample_32.html. Accessed Feb 02 2009

  • von Mayrhauser A, Vans AM (1995) Program comprehension during software maintenance and evolution. Computer 28(8):44–55

    Article  Google Scholar 

  • Woods D, Transana—Qualitative analysis software for video and audio data. Developed at the University of Wisconsin-Madison Center for Education Research. Available at: http://www.transana.org/. Accessed Sep 26 2008

  • Xu B, Yang M, Liang H, Zhu H (2005) Maximizing customer satisfaction in maintenance of software product family. In: Proceedings of 18th Canadian Conference on Electrical and Computer Engineering, IEEE Computer Society Press: Los Alamitos CA, pp 1320–1323

  • Yin RK (2003) Designing case studies. In: Case study research: Design and methods. Sage Publications:Thousand Oaks, CA, 19–53

Download references

Acknowledgements

We are indebted to the managers and developers at Esito and Know IT who provided us with high quality empirical data. The research was funded by the Simula School of Research and Innovation.

Author information

Authors and Affiliations

  1. Simula Research Laboratory and University of Oslo, P.O. Box 134, 1325, Lysaker, Norway

    Hans Christian Benestad & Erik Arisholm

  2. University of Oslo, P.O. Box 1080, Blindern, 0316, Oslo, Norway

    Bente Anda

Authors
  1. Hans Christian Benestad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bente Anda
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Erik Arisholm
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hans Christian Benestad.

Additional information

Editor: Arie van Deursen

Appendix A

Appendix A

1.1 Interview Guide

Part 1. (Only in first interview with each developers—Information about the purpose of the research. Agree on procedures, confidentiality voluntariness, audio-recording).

Question: Can you describe your work and your role in the project?

Part 2. Project context (factors intrinsic to the time period covered by the changes under discussion)

How would you describe the project and your work in the last time period? Did any particular event require special focus in the period?

For each change (CR-nnnn, CR-nnnn, CR-nnnn….,)

Part 3. Measurement control (change effort and name of changed components shown to the interviewee)

Are change effort and code changes correctly registered?

Part 4. Change request characteristics (change tracker information shown on screen to support discussion)

Can you describe the change from the viewpoint of the user? Why was the change needed?

Part 5. General cost factors

Can you roughly indicate how the X hours were distributed on different activities?

Part 6. Properties of relevant code (output from windiff showed on screen to support the discussions)

Can you summarize the changes that you made to the components?

What can you say about the code that was relevant for the change? Was it easy or difficult to understand and make changes to the code?

Part 7. Stability

Did you go through several iterations before you reached the final solution? If so, why?

Did anything not go as expected?

How did you proceed to test the change?

Go to Part 3 for next change

Part 8. Concluding remarks

Do you think this interview covered your activities during the last period?

Rights and permissions

Reprints and permissions

About this article

Cite this article

Benestad, H.C., Anda, B. & Arisholm, E. Understanding cost drivers of software evolution: a quantitative and qualitative investigation of change effort in two evolving software systems. Empir Software Eng 15, 166–203 (2010). https://doi.org/10.1007/s10664-009-9118-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-009-9118-8

Keywords

Search

Navigation