Skip to main content
SpringerLink
Log in

An empirical investigation into merge conflicts and their effect on software quality

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Merge conflicts are known to cause extra effort for developers, but little is known about their effect on software. While some research has been done, many questions remain. To better understand merge conflicts and their impact we performed an empirical study about the types, frequency, and impact of merge conflicts, where impact is measured in terms of bug fixing commits associated with conflicts. We analyzed 143 open source projects and found that almost 1 in 5 merges cause conflicts. In 75.23% of these cases, a developer needed to reflect on the program logic to resolve it. We also found that the code associated with a merge conflict is twice as likely to have a bug. When the code associated with merge conflicts require manual intervention, the code is 26× more likely to have a bug.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Listing 1
Listing 2
Listing 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. Some projects may have migrated to GitHub from other platforms, so this is a lower-bound figure

References

  • Accioly P, Borba P, Cavalcanti G (2018) Understanding semi-structured merge conflict characteristics in open-source java projects. Empir Softw Eng 23(4):2051–2085. https://doi.org/10.1007/s10664-017-9586-1

    Article  Google Scholar 

  • Ahmed I, Gopinath R, Brindescu C, Groce A, Jensen C (2016) Can testedness be effectively measured?. In: Proceedings of the 2016 24th ACM SIGSOFT international symposium on foundations of software engineering, FSE 2016. ACM, Seattle, pp 547–558, https://doi.org/10.1145/2950290.2950324

  • Ahmed I, Brindescu C, Mannan UA, Jensen C, Sarma A (2017) An empirical examination of the relationship between code smells and merge conflicts. In: Proceedings of the 11th ACM/IEEE international symposium on empirical software engineering and measurement, ESEM ’17. IEEE Press, Piscataway, pp 58–67, https://doi.org/10.1109/ESEM.2017.12

  • Apache Maven (2018) Maven. The Apache Software Foundation. https://maven.apache.org/. Accessed 1 May 2018

  • Apel S, Liebig J, Brandl B, Lengauer C, Kästner C (2011) Semistructured merge: rethinking merge in revision control systems. In: Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on foundations of software engineering, ESEC/FSE ’11. ACM, New York, pp 190–200, https://doi.org/10.1145/2025113.2025141

  • Apel S, Lessenich O, Lengauer C (2012) Structured Merge with auto-tuning: balancing precision and performance. In: Proceedings of the 27th IEEE/ACM international conference on automated software engineering, ASE 2012. ACM, Essen, pp 120–129, https://doi.org/10.1145/2351676.2351694

  • Biehl JT, Czerwinski M, Smith G, Robertson GG (2007) FASTDash: a visual dashboard for fostering awareness in software teams. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’07. ACM, New York, pp 1313–1322, https://doi.org/10.1145/1240624.1240823

  • Bird C, Bachmann A, Aune E, Duffy J, Bernstein A, Filkov V, Devanbu P (2009) Fair and balanced?: bias in bug-fix datasets. In: Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering, pp 121–130

  • Boehm BW, Brown JR, Lipow M (1976) Quantitative evaluation of software quality. In: Proceedings of the 2nd international conference on software engineering, ICSE ’76. IEEE Computer Society Press, Los Alamitos, pp 592–605

  • Brun Y, Holmes R, Ernst MD, Notkin D (2011) Proactive detection of collaboration conflicts. In: Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on foundations of software engineering, ESEC/FSE ’11. ACM, Szeged, pp 168–178, https://doi.org/10.1145/2025113.2025139

  • Brun Y, Holmes R, Ernst MD, Notkin D (2013) Early detection of collaboration conflicts and risks. IEEE Trans Softw Eng 39(10):1358–1375. https://doi.org/10.1109/TSE.2013.28

    Article  Google Scholar 

  • Buckley J, Mens T, Zenger M, Rashid A, Kniesel G (2005) Towards a taxonomy of software change. J Softw Maint Evol Res Pract 17(5):309–332. https://doi.org/10.1002/smr.319

    Article  Google Scholar 

  • Canfora G, Cerulo L, Penta MD (2007) Identifying changed source code lines from version repositories. In: Fourth international workshop on mining software repositories (MSR’07:ICSE Workshops 2007), pp 14–14, https://doi.org/10.1109/MSR.2007.14

  • Cataldo M, Herbsleb JD (2013) Coordination breakdowns and their impact on development productivity and software failures. IEEE Trans Softw Eng 39(3):343–360. https://doi.org/10.1109/TSE.2012.32

    Article  Google Scholar 

  • Cavalcanti G, Borba P, Accioly P (2017) Evaluating and improving semistructured merge. Proc ACM Program Lang 1(OOPSLA):59:1–59:27. https://doi.org/10.1145/3133883

    Article  Google Scholar 

  • Cohen J, Cohen P, West S, Aiken L (2013) Applied multiple regression/correlation analysis for the behavioral sciences. Taylor & Francis

  • Companion website (2016) http://caius.brindescu.com/research/mc-classification

  • Corbet J, Kroah-Hartman G (2016) 2016 Linux Kernel Development Report. Tech. rep., The Linux Foundation

  • Costa C, Figueiredo JJC, Ghiotto G, Murta LGP (2014) Characterizing the problem of developers’ assignment for merging branches. Int J Softw Eng Knowl Eng 24(10):1489–1508. https://doi.org/10.1142/S0218194014400166

    Article  Google Scholar 

  • Cruzes DS, Dyba T (2011) Recommended steps for thematic synthesis in software engineering. In: 2011 international symposium on empirical software engineering and measurement, pp 275–284, https://doi.org/10.1109/ESEM.2011.36

  • da Silva IA, Chen PH, Van der Westhuizen C, Ripley RM, van der Hoek A (2006) Lighthouse: coordination through emerging design. In: Proceedings of the 2006 OOPSLA workshop on Eclipse Technology eXchange, eclipse ’06. ACM, New York, pp 11–15, https://doi.org/10.1145/1188835.1188838

  • de Souza CRB, Redmiles D, Dourish P (2003) “Breaking the code”, Moving between private and public work in collaborative software development. In: Proceedings of the 2003 International ACM SIGGROUP Conference on Supporting Group Work, GROUP ’03. ACM, New York, pp 105–114, https://doi.org/10.1145/958160.958177

  • De Souza LBL, Maia MDA (2013) Do software categories impact coupling metrics?. In: Proceedings of the 10th working conference on mining software repositories, MSR ’13. IEEE Press, Piscataway, pp 217–220

  • Dewan P, Hegde R (2007) Semi-synchronous conflict detection and resolution in asynchronous software development. In: Bannon LJ, Wagner I, Gutwin C, Harper RHR, Schmidt K (eds) ECSCW 2007. Springer, London, Limerick, pp 159–178, https://doi.org/10.1007/978-1-84800-031-5_9

  • El Emam K, Benlarbi S, Goel N, Rai SN (2001) The confounding effect of class size on the validity of object-oriented metrics. IEEE Trans Softw Eng 27 (7):630–650. https://doi.org/10.1109/32.935855

    Article  Google Scholar 

  • Falleri JR, Morandat F, Blanc X, Martinez M, Montperrus M (2014) Fine-grained and accurate source code differencing. In: Proceedings of the 29th ACM/IEEE international conference on automated software engineering, ASE ’14. ACM, New York, pp 313–324, https://doi.org/10.1145/2642937.2642982

  • GitHub (2017) https://www.github.com/

  • Godfrey MW, Zou L (2005) Using origin analysis to detect merging and splitting of source code entities. IEEE Trans Softw Eng 31(2):166–181. https://doi.org/10.1109/TSE.2005.28

    Article  Google Scholar 

  • Gorton I, Liu A (2002) Software component quality assessment in practice: successes and practical impediments. In: Proceedings of the 24th international conference on software engineering, ICSE ’02. ACM, New York, pp 555–558, https://doi.org/10.1145/581339.581408

  • Grinter RE (1995) Using a configuration management tool to coordinate software development. In: Proceedings of conference on organizational computing systems, COCS ’95. ACM, New York, pp 168–177, https://doi.org/10.1145/224019.224036

  • Groce A, Zhang C, Eide E, Chen Y, Regehr J (2012) Swarm Testing. In: Proceedings of the 2012 international symposium on software testing and analysis, ISSTA 2012. ACM, Minneapolis, pp 78–88, https://doi.org/10.1145/2338965.2336763

  • Guimarães ML, Silva AR (2012) Improving early detection of software merge conflicts. In: Proceedings of the 34th international conference on software engineering, ICSE ’12. IEEE Press, Zurich, pp 342–352

  • Hensher D, Stopher P (1979) Behavioural travel modelling, Croom Helm, London

  • Holzmann GJ, Joshi R, Groce A (2011) Swarm verification techniques. IEEE Trans Softw Eng 37(6):845–857. https://doi.org/10.1109/TSE.2010.110

    Article  Google Scholar 

  • Johnson EJ, Goldstein D (2003) Do defaults save lives? Science 302(5649):1338–1339. https://doi.org/10.1126/science.1091721

  • Johnson EJ, Bellman S, Lohse GL (2002) Defaults, framing and privacy: why opting in-opting out1. Mark Lett 13(1):5–15. https://doi.org/10.1023/A:1015044207315

    Article  Google Scholar 

  • Kalliamvakou E, Gousios G, Blincoe K, Singer L, German DM, Damian D (2014) The promises and perils of mining GitHub. In: Proceedings of the 11th working conference on mining software repositories, MSR 2014. ACM, New York, pp 92–101, https://doi.org/10.1145/2597073.2597074

  • Kasi BK, Sarma A (2013) Cassandra: proactive conflict minimization through optimized task scheduling. In: Proceedings of the 2013 international conference on software engineering, ICSE ’13. IEEE Press, Piscataway, pp 732–741

  • Kim S, Zimmermann T, Pan K, Whitehead EJJ (2006) Automatic identification of bug-introducing changes. In: Proceedings of the 21st IEEE/ACM international conference on automated software engineering, ASE ’06. IEEE Computer Society, Washington, DC, pp 81–90, https://doi.org/10.1109/ASE.2006.23

  • Kim S, Whitehead EJ Jr, Zhang Y (2008) Classifying software changes: clean or buggy? IEEE Trans Softw Eng 34(2):181–196. https://doi.org/10.1109/TSE.2007.70773

    Article  Google Scholar 

  • Lippe E, van Oosterom N (1992) Operation-based merging. In: Proceedings of the fifth ACM SIGSOFT symposium on software development environments, SDE 5. ACM, New York, pp 78–87, https://doi.org/10.1145/142868.143753

  • Marinescu R (2001) Detecting design flaws via metrics in object-oriented systems. In: Proceedings of the 39th international conference and exhibition on technology of object-oriented languages and systems (TOOLS39), TOOLS ’01. IEEE Computer Society, Washington, DC, p 173

  • Marinescu R (2004) Detection strategies: metrics-based rules for detecting design flaws. In: 20th IEEE international conference on software maintenance, 2004. Proceedings, pp 350–359

  • McKee S, Nelson N, Sarma A, Dig D (2017) Software practitioner perspectives on merge conflicts and resolutions. In: 2017 IEEE international conference on software maintenance and evolution, ICSME 2017, Shanghai, China, September 17–22, 2017, pp 467–478, https://doi.org/10.1109/ICSME.2017.53

  • Menezes GGLD (2016) On the nature of software merge conflicts. Ph.D. thesis, Universidade Federal Fluminense

  • Mens T (2002) A state-of-the-art survey on software merging. IEEE Trans Softw Eng 28(5):449–462. https://doi.org/10.1109/TSE.2002.1000449

    Article  Google Scholar 

  • Mockus A, Weiss DM (2000) Predicting risk of software changes. Bell Labs Tech J 5(2):169–180. https://doi.org/10.1002/bltj.2229

    Article  Google Scholar 

  • Mockus A, Fielding RT, Herbsleb JD (2002) Two case studies of open source software development: Apache and Mozilla. ACM Trans Softw Eng Methodol 11 (3):309–346. https://doi.org/10.1145/567793.567795

    Article  Google Scholar 

  • Nieminen A (2012) Real-time collaborative resolving of merge conflicts. In: Proceedings of the 2012 8th international conference on collaborative computing: networking, applications and worksharing (CollaborateCom 2012), COLLABORATECOM ’12. IEEE Computer Society, Washington, DC, pp 540–543, https://doi.org/10.4108/icst.collaboratecom.2012.250435

  • Parsons J, Saunders C (2004) Cognitive heuristics in software engineering applying and extending anchoring and adjustment to artifact reuse. IEEE Trans Softw Eng 30 (12):873–888

    Article  Google Scholar 

  • Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12:2825–2830

    MathSciNet  MATH  Google Scholar 

  • Perry DE, Siy HP, Votta LG (2001) Parallel changes in large-scale software development: an observational case study. ACM Trans Softw Eng Methodol 10(3):308–337. https://doi.org/10.1145/383876.383878

    Article  Google Scholar 

  • Raymond ES (1999) The Cathedral and the bazaar, 1st edn. O’Reilly & Associates, Inc., Sebastopol

    Google Scholar 

  • Ripley RM, Yasui RY, Sarma A, van der Hoek A (2004) Workspace awareness in application development. In: Proceedings of the 2004 OOPSLA workshop on Eclipse Technology eXchange, eclipse ’04. ACM, New York, pp 17–21, https://doi.org/10.1145/1066129.1066133

  • Samuelson W, Zeckhauser R (1988) Status quo bias in decision making. J Risk Uncertain 1(1):7–59

    Article  Google Scholar 

  • Sarma A, Noroozi Z, Hoek AVD (2003) Palantír: raising awareness among configuration management workspaces. In: Proceedings of the 25th international conference on software engineering, May 3-10, 2003, Portland, Oregon, USA, pp 444–454. https://doi.org/10.1109/ICSE.2003.1201222

  • Servant F, Jones JA, van der Hoek A (2010) CASI: preventing indirect conflicts through a live visualization. In: Proceedings of the 2010 ICSE workshop on cooperative and human aspects of software engineering, CHASE ’10. ACM, New York, pp 39–46, https://doi.org/10.1145/1833310.1833317

  • Shihab E, Bird C, Zimmermann T (2012) The effect of branching strategies on software quality. In: Proceedings of the ACM-IEEE international symposium on empirical software engineering and measurement, ESEM ’12. ACM, New York, pp 301–310, https://doi.org/10.1145/2372251.2372305

  • Sun Z, Song Q, Zhu X (2012) Using coding-based ensemble learning to improve software defect prediction. IEEE Trans Syst Man Cybern Part C Appl Rev 42 (6):1806–1817

    Article  Google Scholar 

  • Sunstein CR, Thaler RH (2003) Libertarian paternalism is not an oxymoron. The University of Chicago Law Review, pp 1159–1202

  • Tian Y, Lawall J, Lo D (2012) Identifying linux bug fixing patches. In: 2012 34th international conference on software engineering (ICSE), pp 386–396

  • Understand ™ (2017) Static Code Analysis Tool

  • Weyuker EJ, Ostrand TJ, Bell RM (2008) Do too many cooks spoil the broth? Using the number of developers to enhance defect prediction models. Empir Softw Eng 13(5):539–559. https://doi.org/10.1007/s10664-008-9082-8

    Article  Google Scholar 

  • Zimmermann T, Kim S, Zeller A, Whitehead EJ Jr (2006) Mining version archives for co-changed lines. In: Proceedings of the 2006 international workshop on mining software repositories, MSR ’06. ACM, New York, pp 72–75, https://doi.org/10.1145/1137983.1138001

Download references

Acknowledgements

We would like to thank the Software Engineering and HCI research groups at Oregon State University for their input and feedback on this project. This work was in part made possible through support from the National Science Foundation (Grants IIS:1559657, CCF:1253786), and IBM.

Author information

Authors and Affiliations

  1. Oregon State University, Corvallis, OR, 97331, USA

    Caius Brindescu, Carlos Jensen & Anita Sarma

  2. University of California, Irvine, Irvine, CA, 92697, USA

    Iftekhar Ahmed

Authors
  1. Caius Brindescu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Iftekhar Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Carlos Jensen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anita Sarma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Caius Brindescu.

Additional information

Communicated by: Sven Apel

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Brindescu, C., Ahmed, I., Jensen, C. et al. An empirical investigation into merge conflicts and their effect on software quality. Empir Software Eng 25, 562–590 (2020). https://doi.org/10.1007/s10664-019-09735-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-019-09735-4

Keywords

Search

Navigation