Skip to main content
SpringerLink
Log in

Upstream bug management in Linux distributions

An empirical study of Debian and Fedora practices

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

A Linux distribution consists of thousands of packages that are either developed by in-house developers (in-house packages) or by external projects (upstream packages). Leveraging upstream packages speeds up development and improves productivity, yet bugs might slip through into the packaged code and end up propagating into downstream Linux distributions. Maintainers, who integrate upstream projects into their distribution, typically lack the expertise of the upstream projects. Hence, they could try either to propagate the bug report upstream and wait for a fix, or fix the bug locally and maintain the fix until it is incorporated upstream. Both of these outcomes come at a cost, yet, to the best of our knowledge, no prior work has conducted an in-depth analysis of upstream bug management in the Linux ecosystem. Hence, this paper empirically studies how high-severity bugs are fixed in upstream packages for two Linux distributions, i.e., Debian and Fedora. Our results show that 13.9% of the upstream package bugs are explicitly reported being fixed by upstream, and 13.3% being fixed by the distribution, while the vast majority of bugs do not have explicit information about this in Debian. When focusing on the 27.2% with explicit information, our results also indicate that upstream fixed bugs make users wait for a longer time to get fixes and require more additional information compared to fixing upstream bugs locally by the distribution. Finally, we observe that the number of bug comment links to reference information (e.g., design docs, bug reports) of the distribution itself and the similarity score between upstream and distribution bug reports are important factors for the likelihood of a bug being fixed upstream. Our findings strengthen the need for traceability tools on bug fixes of upstream packages between upstream and distributions in order to find upstream fixes easier and lower the cost of upstream bug management locally.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Notes

  1. Unless noted otherwise, the remainder of this paper refers to high-severity bugs as “bugs”.

  2. https://distrowatch.com/dwres.php?resource=popularity

  3. https://getfedora.org/en/sponsors/

  4. https://www.redhat.com/en/blog/what-open-source-upstream

  5. https://github.com/SAILResearch/suppmaterial-20-justina-upstream_bug_linux

  6. https://packages.debian.org/buster/gedit

  7. https://dep-team.pages.debian.net/deps/dep3/

  8. https://salsa.debian.org/debian/openssl/-/commit/96db9023b881d7cd9f379b0c154650d6c108e9a3

  9. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=743883

  10. https://salsa.debian.org/debian/openssl/-/commit/2f21d2da895a465e151f4cb8f100040b897b5c1c

  11. https://www.debian.org/releases/

  12. https://fedoraproject.org/wiki/Fedora_Release_Life_Cycle

  13. https://bugzilla.redhat.com/show_bug.cgi?id=1150444

  14. https://bugzilla.gnome.org/show_bug.cgi?id=737456

  15. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=586924

  16. https://www.redhat.com/en/blog/what-open-source-upstream

  17. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=489553

  18. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=424456

  19. https://www.rdocumentation.org/packages/Hmisc/versions/4.2-0/topics/redun

  20. https://www.ntia.gov/SBOM

  21. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=931234

  22. https://bugzilla.redhat.com/show_bug.cgi?id=772257

  23. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=731132

  24. https://salsa.debian.org/ssh-team/openssh/-/commits/master

  25. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=851052

  26. https://distrowatch.com/index.php?dataspan=2020

References

  • Debian - bug triage wiki (online). https://wiki.debian.org/BugTriage. Last accessed: 2020-12-01

  • Debian - information regarding the bug processing system for package maintainers and bug triagers (online). https://www.debian.org/Bugs/Developer. Last accessed: 2020-12-01

  • Debian - list of package categories (online). https://packages.debian.org/stable/. Last accessed: 2020-12-01

  • Debian - managing packages (online). https://www.debian.org/doc/manuals/developers-reference/pkgs.html#recording-changes-in-the-package. Last accessed: 2020-12-01

  • Debian - managing packages (online). https://www.debian.org/doc/manuals/developers-reference/pkgs.html#upload-bugfix,. Last accessed: 2020-12-01

  • Debian - quilt for debian maintainers (online). https://perl-team.pages.debian.net/howto/quilt.html. Last accessed: 2020-12-01

  • Debian developer’s reference (online). https://www.debian.org/doc/manuals/developers-reference/index.en.html. Last accessed: 2020-12-01

  • Debian releases (online). https://wiki.debian.org/DebianReleases. Last accessed: 2020-12-01

  • Fedora - anitya - upstream release monitoring system (online). https://release-monitoring.org/. Last accessed: 2020-12-01

  • Fedora - bug status workflow (online). https://fedoraproject.org/wiki/BugZappers/BugStatusWorkFlow. Last accessed: 2020-12-01

  • Fedora - creating a patch (online). https://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/Documentation_Guide/sect-workflow-patching.html. Last accessed: 2020-12-01

  • Fedora - fedora engineering steering committee (online). https://docs.fedoraproject.org/en-US/fesco/. Last accessed: 2020-12-01

  • Fedora - join the package collection maintainers (online). https://fedoraproject.org/wiki/Join_the_package_collection_maintainers#Create_a_Fedora_Account. Last accessed: 2020-12-01

  • Fedora - upstream release monitoring (online). https://fedoraproject.org/wiki/Upstream_release_monitoring. Last accessed: 2020-12-01

  • Fedora historical schedules (online). https://fedoraproject.org/wiki/Releases/HistoricalSchedules. Last accessed: 2020-12-01

  • Fedora packaging guidelines (online). https://docs.fedoraproject.org/en-US/packaging-guidelines/#changelogs. Last accessed: 2020-12-01

  • Staying close to upstream projects (online). https://fedoraproject.org/wiki/Staying_close_to_upstream_projects. Last accessed: 2020-12-01

  • Ubuntu - adopt an upstream (online). https://wiki.ubuntu.com/Upstream/Adopt. Last accessed: 2020-12-01

  • Upstream guide (online). https://wiki.debian.org/UpstreamGuide. Last accessed: 2020-12-01

  • Upstream guidelines for linux distributions (online). https://www.freedesktop.org/wiki/Distributions/Packaging/WhyUpstream/. Last accessed: 2020-12-01

  • Adams B, Kavanagh R, Hassan AE, German DM (2016) An empirical study of integration activities in distributions of open source software. Empirical Software Engineering (EMSE’16) 21(3):960–1001

    Article  Google Scholar 

  • Alipour A, Hindle A, Stroulia E (2013) A contextual approach towards more accurate duplicate bug report detection. In: 2013 10th Working Conference on Mining Software Repositories (MSR’13). IEEE, pp 183–192

  • Anbalagan P, Vouk M (2009) On predicting the time taken to correct bug reports in open source projects. In: 2009 IEEE international conference on software maintenance (ICSM’09). IEEE, pp 523–526

  • Barr E T, Harman M, Jia Y, Marginean A, Petke J (2015) Automated software transplantation. In: Proceedings of the 2015 international symposium on software testing and analysis (ISSTA’15), pp 257–269

  • Bettenburg N, Premraj R, Zimmermann T, Kim S (2008) Duplicate bug reports considered harmful... really?. In: 2008 IEEE international conference on software maintenance (ICSM’08). IEEE, pp 337–345

  • Bhattacharya P, Neamtiu I (2011) Bug-fix time prediction models: can we do better?. In: proceedings of the 8th working conference on mining software repositories (MSR’11), pp 207–210

  • Boisselle V, Adams B (2015) The impact of cross-distribution bug duplicates, empirical study on debian and ubuntu. In: 2015 IEEE 15th international working conference on source code analysis and manipulation (SCAM’15). IEEE, pp 131–140

  • Canfora G, Cerulo L, Cimitile M, Di Penta M (2011) Social interactions around cross-system bug fixings: the case of freebsd and openbsd. In: Proceedings of the 8th working conference on mining software repositories (MSR’11), pp 143–152

  • Canfora G, Di Sorbo A, Forootani S, Pirozzi A, Visaggio C A (2020) Investigating the vulnerability fixing process in oss projects: Peculiarities and challenges. Computers & Security 99:102067

    Article  Google Scholar 

  • Claes M, Mens T, Di Cosmo R, Vouillon J (2015) A historical analysis of debian package incompatibilities. In: 2015 IEEE/ACM 12th working conference on mining software repositories (MSR’15). IEEE, pp 212–223

  • Crowston K, Scozzi B (2008) Bug fixing practices within free/libre open source software development teams. Journal of Database Management (JDM’08) 19(2):1–30

    Article  Google Scholar 

  • da Costa D A, McIntosh S, Treude C, Kulesza U, Hassan A E (2018) The impact of rapid release cycles on the integration delay of fixed issues. Empirical Software Engineering (EMSE’18) 23(2):835–904

    Article  Google Scholar 

  • Davies J, Zhang H, Nussbaum L, German DM (2010) Perspectives on bugs in the debian bug tracking system. In: 2010 7th IEEE working conference on mining software repositories (MSR’10). IEEE, pp 86–89

  • Ding H, Ma W, Chen L, Zhou Y, Xu B (2017) An empirical study on downstream workarounds for cross-project bugs. In: 2017 24th Asia-Pacific software engineering conference. IEEE, pp 318–327

  • Duc A N, Cruzes D S, Ayala C, Conradi R (2011) Impact of stakeholder type and collaboration on issue resolution time in oss projects. In: IFIP international conference on open source systems. Springer, pp 1–16

  • Fan Y, Xia X, Lo D, Hassan A E (2018) Chaff from the wheat: Characterizing and determining valid bug reports. IEEE Transactions on Software Engineering (TSE’18) 46(5):495–525

    Article  Google Scholar 

  • Giger E, Pinzger M, Gall H (2010) Predicting the fix time of bugs. In: Proceedings of the 2nd international workshop on recommendation systems for software engineering, pp 52–56

  • Guo P J, Zimmermann T, Nagappan N, Murphy B (2010) Characterizing and predicting which bugs get fixed: an empirical study of microsoft windows. In: Proceedings of the 32Nd ACM/IEEE international conference on software engineering-Volume 1 (ICSE’10), pp 495–504

  • Harrell Jr FE, Lee KL, Califf RM, Pryor DB, Rosati RA (1984) Regression modelling strategies for improved prognostic prediction. Statistics in Medicine 3(2):143–152

    Article  Google Scholar 

  • Hauge O, Ayala C, Conradi R (2010) Adoption of open source software in software-intensive organizations–a systematic literature review. Inf Softw Technol 52(11):1133–1154

    Article  Google Scholar 

  • Herraiz I, Shihab E, Nguyen Thanh HD, Hassan A E (2011) Impact of installation counts on perceived quality: A case study on debian. In: 2011 18th working conference on reverse engineering. IEEE, pp 219–228

  • Hooimeijer P, Weimer W (2007) Modeling bug report quality. In: Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering (ASE’07), pp 34–43

  • Hu H, Zhang H, Xuan J, Sun W (2014) Effective bug triage based on historical bug-fix information. In: 2014 IEEE 25th international symposium on software reliability engineering. IEEE, pp 122–132

  • Huelsenbeck J P, Crandall K A (1997) Phylogeny estimation and hypothesis testing using maximum likelihood. Annual Review of Ecology and Systematics 28(1):437–466

    Article  Google Scholar 

  • Jeong G, Kim S, Zimmermann T (2009) Improving bug triage with bug tossing graphs. In: Proceedings of the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering (FSE’09), pp 111–120

  • Just S, Premraj R, Zimmermann T (2008) Towards the next generation of bug tracking systems. In: 2008 IEEE symposium on visual languages and human-centric computing. IEEE, pp 82–85

  • Kim S, Whitehead Jr EJ (2006) How long did it take to fix bugs?. In: Proceedings of the 2006 international workshop on Mining software repositories (MSR’06), pp 173–174

  • Lamkanfi A, Demeyer S, Giger E, Goethals B (2010) Predicting the severity of a reported bug. In: 2010 7th IEEE working conference on mining software repositories (MSR’10). IEEE, pp 1–10

  • Lee D, Rajbahadur G K, Lin D, Sayagh M, Bezemer C-P, Hassan A E (2020) An empirical study of the characteristics of popular minecraft mods. Empirical Software Engineering (EMSE’20) 25(5):3396–3429

    Article  Google Scholar 

  • Li J, Conradi R, Slyngstad O P N, Bunse C, Khan U, Torchiano M, Morisio M (2005) An empirical study on off-the-shelf component usage in industrial projects. In: International conference on product focused software process improvement. Springer, pp 54–68

  • Ling C X, Huang J, Zhang H (2003) Auc: a better measure than accuracy in comparing learning algorithms. In: Conference of the canadian society for computational studies of intelligence. Springer, pp 329–341

  • Ma W, Chen L, Zhang X, Feng Y, Xu Z, Chen Z, Zhou Y, Xu B (2020) Impact analysis of cross-project bugs on software ecosystems. In: Proceedings of the ACM/IEEE 42nd international conference on software engineering (ICSE’20), pp 100–111

  • Ma W, Chen L, Zhang X, Zhou Y, Xu B (2017) How do developers fix cross-project correlated bugs? a case study on the github scientific python ecosystem. In: 2017 IEEE/ACM 39th international conference on software engineering (ICSE’17). IEEE, pp 381–392

  • Marks L, Zou Y, Hassan A E (2011) Studying the fix-time for bugs in large open source projects. In: Proceedings of the 7th international conference on predictive models in software engineering (PROMISE’11), pp 1–8

  • McIntosh S, Kamei Y, Adams B, Hassan A E (2016) An empirical study of the impact of modern code review practices on software quality. Empirical Software Engineering (EMSE’16) 21(5):2146–2189

    Article  Google Scholar 

  • Menzies T, Marcus A (2008) Automated severity assessment of software defect reports. In: 2008 IEEE international conference on software maintenance (ICSM’08). IEEE, pp 346–355

  • Merilinna J, Matinlassi M (2006) State of the art and practice of opensource component integration. In: 32nd EUROMICRO conference on software engineering and advanced applications (EUROMICRO’06). IEEE, pp 170–177

  • Mockus A, Fielding R T, Herbsleb J D (2002) Two case studies of open source software development: Apache and mozilla. ACM Transactions on Software Engineering and Methodology (TOSEM’02) 11(3):309–346

    Article  Google Scholar 

  • Ohira M, Hassan A E, Osawa N, Matsumoto K- (2012) The impact of bug management patterns on bug fixing: A case study of eclipse projects. In: 2012 28th IEEE international conference on software maintenance (ICSM’12). IEEE, pp 264–273

  • Panichella S, Bavota G, Di Penta M, Canfora G, Antoniol G (2014) How developers’ collaborations identified from different sources tell us about code changes. In: 2014 IEEE international conference on software maintenance and evolution (ICSME’14). IEEE, pp 251–260

  • Rajbahadur GK, Wang S, Ansaldi G, Kamei Y, Hassan AE (2021) The impact of feature importance methods on the interpretation of defect classifiers. IEEE Transactions on Software Engineering (TSE’21)

  • Ray B, Kim M (2012) A case study of cross-system porting in forked projects. In: Proceedings of the ACM SIGSOFT 20th international symposium on the foundations of software engineering (FSE’12), pp 1–11

  • Ray B, Kim M, Person S, Rungta N (2013) Detecting and characterizing semantic inconsistencies in ported code. In: 2013 28th IEEE/ACM international conference on automated software engineering (ASE’13). IEEE, pp 367–377

  • Shihab E, Ihara A, Kamei Y, Ibrahim W M, Ohira M, Adams B, Hassan A E, Matsumoto K- (2013) Studying re-opened bugs in open source software. Empirical Software Engineering (EMSE’13) 18(5):1005–1042

    Article  Google Scholar 

  • Stol K-J, Babar M A, Avgeriou P, Fitzgerald B (2011) A comparative study of challenges in integrating open source software and inner source software. Inf Softw Technol 53(12):1319–1336

    Article  Google Scholar 

  • Storey M-A, Zagalsky A, Figueira Filho F, Singer L, German D M (2016) How social and communication channels shape and challenge a participatory culture in software development. IEEE Transactions on Software Engineering (TSE’16) 43(2):185–204

    Article  Google Scholar 

  • Thongtanunam P, Hassan AE (2020) Review dynamics and their impact on software quality. IEEE Transactions on Software Engineering (TSE’20)

  • Thongtanunam P, McIntosh S, Hassan A E, Iida H (2016) Revisiting code ownership and its relationship with software quality in the scope of modern code review. In: Proceedings of the 38th international conference on software engineering (ICSE’16), pp 1039–1050

  • Van Den Berk I, Jansen S, Luinenburg L (2010) Software ecosystems: a software ecosystem strategy assessment model. In: Proceedings of the Fourth European conference on software architecture: companion volume, pp 127–134

  • Weimer W (2006) Patches as better bug reports. In: Proceedings of the 5th international conference on Generative programming and component engineering (GPCE’06), pp 181–190

  • Weiss C, Premraj R, Zimmermann T, Zeller A (2007) How long will it take to fix this bug?. In: Fourth international workshop on mining software repositories (MSR’07). IEEE, pp 1–1

  • Xia X, Lo D, Wen M, Shihab E, Zhou B (2014) An empirical study of bug report field reassignment. In: 2014 software evolution Week-IEEE conference on software maintenance, reengineering, and reverse engineering (CSMR-WCRE’14). IEEE, pp 174–183

  • Xuan J, Jiang H, Ren Z, Zou W (2012) Developer prioritization in bug repositories. In: 2012 34th international conference on software engineering (ICSE’12). IEEE, pp 25–35

  • Zhang F, Khomh F, Zou Y, Hassan A E (2012) An empirical study on factors impacting bug fixing time. In: 2012 19th working conference on reverse engineering. IEEE, pp 225–234

  • Zhang H, Gong L, Versteeg S (2013) Predicting bug-fixing time: an empirical study of commercial software projects. In: 2013 35th international conference on software engineering (ICSE’13). IEEE, pp 1042–1051

  • Zhang Y, Yu Y, Wang H, Vasilescu B, Filkov V (2018) Within-ecosystem issue linking: a large-scale study of rails. In: Proceedings of the 7th international workshop on software mining, pp 12–19

  • Zhou B, Neamtiu I, Gupta R (2015) A cross-platform analysis of bugs and bug-fixing in open source projects: Desktop vs. android vs. ios. In: Proceedings of the 19th international conference on evaluation and assessment in software engineering (EASE’15), pp 1–10

  • Zimmermann T, Premraj R, Bettenburg N, Just S, Schroter A, Weiss C (2010) What makes a good bug report?. IEEE Transactions on Software Engineering (TSE’10) 36(5):618–643

    Article  Google Scholar 

Download references

Acknowledgments

We would like to thank the Debian and Fedora maintainers that graciously provided us feedback. Furthermore, special thanks to Rahul Bajaj and the anonymous reviewers for their insightful comments. The findings and opinions in this paper belong solely to the authors, and are not necessarily those of Huawei. Moreover, our results do not in any way reflect the quality of Huawei software products.

Author information

Authors and Affiliations

  1. Software Analysis and Intelligence Lab (SAIL), Queen’s University, Kingston, Ontario, Canada

    Jiahuei Lin, Bram Adams & Ahmed E. Hassan

  2. Centre for Software Excellence at Huawei Canada, Kingston, ON, Canada

    Haoxiang Zhang

Authors
  1. Jiahuei Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haoxiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bram Adams
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ahmed E. Hassan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haoxiang Zhang.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Communicated by: Walid Maalej

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Table 5 shows the heuristics derived from the naming convention of packages in Debian and their associated categories.

Table 5 The heuristics and their categories
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, J., Zhang, H., Adams, B. et al. Upstream bug management in Linux distributions. Empir Software Eng 27, 134 (2022). https://doi.org/10.1007/s10664-022-10173-y

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10664-022-10173-y

Keywords

Search

Navigation